FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents
cs.AI, q-fin.CP
Submitted: 2026-09-28
Updated: 2026-09-28
Sources
- FinTradeBench: A Financial Reasoning Benchmark for LLMs
- PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning
- Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks
- TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generation
- Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge
- Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
- Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction
- FinanceBench: A New Benchmark for Financial Question Answering
- Fin-RATE: A Real-world Financial Analytics and Tracking Evaluation Benchmark for LLMs on SEC Filings
- FinRetrieval: A Benchmark for Financial Data Retrieval by AI Agents
- When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysis
- JADE: Expert-Grounded Dynamic Evaluation for Open-Ended Professional Tasks
- OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
- FinResearchBench II: A Deep Research Benchmark with Consensus-Derived Gold Rubrics for Distinguishing Financial Report Quality
- FinanceQA: A Benchmark for Evaluating Financial Analysis Capabilities of Large Language Models
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks
- FinSheet-Bench: From Simple Lookups to Complex Reasoning, Where LLMs Break on Financial Spreadsheets
- PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges
- DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems