Better Deck or Different Judge? Evaluating Agentic Harness Gains in Corporate and Investment Banking
cs.AI
Submitted: 2026-09-30
Updated: 2026-09-30
Sources
- Non-Determinism of "Deterministic" LLM Settings
- Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
- A Survey on LLM-as-a-Judge
- FinanceBench: A New Benchmark for Financial Question Answering
- Who Drifted: the System or the Judge? Anytime-Valid Attribution in LLM Evaluation Pipelines
- Quantifying Variance in Evaluation Benchmarks
- Categorizing Variants of Goodhart's Law
- Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
- Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge
- Rethinking the Evaluation of Harness Evolution for Agents
- MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance
- JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems