Welfare, Improvability, and Variance: A Principal-Agent Approach to Optimal Benchmark Item Aggregation
cs.LG, cs.GT, econ.TH
Submitted: 2026-05-29
Updated: 2026-11-08
Code: https://github.com/project-numina/aimo-progress-prize
Terminology
Sources
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- The Benchmark Lottery
- Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation
- Strategic Candidacy in Generative AI Arenas
- Training Compute-Optimal Large Language Models
- ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
- Categorizing Variants of Goodhart's Law
- Efficient Benchmarking of Language Models
- BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
- Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
- Proximal Policy Optimization Algorithms
- Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce
- AI Sandbagging: Language Models can Strategically Underperform on Evaluations
- Benchmark Data Contamination of Large Language Models: A Survey
- Don't Make Your LLM an Evaluation Benchmark Cheater
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis