How Hard Is It to Rig a Benchmark? A Social Choice Analysis of Leaderboard Robustness
cs.LG
Submitted: 2026-05-22
Updated: 2026-11-12
Journal ref: 40th Conference on Neural Information Processing Systems (NeurIPS 2026)
License: http://creativecommons.org/licenses/by/4.0/
The gist: Multi-task benchmarks have become a central pillar of machine learning research, yet their growing influence has incentivised benchmark gaming -- strategic actions taken to improve the leaderboard…
Terminology
Abstract
Multi-task benchmarks have become a central pillar of machine learning research, yet their growing influence has incentivised benchmark gaming -- strategic actions taken to improve the leaderboard rank of a specific model. Treating datasets as voters and models as candidates, we consider benchmark-specific training -- the inclusion of benchmark data in training -- as a form of election manipulation. For any ordinal benchmark, the problem of choosing datasets to train on so that a target model becomes top-ranked corresponds to shift bribery, a class of manipulation problems from computational social choice. Leveraging this identification, we show that the benchmark-specific training problem is NP-hard under Borda count and mean win rate. Complementing this worst-case perspective, we introduce the instance-level robustness, the minimum number of datasets a model developer must include in training to top a given leaderboard, and derive expressions for it under arithmetic mean, median, mean win rate and pairwise majority. We evaluate these expressions on MMLU under HELM and on BIG-Bench Hard (BBH) under the Open LLM Leaderboard. Across both suites, mean win rate is hardest to manipulate: this gap is clear on BBH (24 tasks, 4507 models), where its median robustness is 22 tasks (92%), compared with 13 (54%) under arithmetic mean and 12 (50%) under median and pairwise majority.
Sources
- Leaderboard Incentives: Model Rankings under Strategic Post-Training
- Maintaining MTEB: Towards Long Term Usability and Reproducibility of Embedding Benchmarks
- The Benchmark Lottery
- BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validity
- The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models
- Investigating Data Contamination for Pre-training Language Models
- Leakage and the Reproducibility Crisis in ML-based Science
- Evaluating Agents using Social Choice Theory
- A Survey on Large Language Model Benchmarks
- Data Contamination Through the Lens of Time
- The Leaderboard Illusion
- Rethinking Benchmark and Contamination for Language Models with Rephrased Samples
- Don't Make Your LLM an Evaluation Benchmark Cheater
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis