Submodular Benchmark Selection
cs.AI, cs.LG
Submitted: 2026-05-04
Updated: 2026-11-12
Code: https://github.com/smolix/benchmark-selection
Terminology
Sources
- Program Synthesis with Large Language Models
- Lessons from the Trenches on Reproducible Evaluation of Language Models
- Evaluating Large Language Models Trained on Code
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
- WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
- Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench
- Instruction-Following Evaluation for Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents