Benchmark Health Index: A Systematic Framework for Benchmarking the Benchmarks of LLMs
cs.AI
Submitted: 2026-02-12
Updated: 2026-11-07
Code: https://github.com/SKYLENAGE-AI/benchmark-health-index
Terminology
Sources
- HealthBench: Evaluating Large Language Models Towards Improved Human Health
- Program Synthesis with Large Language Models
- MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- Evaluating Large Language Models Trained on Code
- ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
- On the Measure of Intelligence
- Training Verifiers to Solve Math Word Problems
- The Benchmark Lottery
- FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
- Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmark
- Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts
- Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
- Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos
- PyTorch Image Quality: Metrics for Image Quality Assessment
- On Robustness and Reliability of Benchmark-Based Evaluation of LLMs
- Quantifying Variance in Evaluation Benchmarks
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models
- Humanity's Last Exam
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents