The AI Evaluation Ecosystem
cs.AI, cs.CY
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: 70 pages, 15 figures, 23 tables
Code: https://github.com/aims-foundations/evarium
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Program Synthesis with Large Language Models
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
- On the Opportunities and Risks of Foundation Models
- Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
- SoK: Machine Learning Governance
- Evaluating Large Language Models Trained on Code
- On the Measure of Intelligence
- The Benchmark Lottery
- S$^3$: Social-network Simulation System with Large Language Model-Empowered Agents
- Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting
- Benchmark Inflation: Revealing LLM Performance Gaps Using Retro-Holdouts
- RecSim: A Configurable Simulation Platform for Recommender Systems
- Categorizing Variants of Goodhart's Law
- Artificial Intelligence Index Report 2024
- Artificial Intelligence Index Report 2025
- Model evaluation for extreme risks
- The Leaderboard Illusion
- From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulation
- Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia
- LiveBench: A Challenging, Contamination-Limited LLM Benchmark
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems