EvalResearchBench: Can AI Agents Design Their Own Evaluations?
cs.AI
Submitted: 2026-10-03
Updated: 2026-10-03
Code: https://github.com/openai/codex
Terminology
Sources
- Program Synthesis with Large Language Models
- BenchAgents: Multi-Agent Systems for Structured Benchmark Creation
- Evaluating Large Language Models Trained on Code
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Agent psychometrics: Task-level performance prediction in agentic coding benchmarks
- Gemma 4 Technical Report
- AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
- Measuring Massive Multitask Language Understanding
- MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
- DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
- Dynabench: Rethinking Benchmarking in NLP
- Kimi K3: Open Frontier Intelligence
- Holistic Evaluation of Language Models
- LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
- RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
- RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems