Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks
cs.AI
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/harbor-framework/terminal-bench-2-1
Project page: https://ekbuchanan.com/terminal-bench
Terminology
Sources
- Evaluating Large Language Models Trained on Code
- Kimi K2.5: Visual Agentic Intelligence
- Do ImageNet Classifiers Generalize to ImageNet?
- Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- ReAct: Synergizing Reasoning and Acting in Language Models
- UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench
- GLM-5: from Vibe Coding to Agentic Engineering
- WebArena: A Realistic Web Environment for Building Autonomous Agents
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents