EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses
cs.SE, cs.AI
Submitted: 2026-09-30
Updated: 2026-09-30
Code: https://github.com/Virgil-Tan/EngramBench
Terminology
Sources
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- AI Agents That Matter
- Measuring AI Ability to Complete Long Software Tasks
- LLMs Get Lost In Multi-Turn Conversation
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- SkillHone: A Harness for Continual Agent Skill Evolution Through Persistent Decision History
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
- Reflexion: Language Agents with Verbal Reinforcement Learning
- AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties