LiteEvo: Automated, Cost-Efficient Harness Evolution for Generalization to Unseen Tasks
cs.AI
Submitted: 2026-09-27
Updated: 2026-09-27
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: An LLM agent is defined by two things: the weights inside its model and the harness of components assembled around it.
Terminology
Abstract
An LLM agent is defined by two things: the weights inside its model and the harness of components assembled around it. Harnesses are still handcrafted, and HarnessX, which evolves them automatically, starts each benchmark from a handcrafted harness, reports gains on the tasks it evolved on, and budgets 100 to 175 million meta-agent tokens per benchmark. We propose LiteEvo, a lightweight harness-evolution algorithm whose tool-free meta-agents mine agent trajectories for reusable components, curate them into a versioned library, and compose each round's harness from it, starting every benchmark from the same neutral harness and never naming the benchmark. Evolving on the graded tasks of five agentic benchmarks with a frozen Qwen3.5-9B, LiteEvo lifts pass@2 by 10.5 to 67.7pp and reaches comparable or higher pass@2 than a reproduction of HarnessX (71.0 against 67.3 on average) at 13.0 lower mean API cost. Harnesses evolved on train tasks keep their gains on unseen test tasks of four benchmarks, and LiteEvo also lifts Claude Code with Sonnet 4.6 by 1.2 to 71.4pp.
Sources
- Training-Free Group Relative Policy Optimization
- Evaluating Large Language Models Trained on Code
- Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
- Evo-Bench: Can Language Models Improve Agent Harness?
- Meta-Harness: End-to-End Optimization of Model Harnesses
- Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- AutoHarness: improving LLM agents by automatically synthesizing a code harness
- A Self-Improving Coding Agent
- AgentSquare: Automatic LLM Agent Search in Modular Design Space
- Rethinking the Evaluation of Harness Evolution for Agents
- Agent Workflow Memory
- Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- AFlow: Automating Agentic Workflow Generation
- HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems