Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference
cs.AI, cs.CL, cs.LG
Submitted: 2026-06-04
Updated: 2026-08-29
Comments: Accepted to EMNLP 2026 (Findings). Code: https://github.com/wbopan/retro-harness ; Project website: https://paper-rho.wenbo.io
Project page: https://paper-rho.wenbo.io
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: AI agents rely on a harness of skills, tools, and workflows to solve complex problems.
Terminology
Abstract
AI agents rely on a harness of skills, tools, and workflows to solve complex problems. Continually improving this harness is essential for adapting to new tasks. However, existing optimization methods typically require ground-truth validation sets, yet such labeled data is difficult to acquire in practical deployment settings. To address this problem, we introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimizes the agent harness using only past trajectories. Specifically, RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel. The agent analyzes these rollouts using self-validation and self-consistency, then generates candidate harness updates and selects the most effective one by its own pairwise self-preference. We evaluate RHO across three diverse domains, spanning software engineering, technical work, and knowledge work. Notably, a single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading. Furthermore, our analysis demonstrates that RHO effectively targets prior failure modes. As a result, the optimized harness alters the agent's behavior patterns and sustains higher accuracy during long-horizon sessions.
Sources
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments
- Automated Design of Agentic Systems
- DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
- Meta-Harness: End-to-End Optimization of Model Harnesses
- Sleep-time Compute: Beyond Inference Scaling at Test-time
- Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
- MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution
- WebTrap: Stealthy Mid-Task Hijacking of Browser Agents During Navigation
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- C-Pack: Packed Resources For General Chinese Embeddings
- Large Language Models as Optimizers
- SkillOS: Learning Skill Curation for Self-Evolving Agents
- TextGrad: Automatic "Differentiation" via Text
- ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
- Large Language Models Are Human-Level Prompt Engineers
- MemGPT: Towards LLMs as Operating Systems
- M$^\star$: Every Task Deserves Its Own Memory Harness
- Towards Long-Horizon Interpretability: Efficient and Faithful Multi-Token Attribution for Reasoning LLMs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems