CoTrace: Data Recipes for Training Terminal Agents with Harness-Model Co-Evolution
cs.CL
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: Preprint. 32 pages, 7 figures, 17 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery.
Abstract
Terminal-agent capability depends jointly on model weights and the runtime harness that formats prompts, binds tools, and handles error recovery. Existing harness-model co-evolution approaches improve both components, yet often treat trajectories produced during harness search as an undifferentiated replay buffer. This practice overlooks that a trajectory's value for model training depends on the harness under which it was generated. To systematically analyze this interface, we establish an alternating co-evolution framework that decouples harness search and policy training through component-wise promotion decisions. Within this framework, we introduce CoTrace, a harness-aware data recipe that explicitly governs trajectory routing, provenance matching, and curriculum refresh. Under CoTrace, recurring execution failures guide harness synthesis, while policy training is strictly conditioned on verified rollouts matched to the adopted runtime for supervised fine-tuning (SFT) or fresh online interactions for reinforcement learning (RL). On the Tmax promotion split, CoTrace advances Qwen3.5-9B from 78 to 88 solved tasks under supervised fine-tuning while an online reinforcement variant reaches 90. Specifically, a compact harness-matched corpus produces steady model gains at substantially lower compute than much larger corpora pooled across sibling harnesses. Furthermore, evaluations on Terminal-Bench 2.1 and SWE-bench Lite show that out-of-distribution transfer depends fundamentally on harness compatibility, where maintaining consistency between training and evaluation runtimes prevents procedural execution breakdowns observed under foreign scaffolds.
Sources
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- A Survey on Data Selection for Language Models
- EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
- HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry
- Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
- Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents
- Reinforced Self-Training (ReST) for Language Modeling
- Automated Design of Agentic Systems
- Tmax: A simple recipe for terminal agents
- Prioritized Level Replay
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
- The Interplay of Harness Design and Post-Training in LLM Agents
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- Meta-Harness: End-to-End Optimization of Model Harnesses
- OpenSage: Self-programming Agent Generation Engine
- Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
- Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions
- Quantifying Variance in Evaluation Benchmarks
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering