Scaling Agentic Data for Long-Horizon Terminal Intelligence
cs.CL
Submitted: 2026-10-09
Updated: 2026-10-09
Comments: https://zs1314.github.io/TerminalHorizon/
Code: https://github.com/openai/codex
Project page: https://zs1314.github.io/TerminalHorizon
License: http://creativecommons.org/licenses/by/4.0/
The gist: Terminal agents are moving beyond local operations toward completing real-world professional tasks.
Abstract
Terminal agents are moving beyond local operations toward completing real-world professional tasks. Short trajectories can demonstrate commands and tool calls, but they do not fully show how agents accumulate evidence through exploration, use it to inform judgments, continually adjust strategies based on feedback, and turn intermediate results into final deliverables. High-quality long-horizon trajectories capture these agentic behavioral patterns and their connections within a single problem-solving process, providing richer experience for sustained autonomous work. However, environment synthesis may complete or simplify the key work underlying these patterns in advance and thus remove the need for exploration, implementation, and verification that the agent was meant to perform. To address this, we introduce TerminalHorizon, a fully automated data synthesis engine for long-horizon terminal agents. Starting from real-world professional work, it uses independent execution to identify key work omitted during construction and uses the resulting evidence to guide environment reconstruction. To extend these patterns across stages, it progressively expands tasks under the same goal, introducing new requirements that build on prior results and call for further exploration and improvement. Using this engine, we construct 1.5K task environments and perform supervised fine-tuning of Qwen3.5-27B on the resulting dataset to obtain TerminalHorizon-27B. We achieve substantial performance gains on all 12 benchmarks spanning coding-agent tasks, agentic tasks in real-world production settings, and general reasoning. Multiple RSI case studies, each with a 12-hour research budget, further demonstrate the model's ability to continue experimenting beyond an initial solution, improve its methods based on feedback, and retain effective improvements in its final deliverables.
Sources
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- Terminal-World: Scaling Terminal-Agent Environments via Agent Skills
- Training Verifiers to Solve Math Word Problems
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
- Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
- Toward Scalable Terminal Task Synthesis via Skill Graphs
- VitaBench: Benchmarking LLM Agents with Versatile Interactive Tasks in Real-world Applications
- CLI-Universe: Towards Verifiable Task Synthesis Engine for Terminal Agents
- Tmax: A simple recipe for terminal agents
- Multi-Agent Computer Use
- Recursive Synthesis for Long-Horizon Terminal Tasks
- Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
- Let's Verify Step by Step
- CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion
- LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
- CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- Meta-Task: Turning Terminal Task Synthesis into a Terminal Task for Scalable Agent Training
- On Data Engineering for Scaling LLM Terminal Capabilities
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering