Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments
Haomin Qi, Xingliang Wang, Xuanqi Gao, Baihui Sang, Xin Zhang, Minghua Ma, Pengfei Gao, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang
cs.SE, cs.CL, cs.LG
Submitted: 2026-07-30
Comments: 15 pages, 7 figures, and 15 tables, including appendices
License: http://creativecommons.org/licenses/by/4.0/
The gist: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation.
Terminology
Abstract
Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification. To expand this supply, we present Change2Task, a system grounded in repository history that converts merged pull requests into verified tasks on healthy modern revisions of the same repository. It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction, and validates the lifecycle from a healthy base to a task state and a restored state. By deriving multiple tasks grounded in developer evidence from maintained environments, Change2Task provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort. We evaluate the system through five common and widely adopted coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface Migration, and Security Repair. Starting from 1,130 source changes eligible for construction, Change2Task achieves 79.6% verified task construction success across these task families. On a matched candidate set, it recovers 29.2% more verified tasks than a construction baseline based on pull requests. Historical and reconstructed cases achieve up to 98.0% matched outcome agreement under agent evaluation, while reuse of modern bases reduces measured expenditure across the complete pipeline by 10.8%.
Sources
- SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments
- Qwen3-Coder-Next Technical Report
- SWE-Universe: Scale Real-World Verifiable Environments to Millions
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- daVinci-Env: Open SWE Environment Synthesis at Scale
- GLM-5: from Vibe Coding to Agentic Engineering
- SWE-Cycle: Benchmarking Code Agents across the Complete Issue Resolution Cycle
- SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios
- Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application
- RepoLaunch: Automating Build and Management of Code Repositories across Languages and Platforms
- TestEvo-Bench: An Executable and Live Benchmark for Test and Code Co-Evolution
- SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
- PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilities
- Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties