E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch
cs.SE, cs.AI
Submitted: 2026-09-29
Updated: 2026-09-29
Code: https://github.com/facebookresearch/E2E-SWE
Terminology
Sources
- MirrorCode: AI can rebuild entire programs from behavior alone
- BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?
- Evaluating Large Language Models Trained on Code
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
- Evaluating LLM-Based 0-to-1 Software Generation in End-to-End CLI Tool Scenarios
- DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
- Kimi K3: Open Frontier Intelligence
- AI-assisted Code Authoring at Scale: Fine-tuning, deploying, and mixed methods evaluation
- Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning
- ProgramBench: Can Language Models Rebuild Programs From Scratch?
- Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving
- RepoZero: Can LLMs Generate a Code Repository from Scratch?
- DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties