Do Coding Agents Catch Their Own Bugs? (They Don't.)
cs.SE
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/swe-agent/mini-swe-agent
Terminology
Sources
- SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale
- MAS-SZZ: Multi-Agentic SZZ Algorithm for Vulnerability-Inducing Commit Identification
- SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration
- SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarios
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All
- daVinci-Agency: Unlocking Long-Horizon Agency Data-Efficiently
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance
- ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities
- LLMs Get Lost In Multi-Turn Conversation
- SWE-Chain: Benchmarking Coding Agents on Chained Release-Level Package Upgrades
- When Prompts Go Wrong: Evaluating Code Model Robustness to Ambiguous, Contradictory, and Incomplete Task Descriptions
- Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports
- WEAVE: Weaving Real-World Code Evolution into Dependency-Structured Tasks for Benchmarking and Training Code Agents
- CodeFlowLM: Incremental Just-In-Time Defect Prediction with Pretrained Language Models and Exploratory Insights into Defect Localization
- ReDef: Do Code Language Models Truly Understand Code Changes for Just-in-Time Software Defect Prediction?
- SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks
- Is Agent Code Less Maintainable Than Human Code?
Related papers
- ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
- TestPrism: Rethinking Test Evaluation Beyond a Single Reference
- Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- For Coding Agents, the Repository Matters