From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agents
cs.SE, cs.AI
Submitted: 2026-09-28
Updated: 2026-09-28
Comments: 14 pages, 3 figures
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- SWE-Bench+: Enhanced Coding Benchmark for LLMs
- Program Synthesis with Large Language Models
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- Evaluating Large Language Models Trained on Code
- ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
- MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation
- ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation
- LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems
- A Survey on LLM-as-a-Judge
- LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- RealBench: Benchmarking Verilog Generation Models with Real-World IP Designs
- The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
- VerilogEval: Evaluating Large Language Models for Verilog Code Generation
- Benchmarking AI Agents for Addressing Scientific Challenges Across Scales
- Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agents
- RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model
- Towards Understanding Specification Gaming in Reasoning Models
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties