When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs

arXiv:2606.28438 · cs.SE, cs.AI · Submitted 2026-06-26 · Read on arXiv

cs.SE, cs.AI

Submitted: 2026-06-26

Updated: 2026-10-12

Code: https://github.com/Hik289/code-retraining

Terminology

• Benchmark — 2× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 3 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Self-Consuming Generative Models Go MAD
• Program Synthesis with Large Language Models
• Rethinking Code Review Workflows with LLM Assistance: An Empirical Study
• Efficient Training of Language Models to Fill in the Middle
• SantaCoder: don't reach for the stars!
• MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation
• Evaluating Large Language Models Trained on Code
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• A Tale of Tails: Model Collapse as a Change of Scaling Laws
• Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences
• InCoder: A Generative Model for Code Infilling and Synthesis
• A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops
• RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
• Self-Correcting Self-Consuming Loops for Generative Model Training
• Textbooks Are All You Need
• DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models
• Qwen2.5-Coder Technical Report
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• The Stack: 3 TB of permissively licensed source code

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties