Coding or Cheating? An Investigation of Frontier Coding LLMs
cs.CL, cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Code: https://github.com/scaleapi/SWE-bench_Proos
Terminology
Sources
- Terminal Wrench: A Dataset of 331 Reward-Hackable Environments and 3,632 Exploit Trajectories
- EvilGenie: A Reward Hacking Benchmark
- DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
- DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks
- R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
- Natural Emergent Misalignment from Reward Hacking in Production RL
- Towards Understanding Specification Gaming in Reasoning Models
- Training Software Engineering Agents and Verifiers with SWE-Gym
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
- SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
- SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data