Re-executing Coding-Agent Evaluations: Unchanged Decisions and Unresolved Policy Risk
cs.SE, cs.CL
Submitted: 2026-10-10
Updated: 2026-10-10
Code: https://github.com/LegoX/Lego-RL
Sources
- SWE-Bench+: Enhanced Coding Benchmark for LLMs
- Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
- Understanding and Detecting Flaky Builds in GitHub Actions
- The Rollout Infrastructure Tax in Coding-Agent Reinforcement Learning
- KAT-Coder-V2.5 Technical Report
- Probe to Generate: Program Variant-Guided Test Augmentation for Repository-Level Repair Benchmarks
- The Implementation Lottery: Auditing Idea Reliability in Automated Research
- Auditing Reward Hackability in Code RL Training Environments
- Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
- Huxley-G\"odel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine
- Auditing Harness Tampering in Self-Improving Agents
- Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR
- SWE-MiniSandbox: Container-Free Reinforcement Learning for Building Software Engineering Agents
- SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
Related papers
- Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
- GitSkills: A Dataset of Agent Skills on GitHub
- SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
- PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
- IntentCoding: Amplifying User Intent in Code Generation
- Incentives and Outcomes in Bug Bounties