ORACLE-SWE: Quantifying the Contribution of Critical Information Signals on SWE Agents
This work introduces Oracle-SWE, a method to isolate and quantify five critical information signals—Reproduction Test, Regression Test, Edit Location, Execution Context, and API Usage—from software engineering benchmarks. By comparing these oracle signals to agent-extracted ones, the study determines which factors contribute most to task success and provides a roadmap for future research in improving SWE agents.