Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?
The PDB framework introduces two new metrics—edit-level precision and bug-level recall—to test how well large language models perform precise debugging. By measuring necessary edits versus resolved bugs, the framework reveals that current frontier models often prioritize superficial fixes or broad rewrites over targeted, minimal code changes.