Writing Bug Reports for Software Repair Agents: What Information Matters Most?

arXiv:2607.09553 · cs.SE · Submitted 2026-07-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Writing Bug Reports for Software Repair Agents".

Jane: Here's the gist of the paper. Software development is increasingly moving toward agentic-first workflows This paper studies what makes a bug report agent-ready Our results suggest that agentic-first reports benefit most from information that narrows the agent’s search and repair space Software development workflows are increasingly incorporating LLM-based agents that can inspect repositories, reason over issues, modify code, and propose patches In this setting, issue reports are no longer merely documentation for human maintainers; they become the primary task specification for the agent The practical question is no longer only what makes a bug report useful for humans but also what makes a bug report effective for agents Information Types and Annotation The authors manually classified each issue by change type and annotated each sentence with its information type, such as observed behavior, expected behavior, reproduction steps, localization cues, and suggested fixes They focused on the 441 issues representing bug reports from the SWE-bench Verified benchmark The taxonomy includes bug summary, observed behavior, expected behavior, reproduction steps, environment details, references to external resources (e.g., other issues, Stack Overflow discussions), localization information at different granularity (i.e., lines/functions/files likely affected by the bug), suggested fixes in code or natural language, and current workarounds Empirical Analysis of Agent Success They ran on them mini-swe-agent using three LLM backbones (i.e., GPT-5-mini, MiniMax M2.5, and Gemini 3 Flash) They then fit a binomial regression model to estimate the incremental association between each information type and agent success Their results suggest that agent-readiness is not simply traditional bug-report quality under a new name Ablation Study Findings An ablation study removing selected information types confirms that agents benefit less from information traditionally useful to humans, such as reproduction steps, and more from sentences that expose a repair direction, either through bug localization or a suggested fix The strongest degradation appears when localization cues and suggested fixes are removed together Implications for Agentic Development The implication for agentic-first development is therefore concrete: issue reports should be written less as exhaustive narratives and more as repair specifications that tell the agent what failed, where to start looking, and what kind of change may be needed Future research should explore multi-stage agent architectures in which dedicated localization or issue-understanding agents enrich reports before patch generation The per-LLM analyses show that different backbones do not exploit issue information in the same way Taken together, the observational analysis suggests that agent-ready reports are those that transform a bug report from a descriptive artifact into an actionable repair specification The results provide several lessons for agentic-first development where issue reports are not only read by developers but also serve as the initial task specification for repair agents Agentic workflows should be incremental, not one-shot Localization may need to be automatically derived Issue templates should be treated as agent-specific interfaces References should be summarized and properly contextualized, not merely linked References

Paper summary: Jane: So, the paper sets up this experiment by manually classifying every sentence in those bug reports into different info types. They looked at things like observed behavior, expected behavior, and reproduction steps >

Lu: They also included stuff like environment details and localization cues. The taxonomy they used was pretty detailed: bug summary, observed behavior, expected behavior, reference links to other issues or Stack Overflow discussions >

Meng: And they also included things that are super useful for us developers like suggested fixes in code or natural language and even current workarounds >

Tom: They ran this through a mini SWE-agent system on those four hundred forty-one reports using those three different LLM backbones and fitted a binomial regression model to see how much each information type actually helped the agent succeed > <ref:2607.09553#pg2>

Jane: The key finding is that localization cues, which are mentions of affected lines or functions in the code, were positively associated with successful patches >

Lu: But even stronger was the information about suggested fixes. This held true whether that fix was provided as a code snippet or just written in natural language >

Meng: They did an ablation study where they took out certain pieces of info to see what happened. Removing things like reproduction steps, which are super helpful for humans, didn't help the agent much >

Tom: The strongest drop in success came when they removed localization cues and suggested fixes at the same time >

Jane: So it really suggests that agent-readiness isn't just about having a nicely written report anymore. It’s about structuring the report so it tells the agent exactly where to start looking and what kind of change is needed >

Conclusion: Tom: The paper, 'Writing Bug Reports for Software Repair Agents: What Information Matters Most?', really shows us that we need to stop writing bug reports as long descriptions of a failure and start writing them as repair specifications >

Jane: Exactly. The authors found that reports need to transform from descriptive artifacts into actionable instructions for the agent, telling it what failed, where to begin searching, and what kind of change is expected >

Lu: What this means for the future is that we might need multi-stage agent architectures. We could have one agent dedicated just to understanding the issue and localizing it, which then hands off to another agent that proposes the actual patch >

Meng: From an engineering standpoint, this implies that we shouldn't treat every bug report as a single input prompt. It suggests reports should be incremental, maybe not one-shot fixes from start to finish >

Tom: And they also pointed out that localization cues might need to be derived automatically instead of being manually written in every report >

Jane: They also said that those reference links we use—to other issues or Stack Overflow discussions—should probably be summarized and contextualized properly, not just thrown in there as a link >

Lu: So the lesson here is that these reports are starting to act like interfaces for an agent. The goal is to make them clear enough for the AI to operate without needing a human intermediary step between every single piece of data >

USI Universita della Svizzera italiana University of Sannio

cs.SE

Submitted: 2026-07-10

Updated: 2026-11-10

Code: https://github.com/SWE-agent/mini-swe-agent

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: Software development is increasingly moving toward agentic-first workflows This paper studies what makes a bug report agent-ready Our results suggest that agentic-first reports benefit most from…

Key concepts

Agentic-first workflows
This refers to a new way of software development where AI agents are central to the process. Instead of humans just reading reports, these agents actively inspect code, reason about issues, modify code, and propose solutions based on task specifications provided in bug reports.
Information Types and Annotation
The authors manually categorized sentences in bug reports into specific types like observed behavior or suggested fixes. This detailed labeling allowed them to empirically test which specific pieces of information an agent actually uses to succeed or fail at its repair task.
Ablation Study Findings
This involved removing different information types from the reports and seeing how much the agent's performance dropped. The results showed that removing localization cues and suggested fixes together caused the biggest degradation in agent success, indicating these are crucial for guiding an agent's repair actions.
Repair Specifications
The paper argues that bug reports should evolve from being long descriptions into concise 'repair specifications.' These specifications tell an AI exactly what went wrong, where to focus its search, and what kind of change is needed, making them actionable instructions for the agent.

Terminology

Summary

Software development is increasingly moving toward agentic-first workflows This paper studies what makes a bug report agent-ready Our results suggest that agentic-first reports benefit most from information that narrows the agent’s search and repair space

Introduction to Agentic Issue Reports

Software development workflows are increasingly incorporating LLM-based agents that can inspect repositories, reason over issues, modify code, and propose patches In this setting, issue reports are no longer merely documentation for human maintainers; they become the primary task specification for the agent The practical question is no longer only what makes a bug report useful for humans but also what makes a bug report effective for agents

Information Types and Annotation

The authors manually classified each issue by change type and annotated each sentence with its information type, such as observed behavior, expected behavior, reproduction steps, localization cues, and suggested fixes They focused on the 441 issues representing bug reports from the SWE-bench Verified benchmark The taxonomy includes bug summary, observed behavior, expected behavior, reproduction steps, environment details, references to external resources (e.g., other issues, Stack Overflow discussions), localization information at different granularity (i. p. 0

Where the paper says it

The taxonomy includes bug summary, observed behavior, expected behavior, reproduction steps, environment details, references to external resources (e. , other issues, Stack Overflow discussions), localization information at different granularity (i.

e., lines/functions/files likely affected by the bug), suggested fixes in code or natural language, and current workarounds

Empirical Analysis of Agent Success

They ran on them mini-swe-agent using three LLM backbones (i.e., GPT-5-mini, MiniMax M2. p. 0

Where the paper says it

We focus on the 441 issues representing bug reports, and we run on them mini-swe-agent using three LLM backbones (i. , GPT-5-mini, MiniMax M2.

5, and Gemini 3 Flash) They then fit a binomial regression model to estimate the incremental association between each information type and agent success Their results suggest that agent-readiness is not simply traditional bug-report quality under a new name

Ablation Study Findings

An ablation study removing selected information types confirms that agents benefit less from information traditionally useful to humans, such as reproduction steps, and more from sentences that expose a repair direction, either through bug localization or a suggested fix The strongest degradation appears when localization cues and suggested fixes are removed together

Implications for Agentic Development

The implication for agentic-first development is therefore concrete: issue reports should be written less as exhaustive narratives and more as repair specifications that tell the agent what failed, where to start looking, and what kind of change may be needed Future research should explore multi-stage agent architectures in which dedicated localization or issue-understanding agents enrich reports before patch generation The per-LLM analyses show that different backbones do not exploit issue information in the same way

Conclusion and Lessons Learned

Taken together, the observational analysis suggests that agent-ready reports are those that transform a bug report from a descriptive artifact into an actionable repair specification The results provide several lessons for agentic-first development where issue reports are not only read by developers but also serve as the initial task specification for repair agents Agentic workflows should be incremental, not one-shot Localization may need to be automatically derived Issue templates should be treated as agent-specific interfaces References should be summarized and properly contextualized, not merely linked

References

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Increase agentic workflow iteration based on issue evolution. Issue trackers should support incremental re-execution when new comments add relevant information to turn issue tracking into an iterative human-agent process, where developers and users progressively enrich the issue until it becomes actionable for the agent.

  2. Implement multi-stage agent architectures for localization. A specialized localization component can first identify suspicious code regions using static analysis or dedicated LLM agents before a repair agent generates a patch.

  3. Treat issue templates as interfaces between humans and specific agents. Since different backbones do not exploit issue information in the same way, project owners should design interfaces that adapt to the specific agent used, rather than relying on a single universal template.

  4. Develop dynamic or derived localization capabilities for agents. Since agentic-first development should not rely solely on reporters to identify the bug’s location, systems can benefit from automated localization components that suggest suspicious code regions based on static analysis or test execution results.

  5. Contextualize external references in agent instructions. When reports include references, the system must summarize the repair-relevant information from those references so the agent does not need to infer which external details matter.

Abstract

Software development is increasingly moving toward agentic-first workflows. This includes AI agents responsible for generating initial fixes for submitted issue reports. In this setting, issue reports are no longer merely documentation for human maintainers; they become the primary task specification for the agent. However, little is known about how such reports should be written to maximize the agent's chances of producing a correct fix. We study what makes a bug report agent-ready. Starting from the SWE-Bench Verified benchmark (i.e. a collection of 500 real repository issues with human-written gold patches and test suites for evaluating generated fixes) we manually classify each issue by change type (e.g. bug fix vs refactoring) and annotate each sentence with its information type, such as observed behavior, expected behavior, reproduction steps, localization cues, and suggested fixes. We focus on the 441 issues representing bug reports, and we run on them mini-swe-agent using three LLM backbones (i.e. GPT-5-mini, MiniMax M2.5, and Gemini 3 Flash). We then fit a binomial regression model to estimate the incremental association between each information type and agent success, controlling for confounding factors. Our results suggest that agentic-first reports benefit most from information that narrows the agent's search and repair space. Suggested fixes, expressed either in code or natural language, show the strongest positive associations with pass probability, while localization cues show positive but non-significant associations. An ablation study removing selected information types confirms that agents benefit less from information traditionally useful to humans, such as reproduction steps, and more from sentences that expose a repair direction.

Sources

Related papers