How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench

arXiv:2507.02976 · cs.CR, cs.LG, cs.SE · Submitted 2025-06-30 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench".

Elias: Here's the gist of the paper. How Safe Are AI-Generated Patches?

Nadia: A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench is the first large-scale security evaluation of an LLM and three agent-generated patches for real-world issue resolution tasks.

Elias: Our findings reveal that Llama introduces many new vulnerabilities, exhibiting unique patterns not found in developers’ code.

Nadia: Pruning redundant reasoning tokens The study evaluates patches proposed by developers, a standalone LLM (Llama 3.3 Instruct-70B), and three top-performing agentic frameworks (OpenHands, AutoCodeRover, HoneyComb) using 20,000+ GitHub issues.

Paper summary: Nadia: So, this paper is called "How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench". It’s basically the first big security check we’re doing for LLM patches and agent generated fixes for real world issues.

Elias: Right, it looks at developers' patches, a standalone Llama three point three Instruct model, and three agentic frameworks: OpenHands, AutoCodeRover, and HoneyComb <ref:2507.02976#pg1>. The main goal is to figure out when and where these automated program repair systems are most likely to generate vulnerable patches in the first place.

Priya: So, it’s not just about looking at the code itself; they’re checking the code, but also the issues and even how complex the whole project is. That context is important for real world risk.

Nadia: Exactly. And what they found is pretty significant: Llama introduces a lot of brand new vulnerabilities, showing patterns that aren't found in typical developer code at all.

Elias: They looked at over twenty thousand GitHub issues on the SWE-bench dataset to do this analysis <ref:2507.02976#pg1>. They used CodeQL, Semgrep, and Bandit with a majority voting strategy to catch things while still catching a lot of them.

Priya: That’s helpful for us because we need to know what the actual data shows before we trust any claimed result. What did the study actually find about where these risks show up?

Nadia: Well, they looked at three different areas: code characteristics, issue characteristics, and project characteristics. They found that vulnerable patches are most strongly linked to things like a higher number of files that are modified or unique file types being changed in a patch.

Elias: And when it comes to the issues themselves, vulnerabilities often show up in bugs where the issue descriptions lack crucial information like examples of the expected behavior or steps to reproduce. That’s a big signal they found about the context surrounding an issue.

Priya: So, it’s not just that the AI made a mistake in one line; it's that the task itself is poorly described or the resulting patch touches too much of a complex part of the system. That changes how we think about testing these systems.

Paper summary: Nadia: It suggests that assessing risk can’t just be about finding specific vulnerable lines of code anymore. You have to look at both what the AI generates and the broader context where it’s operating, which is something we need to develop complementary risk assessment techniques for.

Elias: They also pointed out some recurring vulnerability types they saw, like Eval Injection from Llama three point three-Generated Code and OS Command Injection from developer-written code <ref:2507.02976#pg1>. These often come from the model taking instructions too literally, like using an eval function or defaulting to insecure functions like MD5.

Priya: That makes sense because those are basic security mistakes that models can make when they don't fully grasp the intent of the prompt. But I wonder how much of that is just model misinterpretation versus actual malicious generation.

Nadia: The paper points out that agentic frameworks like OpenHands produced an order of magnitude more vulnerabilities than AutoCodeRover or HoneyComb did. This difference they attribute to the broad operational scope of the agent, meaning it often touches more files than necessary to fix a single issue.

Elias: That’s a cautionary tale about giving agents too much freedom on large codebases; unconstrained autonomy can significantly increase the risk of vulnerabilities, especially with those large cross file edits.

Priya: So for us, that means we should push for explicit limits on edit scope when we build these agentic workflows, like caps on the size of the diff or limits on how many files an agent can touch in one go.

Nadia: And they also found that higher risk patches in frameworks like AutoCodeRover exhibit abnormally high code complexity. That means when a patch touches really complex functions, there’s a higher chance of an issue showing up.

Elias: They also looked at project level factors like repository size and maintainability index, but those didn't show strong correlations with the total count of vulnerabilities found in the patches. The vulnerability counts seemed more tied to the code and issue level characteristics rather than how big or complex the whole project is.

Paper summary: Priya: That’s interesting because I always thought project size matters a lot for security. So, for this study, it seems like the immediate context around the specific bug report is more important than just looking at the whole repository's structure.

Nadia: Exactly. And they suggest that simple signals, like how widely a patch is spread across the codebase and what kind of information is in an issue description, can be combined into a prediction model that scores risk before and after the AI makes a patch.

Elias: That leads to the idea of integrating this risk assessment directly into CI/CD pipelines as an early warning system, which seems like a promising direction for how we handle these tools moving forward.

Priya: So, while they show us where the risks are most likely to appear, the paper’s main limitation is that it focuses on specific vulnerabilities found in the test sets from SWE-bench. It doesn't necessarily cover every single type of real world issue out there.

Nadia: That makes sense because their methodology relies on that specific dataset. They also flag that using these LLM fixes directly as a replacement for developer written patches without extra safeguards is likely to increase both the frequency and the diversity of vulnerabilities in codebases.

Elias: So, while this study gives us concrete signals about what Llama and agentic systems do well, it’s a warning that we need to build more robust guardrails around their deployment.

Priya: It means for anyone using these LLM based APR tools, integrating security scanning and some form of manual inspection into the workflow when you accept an AI patch is a necessary step.

Nadia: That’s the practical application here: we need to treat LLM generated patches as an additional source of risk and not just another quick fix. We need to figure out how to score that risk before it even hits the developer's screen, right?

Conclusion: Nadia: So we're wrapping up this look at how safe AI generated patches are in program repair tasks on SWE-bench, specifically focusing on Llama and three different agent frameworks like OpenHands, AutoCodeRover, and HoneyComb.

Elias: Yeah Nadia it’s really about testing the security risks when these systems actually try to fix real world code issues across thousands of problems.

Priya: The paper is looking at a lot of data here from twenty thousand plus GitHub issues paired with pull requests to see exactly where these vulnerabilities pop up in the process.

Nadia: They found that Llama can introduce up to eleven times more new vulnerabilities compared to when actual developers write the fixes themselves. That’s a pretty big jump.

Elias: What they really highlight is that these new kinds of risks aren't just random; they show unique patterns you wouldn't see in typical developer code.

Priya: And the study looked at three main areas for where these risks come from: the characteristics of the code itself, the details of the issue description, and things about the project as a whole.

Nadia: They concluded that vulnerable patches are most strongly tied to specific file changes and how much unique code is added or modified in a patch.

Elias: But they also found that issues where developers don't clearly explain what should happen or how to reproduce the bug are often where the AI messes up the most.

Priya: So, it means risk isn't just about finding one bad line of code; it’s about looking at the whole context of how much code is changing and how well the problem is described.

Nadia: Exactly. They suggest we need to build these prediction models that score risk based on those signals before an AI even proposes a fix.

Elias: It points toward integrating security checks right into those continuous delivery pipelines as an early warning system for these AI repairs.

Amirali Sajadi, Kostadin Damevski, Preetha Chatterjee

Drexel University · Virginia Commonwealth University

cs.CR, cs.LG, cs.SE

Submitted: 2025-06-30

Updated: 2026-11-01

Code: https://github.com/SWE-bench/experiments

Project page: https://spotbugs.github.io

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench is the first large-scale security evaluation of an LLM and three…

Key concepts

SWE-bench
This dataset consists of real-world GitHub issues paired with pull requests, providing a large collection of tasks for testing automated program repair systems. It simulates practical software maintenance problems that AI tools attempt to solve.
Agentic Automated Program Repair (APR)
This refers to systems where AI agents, like OpenHands or AutoCodeRover, autonomously propose and implement code changes to fix bugs in software. The study examines the security implications of these automated repair workflows.
Vulnerability Characteristics
The research identifies specific traits in code and issues that correlate with insecure patches. These include a high number of modified files, the presence of certain file types, and missing information within the bug report description as indicators of higher risk.
Risk Assessment Integration
The conclusion advocates for adding security checks directly into development workflows (CI/CD). This involves using simple signals—like patch spread across the codebase or issue detail—to predict and score the risk level of an AI-generated patch before it is accepted.

Terminology

Summary

How Safe Are AI-Generated Patches? p. 0

Where the paper says it

How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench AMIRALI SAJADI, Drexel University, USA KOSTADIN DAMEVSKI, Virginia Commonwealth University, USA PREETHA CHATTERJEE, Drexel University, USA Large language models (LLMs) and their agentic frameworks are increasingly adopted to perform development tasks such as automated program repair (APR).

A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench is the first large-scale security evaluation of an LLM and three agent-generated patches for real-world issue resolution tasks. p. 0
Where the paper says it

A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench AMIRALI SAJADI, Drexel University, USA KOSTADIN DAMEVSKI, Virginia Commonwealth University, USA PREETHA CHATTERJEE, Drexel University, USA Large language models (LLMs) and their agentic frameworks are increasingly adopted to perform development tasks such as automated program repair (APR). While prior work has identified security risks in LLM-generated code, most have focused on synthetic, simplified, or isolated tasks that lack the complexity of real-world program repair.

Our findings reveal that Llama introduces many new vulnerabilities, exhibiting unique patterns not found in developers’ code. p. 0
Where the paper says it

Finally, we analyze a wide range of code, issue, and project-level factors to understand the conditions under which generating insecure patches is more likely. Our findings reveal that Llama introduces many new vulnerabilities, exhibiting unique patterns not found in developers’ code.

Pruning redundant reasoning tokens

The study evaluates patches proposed by developers, a standalone LLM (Llama 3.3 Instruct-70B), and three top-performing agentic frameworks (OpenHands, AutoCodeRover, HoneyComb) using 20,000+ GitHub issues. p. 0

Where the paper says it

We evaluate patches proposed by developers, a standalone LLM (Llama 3. 3 Instruct-70B), and three top-performing agentic frameworks (OpenHands, AutoCodeRover, HoneyComb).

The overarching goal with this work is to identify when and where APR systems are most likely to generate vulnerable patches. p. 1
Where the paper says it

We then examine how code-level, issue-level, and projectlevel factors influence the security of the proposed solutions, identifying conditions that result in vulnerability-inducing AI-generated patches. Our overarching goal with this work is to identify when and where APR systems are most likely to generate vulnerable patches, informing guidelines for when to trust their output and when to require additional checks.

The researchers ask three research questions concerning the security of Llama patches, agentic workflows, and associated code, issue, or project characteristics. p. 6
Where the paper says it

We perform this analysis for all vulnerabilities introduced both by Llama and the agentic workflows, allowing us to identify shared and distinct risk factors across different generation techniques. We group these factors into three categories: code-related, issue-related, and project-level characteristics: Code-related Characteristics: We analyze various properties of the LLM-generated code, including: # Files Added or Modified: A larger number of files can indicate broader changes and a higher overall risk [4, 25, 26, 63].

How the training signal is built

The methodology leverages the SWE-bench dataset consisting of real-world GitHub issues paired with pull requests. For the standalone model, Llama 3. p. 2

Where the paper says it

SWE-bench includes a train set of approximately 19,000 issue–PR pairs collected from 37 popular Python repositories and a test set of 2,294 issue–PR pairs from 12 repositories. For the standalone model, we use Llama 3.

3 Instruct (70B) was used to generate patches for approximately 21,000 instances across the train and test sets. p. 2
Where the paper says it

For the standalone model, we use Llama 3. 3 Instruct (70B) to generate patches for both the train and test sets of SWE-bench, comprising approximately 21,000 instances in total.

The agentic frameworks were analyzed using publicly released patches available only for the test set due to computational costs. p. 2
Where the paper says it

In contrast, for the agentic frameworks, we rely on publicly released patches that are available only for the test set. Due to the substantial computational and financial cost required to run these iterative frameworks across the train set, we restrict our framework-based analysis to the test split.

Where the gains come from

The analysis employs a multi-step process to detect vulnerabilities, combining CodeQL, Semgrep, and Bandit using a majority voting strategy to maximize recall and precision. The researchers also perform manual inspections to remove false positives and ensure validity. p. 5

Where the paper says it

This conservative approach can provide a balanced and trustworthy basis for our large-scale comparative analysis across different patch sources, including developer patches, standalone LLMs, and agentic frameworks. Step 3: To ensure validity of all instances and to guarantee that only valid security-relevant issues enter our analysis, we performed a manual inspection in which the first author inspected the patched file, and judged whether the tool-reported issue constituted a plausible security threat rather than a benign code smell.

Vulnerability prevalence is analyzed using CWE categories for RQ1 and RQ2. p. 5
Where the paper says it

This multi-step process addresses known limitations of fully automated static analysis and ensures the quality of our findings prior to our quantitative analyses. 5 Metrics for Empirical Analysis We analyze the prevalence and properties of vulnerabilities (RQ1 & RQ2) using CWE categories.

Code, Issue, and Project Characteristics of Vulnerable Instances (RQ3)

The analysis investigates code-level factors such as files added or modified, unique file types modified, LOC generated by LLM, and cyclomatic complexity of LLM-generated code. Issue-related characteristics include issue type classification and information completeness, such as the presence of expected behavior and steps to reproduce. p. 6

Where the paper says it

Information Completeness: Based on prior work emphasizing the importance of issue clarity [74], we manually annotate whether each issue includes: (1) Expected Behavior: how the software is supposed to behave, (2) Observed Behavior: how it actually behaves, and (3) Steps to Reproduce: the process needed to trigger the issue. The presence of this information can aid in understanding the bug’s context and identifying the root cause of the problem.

Project-level factors examined include contributor count, repository size, maintainability index, and cyclomatic complexity of the project. p. 7
Where the paper says it

Further, size of a project has been shown to have direct associations with the number of potential vulnerabilities [24]. Cyclomatic Complexity and Maintainability Index of Project: We use Radon [1], a popular code analysis tool [29, 42], to compute each project’s average maintainability index and cyclomatic complexity.

The results suggest that vulnerable patches are most strongly associated with code and issue-level characteristics rather than project-level properties. p. 0
Where the paper says it

We find that vulnerabilities in LLM-generated patches are associated with distinctive code characteristics and are commonly observed in issues missing specific types of information. These results suggest that contextual factors play a critical role in the security of the generated patches and point toward the need for proactive risk assessment methods that account for both issue and code-level information.

Conclusions

The study concludes that Llama can introduce up to 11x more new vulnerabilities than developers when addressing issues. Vulnerable outputs are associated with particular code and issue-level characteristics, such as a higher number of modified files and the absence of code snippets or certain information in the issue description. p. 2

Where the paper says it

We found that vulnerable outputs are more likely when LLM patches involve a higher number of files and unique file type modifications. Further, most standalone LLM-generated vulnerabilities were observed in instances involving bug-related issues that do not contain code examples, information about the expected behavior of the code, or steps to reproduce the issue.

This suggests that riskiness can be assessed not only by detecting specific vulnerable lines of code but also by developing complementary risk assessment techniques that evaluate both the generated code and the broader context in which a patch is proposed. Simple signals such as how widely a patch is spread across the codebase and the type of available information in an issue description can be combined into a prediction model that scores risk of APR usage before and after patch generation. The findings point toward integrating risk assessment into CI/CD pipelines as an early warning and triage mechanism. p. 0
Where the paper says it

We find that vulnerabilities in LLM-generated patches are associated with distinctive code characteristics and are commonly observed in issues missing specific types of information. These results suggest that contextual factors play a critical role in the security of the generated patches and point toward the need for proactive risk assessment methods that account for both issue and code-level information.

Llama 3.3-Generated Code CWE-95 (Eval Injection) 39, Developer-Written Code CWE-78 (OS Command Injection) 4, and Llama 3. p. 8

Where the paper says it

Vulnerability Types and Overlaps: Table 2 contains the most common vulnerabilities in Llama-generated and developer-written code 1 . The most common vulnerabilities in Llama-generated patches include command injection (CWE-78, 79 instances), eval injection (CWE-95, 39), insecure deserialization (CWE-502, 17), and path traversal (CWE-22, 14).

3 Test CWE-14 (Path Traversal) 14 are among the most recurring vulnerability types found in the train & test sets. p. 8
Where the paper says it

3-Generated Code CWE-78 (OS Command Injection) 79 CWE-95 (Eval Injection) 39 CWE-502 (Insecure Deserialization) 17 CWE-22 (Path Traversal) 14 Developer-Written Code CWE-78 (OS Command Injection) 4 CWE-377 (Insecure Temporary File) 3 CWE-732 (Permission Misconfiguration) 3 CWE-22 (Path Traversal) 3 Table 2. Most recurring CWE types found in Llama-generated and developer-written patches across the train & test sets.

These issues likely stem from the model’s tendency to translate task instructions into overly literal code, for example, using eval to dynamically execute strings, or defaulting to insecure algorithms like MD5. p. 9
Where the paper says it

Similarly insecure deserialization (CWE-502) was repeated in 17 LLM patches while being detected in only two developer patches. These issues likely stem from the model’s tendency to translate task instructions into overly literal code, for example, using eval() to dynamically execute strings, or defaulting to insecure algorithms like MD5.

OH produced an order of magnitude more vulnerabilities than ACR or HC. This disparity is attributed to OH's broad operational scope, which means it often touches more files than necessary to resolve an issue. The results indicate that giving agents unconstrained autonomy over large repositories can substantially increase the risk of vulnerabilities, especially via large, cross-file edits.

Vulnerable patches tend to modify more files and file types, but involve fewer lines of generated code. Higher-risk patches in ACR exhibit abnormally high code complexity. p. 7

Where the paper says it

Maintainability index is derived from several factors, including cyclomatic complexity, lines of code, and Halstead volume, and is designed to estimate how easy it is to maintain the codebase. A higher maintainability index suggests code that is easier to understand and modify, which may reduce the risk of generating vulnerabilities.

At the issue level, vulnerabilities are commonly found in bugs where the issue descriptions lack information such as clear statements about the observed and expected behavior or steps to reproduce. p. 2
Where the paper says it

We found that vulnerable outputs are more likely when LLM patches involve a higher number of files and unique file type modifications. Further, most standalone LLM-generated vulnerabilities were observed in instances involving bug-related issues that do not contain code examples, information about the expected behavior of the code, or steps to reproduce the issue.

Project-level factors such as repository size, cyclomatic complexity, and contributor count show non-significant correlations with vulnerability counts. p. 7
Where the paper says it

# Files: The number of files in a given project can serve as a proxy for its potential complexity and size. Prior work on open-source software effort estimation has used file count as an indicator of project scale and structural complexity [61].

The results suggest that for some APR frameworks, complexity-sensitive safeguards may be justified when edits occur in highly complex functions. p. 6
Where the paper says it

Generating more lines of code may suggest that the model is undertaking more complex tasks, handling broader context, or replacing larger code chunks, which can raise the likelihood of mistakes or security issues. Cyclomatic Complexity of LLM-Generated Code: This metric reflects the complexity of the functions introduced or rewritten by the LLM.

Frameworks should incorporate explicit limits on edit scope e.g., caps on diff size, limits on the number of files that can be modified in a single attempt. p. 7

Where the paper says it

Projects with more contributors often have greater oversight, which can lower the risk of security vulnerabilities. # Files: The number of files in a given project can serve as a proxy for its potential complexity and size.

Additionally, given that a small number of issues can account for a disproportionate share of vulnerabilities, it is beneficial to detect high-risk repair tasks such as those that repeatedly fail tests or result in unreasonably large diff files. This suggests that simple signals such as how widely a patch is spread across the codebase and the type of available information in an issue description can be combined into a prediction model that scores risk of APR usage before and after patch generation. The signals needed for such risk assessment are already present in typical development workflows, making the integration of risk assessment into CI/CD pipelines a promising direction for future work. p. 0
Where the paper says it

We find that vulnerabilities in LLM-generated patches are associated with distinctive code characteristics and are commonly observed in issues missing specific types of information. These results suggest that contextual factors play a critical role in the security of the generated patches and point toward the need for proactive risk assessment methods that account for both issue and code-level information.

The study presents the first large-scale security evaluation of an LLM and three agent-generated patches for real-world issue resolution tasks. The researchers find that Llama’s patches should be treated as an additional source of security risk. p. 0

Where the paper says it

We find that vulnerabilities in LLM-generated patches are associated with distinctive code characteristics and are commonly observed in issues missing specific types of information. These results suggest that contextual factors play a critical role in the security of the generated patches and point toward the need for proactive risk assessment methods that account for both issue and code-level information.

Furthermore, organizations deploying LLM-based APR tools should consider integrating security scanning and some form of manual inspection into workflows that accept LLM patches. p. 1
Where the paper says it

Hence, assessing their security requires evaluation frameworks that capture the full end-to-end repair workflow rather than isolated snippet-based analyses. Although recent benchmarks like SWE-bench [32] have enabled the assessment of LLMs on real-world software engineering tasks, security-specific analyses in these contexts remain limited, and we still lack a systematic understanding of when APR-style LLM and agentic frameworks introduce new vulnerabilities, or how those vulnerabilities relate to properties of the patches and the surrounding development context.

Using these LLM fixes as a replacement for developer-written patches without extra safeguards is likely to increase both the frequency and diversity of vulnerabilities in codebases. p. 5
Where the paper says it

Step 2: To increase confidence (precision), we adopt a conservative majority voting strategy: a generated or developer-written patch is treated as a security risk only if at least two of the three tools independently identify new vulnerabilities in the same file(s) it modifies or creates. Moving beyond aggregation, the majority vote dismisses findings reported by only a single tool as likely false positives.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Risk Assessment Integration: Integrate a risk model into CI/CD pipelines that scores APR tasks based on issue-level signals like information completeness and code-level features such as number of modified files, to route high-risk tasks to stricter security checks or human review.

  2. Agentic Workflow Constraints: Implement explicit limits on edit scope for autonomous agentic frameworks, such as caps on diff size or limits on the number of files modified in a single attempt, to prevent excessive rewrites and large cross-file edits that lead to vulnerabilities.

  3. Contextual Information Requirements: Require complete and precise problem descriptions from developers for APR systems, as vulnerability-prone patches are associated with issues where expected behavior and reproduction steps are under-specified.

  4. Code Complexity Safeguards: For agentic frameworks like AutoCodeRover, implement checks that flag or require human review when generated code exhibits abnormally high cyclomatic complexity, as this metric was found to be a significant indicator of security risk in those patches.

  5. Distinguishing LLM Behavior: Develop methods to distinguish between LLM-generated vulnerabilities and developer errors by analyzing distinct vulnerability patterns, such as the prevalence of CWE-95 (eval injection) and CWE-502 (insecure deserialization) which were far more prevalent in Llama-generated code.

Abstract

Large language models (LLMs) and agentic frameworks are increasingly used for development tasks such as automated program repair (APR). Prior work has identified security risks in LLM-generated code, but most studies focus on synthetic, simplified, or isolated tasks. We present a large-scale security analysis of LLM-generated patches using 20,000+ GitHub issues from SWE-bench. We evaluate developer-written patches, two standalone LLMs (Llama 3.3 Instruct-70B and GPT-5.4 nano), and four agentic frameworks (OpenHands, AutoCodeRover, HoneyComb, and SWE-agent). We also analyze code, issue, and project-level factors associated with insecure patches. Under our evaluated SWE-bench setup, Llama 3.3 introduced 135 manually confirmed vulnerabilities across the train and test sets, compared with 12 in developer-written patches for the same issues. On the SWE-bench test set, GPT-5.4 nano introduced 3 vulnerabilities compared with 2 in developer patches. All four evaluated agentic frameworks also introduced vulnerabilities. OpenHands had the highest retained count, with 44 vulnerabilities across 11 issue instances. Vulnerabilities are associated with broader code modifications and with issues missing specific information. These results suggest that APR systems require proactive risk assessment methods that consider both generated code and issue context.

Sources

Related papers