A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Models

arXiv:2507.22659 · cs.SE, cs.AI · Submitted 2025-07-30 · Read on arXiv

cs.SE, cs.AI

Submitted: 2025-07-30

Updated: 2025-12-19

Comments: 43 pages + 20 pages references, 7 tables, 13 figures

DOI: 10.1145/3815425

Code: https://github.com/hs-esslingenit-security/Awesome-LLM4SVD

Project page: https://pypl.github.io/PYPL.html

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes

Sources

• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection
• Vulnerability Detection in Popular Programming Languages with Language Models
• From Generalist to Specialist: Exploring CWE-Specific Vulnerability Detection
• Qwen Technical Report
• Using ChatGPT as a Static Application Security Testing Tool
• Case Study: Fine-tuning Small Language Models for Accurate and Private CWE Detection in Python Code
• From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security
• Exploring Software Naturalness through Neural Language Models
• Can LLM Prompting Serve as a Proxy for Static Analysis in Vulnerability Detection
• Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?
• Evaluating Large Language Models Trained on Code
• Improving vulnerability type prediction and line-level detection via adversarial training-based data augmentation and multi-task learning
• Leveraging Self-Paced Learning for Software Vulnerability Detection
• Evaluation of ChatGPT Model for Vulnerability Detection
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
• DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
• Vulnerability Detection with Code Language Models: How Far Are We?
• Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties