Retrieval-Constrained Policy Optimization for Attack Technique Extraction from Cyber Threat Intelligence
Jiayun Zhang, Junshen Xu, Zejun Xie, Yi Fan
cs.CR, cs.CL
Submitted: 2026-08-07
Updated: 2026-08-10
Comments: Accepted to the AI Agent for Information Retrieval (Agent4IR) Workshop at KDD 2026
Code: https://github.com/tumeteor/mitre-ttp-mapping
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Mapping cyber threat intelligence (CTI) text to MITRE ATT&CK techniques is essential for structured threat analysis, yet manual annotation is costly and does not scale.
Terminology
Abstract
Mapping cyber threat intelligence (CTI) text to MITRE ATT&CK techniques is essential for structured threat analysis, yet manual annotation is costly and does not scale. The ATT&CK taxonomy comprises several hundred attack techniques, and a single CTI passage may describe multiple techniques, making accurate and complete extraction challenging. Existing automated approaches fall short in different ways: multi-label classifiers struggle with severe class imbalance and the large label space, while LLM-based methods--retrieval pipelines and fine-tuned generators--optimize token-level objectives that treat technique annotation as sequence generation rather than set prediction, lacking direct supervision on whether the predicted technique set is correct and complete. We propose TTP-R1, a two-stage framework that combines retrieval-augmented supervised fine-tuning (SFT) with reinforcement learning using verifiable rewards (RLVR). A hybrid retriever first narrows the large label space to a candidate set, and a fine-tuned LLM learns to select the correct techniques. We then apply Group Relative Policy Optimization with a decomposed reward that directly supervises the precision, recall, and output format of the predicted technique set. Across four CTI benchmarks, TTP-R1 achieves the best average F1, improving sub-technique-level F1 by 7.4 percentage points over Claude Sonnet 4.5 with retrieval augmentation, while running 28x faster when served as an 8B-parameter model on a single GPU.
Sources
- Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMs
- Constructing Multi-label Hierarchical Classification Models for MITRE ATT&CK Text Tagging
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- From Text to Actionable Intelligence: Automating STIX Entity and Relationship Extraction
- GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
- LLMCloudHunter: Harnessing LLMs for Automated Extraction of Detection Rules from Cloud-Based CTI
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Qwen3 Technical Report
- Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills