A Hybrid Approach for Automated Security Pattern Recovery
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "A Hybrid Approach for Automated Security Pattern Recovery".
Elias: Here's the gist of the paper. Security-by-design advocates integrating security into software development, and security pattern catalogues operationalize this by providing reusable architectural solutions to recurring security problems.
Nadia: A significant challenge in utilizing these patterns is recovering their implementations from source code because security patterns are abstract, span multiple components, and possess numerous valid implementations.
Elias: The paper introduces SecPatRec, a hybrid approach designed for automated security pattern recovery from source code repositories.
Nadia: The core idea treats a security pattern as a reference architecture and frames its recovery as a deductive task: identifying which prescribed roles are present in the codebase, grounding these roles to specific code entities, and subsequently recovering their interaction sequences.
Paper summary: Nadia: So we've just covered how they built SecPatRec and how it works by combining an LLM agent with static analysis to solve the problem of automatically recovering security pattern implementations from source code repositories. Now, let's get a bit more into what the paper claims about this hybrid approach.
Elias: The central thesis is that security patterns are hard to recover because they are abstract and span multiple parts of a system, which makes it difficult to find their actual implementation in code without significant manual effort.
Priya: They tackle this by treating the pattern as a reference architecture and turning recovery into a deductive task where you first check for role presence, then ground those roles in specific code entities, and finally recover the sequence of interactions between them.
Nadia: It’s essentially breaking the big problem down into three manageable tasks. First, detecting which prescribed roles are present in the codebase. Then mapping those abstract roles to concrete code symbols like classes or functions that actually exist in your project. And finally recovering how those mapped symbols interact with each other in a specific sequence dictated by the pattern.
Elias: To do this, they use the LLM agent to iteratively use semantic and lexical code search to locate and inspect the relevant parts of the code, which produces those role mappings and presence statuses.
Priya: And once you have those mappings, they then use static code property graph analysis to take those mappings and recover the interactions between all the identified entities. That’s where they combine semantic understanding with evidence extracted directly from the source code structure.
Nadia: They are essentially combining the LLM's ability to interpret architectural roles with interaction evidence pulled right out of the source code structure itself. It’s a two-pronged attack on finding those patterns, which is what makes it hybrid.
Elias: The problem they formally define is recovering a tuple P equals R B from a security pattern P which is specified as having Roles R and an ordered sequence of Actions B, where the roles are prescribed architectural entities.
Priya: They also defined the evaluation framework around three specific correctness metrics: Role Detection Accuracy RDA, Mapping Localization Score MLS, and Interaction Recovery Correctness IRC. These are used to measure how good the recovery is at each step.
Nadia: The authors built a human-validated ground truth dataset of one hundred pattern instances covering five different security patterns across ninety-three open source Java and Python repositories to test this system against. That’s the material they used for their measurements.
Elias: Those results showed that the SecPatRec approach achieved a mean RDA of zero point eight four, a mean MLS of zero point eight four, and a mean IRC of zero point six six across the five patterns and two languages they tested.
Priya: The key thing here is that this demonstrates what actually happens when you run this system: it gets reasonably accurate results at all three stages without needing perfect manual input for every step. It’s not claiming perfection, but it's showing a solid performance level using a hybrid technique.
Nadia: And compared to a one shot end-to-end large language model baseline, SecPatRec yields higher aggregate recovery scores and lower estimated large language model API costs per run while still maintaining performance even when processing repositories exceeding the baseline's input limits.
Conclusion: Elias: So we've covered the details of SecPatRec and its performance numbers, and now we need to look at what this paper actually means in a broader sense regarding security pattern recovery.
Nadia: The authors introduce this hybrid approach called A Hybrid Approach for Automated Security Pattern Recovery, and their main takeaway is that you can automate the process of finding these reusable security patterns in codebases by framing it as a deductive software architecture recovery problem.
Priya: In simple terms, what they are saying is that we can create tools that automatically find the roles, map them to code symbols, and figure out how those entities interact based on a known pattern specification.
Elias: This has implications because it shifts the focus from just defining patterns to building scalable methods for mining those patterns in large software projects efficiently without requiring massive manual effort.
Nadia: It means that instead of spending all day manually tracing code to see if a specific authentication flow is implemented, you can use this system and get a structured result about its presence, mapping, and interaction sequence much faster.
Priya: The paper provides a solid benchmark—the dataset, the metrics RDA, MLS, and IRC—so other researchers can use it to test their own ideas for automated pattern mining at scale. That dataset is intended to be used as a standard reference point for this kind of work.
Elias: It’s about using the specific findings on SecPatRec to help guide future work in this area, especially looking at how different LLMs perform and how we can improve the hybrid combination between semantic interpretation and static analysis.
Nadia: The authors acknowledge that while they found good results, a limitation is that they rely on a human-validated ground truth dataset to measure their correctness. That means their accuracy depends on the quality of those annotations.
Priya: That’s a fair caveat; you can't ignore the need for human expertise in creating that ground truth, and it shows that even with this sophisticated AI framework, you still need expert input to validate what the AI is doing against.
Elias: But overall, this paper on A Hybrid Approach for Automated Security Pattern Recovery gives us a way to approach complex software architecture recovery by breaking it down into detection, grounding, and interaction recovery.
Nadia: It shows that combining an LLM's understanding of abstract concepts with concrete evidence from static analysis is a solid way to tackle the challenge of finding these security patterns in real world code.
Anh Minh Nguyen, Tuan Dat Tran, Alzubair Hassan, Liliana Pasquale
University College Dublin
cs.CR, cs.AI, cs.SE
Submitted: 2026-11-04
Updated: 2026-11-04
Comments: 35 pages, 41 figures, 11 tables
Code: https://github.com/anonymousspottedtoad/secpat_interaction_recovery
License: http://creativecommons.org/licenses/by-sa/4.0/
Importance score: 92/100
The gist: Security-by-design advocates integrating security into software development, and security pattern catalogues operationalize this by providing reusable architectural solutions to recurring security…
Key concepts
- Security Pattern Catalogues
- These are reusable architectural solutions for common security problems in software development. They provide a blueprint for secure designs, but their abstract nature makes it hard to find the actual code implementation they describe.
- SecPatRec
- A hybrid approach designed to automatically recover security patterns from source code. It uses a Large Language Model (LLM) for semantic understanding of architectural roles and static property graph analysis to find and map these roles within the codebase.
- Role Detection Accuracy (RDA)
- This metric measures how accurately the system identifies which prescribed security roles are present in the codebase. A high RDA indicates that the system correctly detects all required components of a security pattern.
- Interaction Recovery Correctness (IRC)
- This metric assesses how well the system recovers the sequence of interactions between identified code entities. It checks if the order and nature of communication between these entities match the original security pattern's prescribed flow.
Terminology
Summary
Security-by-design advocates integrating security into software development, and security pattern catalogues operationalize this by providing reusable architectural solutions to recurring security problems. p. 0 ie
Abstract—Security-by-design emphasises integrating security
into software design and development. Security pattern catalogues operationalise this principle by prescribing reusable
architectural solutions to recurring security problems. Security pattern catalogues operationalise this principle by prescribing reusable
architectural solutions to recurring security problems. However,
recovering their implementations from source code remains
challenging because security patterns are abstract, span multiple
components, and admit many valid implementations.Where the paper says it
Where the paper says it
The paper introduces SecPatRec, a hybrid approach designed for automated security pattern recovery from source code repositories. p. 0 However,
recovering their implementations from source code remains
challenging because security patterns are abstract, span multiple
components, and admit many valid implementations. We present
SecPatRec, a hybrid approach for automated security pattern
recovery from source code repositories. We present
SecPatRec, a hybrid approach for automated security pattern
recovery from source code repositories. We treat a security
pattern as a reference architecture and frame its recovery
as a deductive task: determining which prescribed roles are
present, grounding them in code entities, and recovering their
interaction sequences.Where the paper says it
Where the paper says it
SecPatRec employs an agent utilizing a ReAct-style Large Language Model (LLM), augmented with both semantic and lexical code search capabilities for role detection and grounding. p. 0 We treat a security
pattern as a reference architecture and frame its recovery
as a deductive task: determining which prescribed roles are
present, grounding them in code entities, and recovering their
interaction sequences. SecPatRec pairs a ReAct-style LLM agent
equipped with semantic and lexical code search for role detection
and grounding with static code property graph analysis for
interaction recovery. Also, individual code unit classification can
not be adopted for the next task of security pattern recovery,
which is recovering a sequence of behaviorial interactions
between the mapped roles. To address the above challenges, we propose a hybrid
approach that combines LLMs with code search methods and
static code property graph (CPG) analysis [24] to recover theWhere the paper says it
Where the paper says it
The problem is formally defined as recovering a tuple (M, D, S) from a security pattern P = (R, B), where R is the set of Roles and B is an ordered sequence dictating behavioral flow. p. 2 • Actions (B): An ordered sequence B = ⟨b1, . , bn⟩
dictating the behavioral flow between those roles. Recovering P from C
requires three tasks: (1) detect which roles of P are present
in C, by identifying the presence status of each role that
composes the pattern, (2) map each prescribed role r ∈ R
to a concrete code symbol e ∈ E(C) where E(C) represents
the union of all Abstract Syntax Trees (AST) nodes inside the
codebase, and (3) recover the sequence of interactions between
those mapped symbols. We denote the recovered architecture
as a tuple (M, D, S), where:
1) Role mappings (M): A set of pairs m = (r, e) with r ∈
R, each mapping a prescribed role to the code symbol e
that implements it, where e is either defined in C, i.Where the paper says it
Where the paper says it
The evaluation relies on a human-validated ground-truth dataset comprising 100 pattern instances across five security patterns (P1: authentication_pwd, P2: verifiable_token_based_authentication, P3: obscure_token-based access control, P4: session-based access control, and P5: limit request rate) spanning 93 open-source Java and Python repositories. Each pattern instance records role-to-code mappings (M), role presence statuses (D), and an interaction sequence (S). p. 1 Each pattern is represented by ten instances per language. Each instance records role-to-code mappings, role presence statuses and an interaction sequence. Each instance records role-to-code mappings, role presence statuses and an interaction sequence. • A three-level evaluation framework that measures recovery correctness against the ground truth through role
detection accuracy (RDA), mapping localization score
(MLS) and interaction recovery correctness (IRC).Where the paper says it
Where the paper says it
Empirical results, averaged over four LLMs (including proprietary and open-source models like GPT-6. p. 0 We evaluate SecPatRec
using four LLMs, including proprietary and open-source models. Averaged over all models and pattern instances, SecPatRec
achieves role-detection accuracy of 0. 1 Sol, Meta’s Muse Spark,
Deepseek-v4. 0-Flash-0731, and GLM-5. • A three-level evaluation framework that measures recovery correctness against the ground truth through role
detection accuracy (RDA), mapping localization score
(MLS) and interaction recovery correctness (IRC). • An empirical evaluation using four LLMs, including
proprietary and open-source models, across five patterns
and two languages. We normalize this distance by the
height 2H of the containment tree as no path in a tree of
height H is longer than 2H edges, then score the tree-induced
similarity as 1 − d/2H. The mapping localization score is the
mean score over all roles in F, and it is 0 when no role is
found by both sides (|F| = 0), as then nothing is mapped
correctly:
MLS =
1
|F|
X
r∈F
1 −
d(ˆer, er)
2H
|F| > 0,
0 |F| = 0. Mean RDA, MLS and IRC against codebase size per LLM: SecPatRec
(solid) against the one-shot baseline (dashed), in five size bins of about 20
instances. Instances over the baseline’s input limit score 0 for the baseline. Most of
the IRC error therefore comes from the mapping stage, not
from the traversal. This is the main argument for our hybrid
design: the LLM handles the part of the task that needs an
understanding of the code, while the static analysis reads
the interactions from the code itself, without degrading as
codebases grow. 28, against 0. 00 for the baseline),
so it is cheaper above 9k to 36k tokens and 6 to 60 times
cheaper at 1M tokens, while RDA and MLS do not drop
as codebases grow. As the CPG
analysis is nearly exact when the roles are mapped exactly,
improving role mapping is the key to further improvement. Our dataset, metrics and pipeline can serve as a benchmark
and an instrument for mining security patterns at scale.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Hybrid approach for automated security pattern recovery: The system can combine an LLM agent with semantic and lexical code search for role detection and grounding with static code property graph analysis for interaction recovery, leading to higher aggregate recovery scores compared to a one-shot end-to-end LLM baseline.
-
Role detection accuracy improvement: The system uses a ReAct-style LLM agent equipped with semantic and lexical code search to find code units by what they mean rather than by their structure, which is effective for Java and Python repositories exceeding single prompt limits.
-
Interaction sequence recovery enhancement: The system utilizes CPG traversal guided by role mappings to recover interaction sequences from the source code, providing structural and behavioral evidence that can support inspection of security pattern implementations.
-
Cost-effective operation: By using the SecPatRec pipeline, the system achieves lower estimated LLM API costs per run compared to a one-shot baseline, while still processing repositories whose packed source exceeds the baseline’s configured input limit.
-
Scalability across codebase size: The system's RDA and MLS do not decrease as codebases grow, unlike the baseline, because CPG analysis scales with codebase size, making it cheaper above 9k to 36k tokens compared to the baseline.
-
LLM selection guidance: The system can recommend GLM for low-cost recovery due to its cost-effectiveness while matching other LLMs on role detection and mapping performance, or GPT-6.1 when exact interaction sequences matter due to its highest IRC score of 0.72 in Java.
-
Code search method optimization: The ReAct loop treats keyword and semantic search as interchangeable ways to find candidate code entities, meaning removing either tool changes RDA and MLS by less than 0.03, allowing resource-constrained environments to use keyword search alone for saving memory without accuracy loss.
Abstract
Security-by-design emphasises integrating security into software design and development. Security pattern catalogues operationalise this principle by prescribing reusable architectural solutions to recurring security problems. However, recovering their implementations from source code remains challenging because security patterns are abstract, span multiple components, and admit many valid implementations. We present SecPatRec, a hybrid approach for automated security pattern recovery from source code repositories. We treat a security pattern as a reference architecture and frame its recovery as a deductive task: determining which prescribed roles are present, grounding them in code entities, and recovering their interaction sequences. SecPatRec pairs a ReAct-style LLM agent equipped with semantic and lexical code search for role detection and grounding with static code property graph analysis for interaction recovery. We curate a human-validated ground-truth dataset of 100 pattern instances covering five security patterns across 93 open-source Java and Python repositories, and introduce a three-level evaluation framework measuring recovery correctness against this ground truth. We evaluate SecPatRec using four LLMs, including proprietary and open-source models. Averaged over all models and pattern instances, SecPatRec achieves role-detection accuracy of 0.84, role-grounding similarity of 0.84, and an interaction-recovery score of 0.66. Compared to a one-shot end-to-end LLM baseline, SecPatRec achieves higher aggregate recovery scores and lower estimated LLM API costs per run, while processing repositories that exceed the baseline's configured input limit. The recovered role mappings and statically derived interaction sequences provide structural and behavioural evidence that can support inspection of security-pattern implementations.
Sources
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data