A Hybrid Approach for Automated Security Pattern Recovery

arXiv:2611.05823 · cs.CR, cs.AI, cs.SE · Submitted 2026-11-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: We're looking at the paper: "A Hybrid Approach for Automated Security Pattern Recovery".

Elias: Here's the gist of the paper. Security-by-design advocates integrating security into software development, and security pattern catalogues operationalize this by providing reusable architectural solutions to recurring security problems.

Nadia: A significant challenge in utilizing these patterns is recovering their implementations from source code because security patterns are abstract, span multiple components, and possess numerous valid implementations.

Elias: The paper introduces SecPatRec, a hybrid approach designed for automated security pattern recovery from source code repositories.

Nadia: The core idea treats a security pattern as a reference architecture and frames its recovery as a deductive task: identifying which prescribed roles are present in the codebase, grounding these roles to specific code entities, and subsequently recovering their interaction sequences.

Paper summary: Nadia: So we've just covered how they built SecPatRec and how it works by combining an LLM agent with static analysis to solve the problem of automatically recovering security pattern implementations from source code repositories. Now, let's get a bit more into what the paper claims about this hybrid approach.

Elias: The central thesis is that security patterns are hard to recover because they are abstract and span multiple parts of a system, which makes it difficult to find their actual implementation in code without significant manual effort.

Priya: They tackle this by treating the pattern as a reference architecture and turning recovery into a deductive task where you first check for role presence, then ground those roles in specific code entities, and finally recover the sequence of interactions between them.

Nadia: It’s essentially breaking the big problem down into three manageable tasks. First, detecting which prescribed roles are present in the codebase. Then mapping those abstract roles to concrete code symbols like classes or functions that actually exist in your project. And finally recovering how those mapped symbols interact with each other in a specific sequence dictated by the pattern.

Elias: To do this, they use the LLM agent to iteratively use semantic and lexical code search to locate and inspect the relevant parts of the code, which produces those role mappings and presence statuses.

Priya: And once you have those mappings, they then use static code property graph analysis to take those mappings and recover the interactions between all the identified entities. That’s where they combine semantic understanding with evidence extracted directly from the source code structure.

Nadia: They are essentially combining the LLM's ability to interpret architectural roles with interaction evidence pulled right out of the source code structure itself. It’s a two-pronged attack on finding those patterns, which is what makes it hybrid.

Elias: The problem they formally define is recovering a tuple P equals R B from a security pattern P which is specified as having Roles R and an ordered sequence of Actions B, where the roles are prescribed architectural entities.

Priya: They also defined the evaluation framework around three specific correctness metrics: Role Detection Accuracy RDA, Mapping Localization Score MLS, and Interaction Recovery Correctness IRC. These are used to measure how good the recovery is at each step.

Nadia: The authors built a human-validated ground truth dataset of one hundred pattern instances covering five different security patterns across ninety-three open source Java and Python repositories to test this system against. That’s the material they used for their measurements.

Elias: Those results showed that the SecPatRec approach achieved a mean RDA of zero point eight four, a mean MLS of zero point eight four, and a mean IRC of zero point six six across the five patterns and two languages they tested.

Priya: The key thing here is that this demonstrates what actually happens when you run this system: it gets reasonably accurate results at all three stages without needing perfect manual input for every step. It’s not claiming perfection, but it's showing a solid performance level using a hybrid technique.

Nadia: And compared to a one shot end-to-end large language model baseline, SecPatRec yields higher aggregate recovery scores and lower estimated large language model API costs per run while still maintaining performance even when processing repositories exceeding the baseline's input limits.

Conclusion: Elias: So we've covered the details of SecPatRec and its performance numbers, and now we need to look at what this paper actually means in a broader sense regarding security pattern recovery.

Nadia: The authors introduce this hybrid approach called A Hybrid Approach for Automated Security Pattern Recovery, and their main takeaway is that you can automate the process of finding these reusable security patterns in codebases by framing it as a deductive software architecture recovery problem.

Priya: In simple terms, what they are saying is that we can create tools that automatically find the roles, map them to code symbols, and figure out how those entities interact based on a known pattern specification.

Elias: This has implications because it shifts the focus from just defining patterns to building scalable methods for mining those patterns in large software projects efficiently without requiring massive manual effort.

Nadia: It means that instead of spending all day manually tracing code to see if a specific authentication flow is implemented, you can use this system and get a structured result about its presence, mapping, and interaction sequence much faster.

Priya: The paper provides a solid benchmark—the dataset, the metrics RDA, MLS, and IRC—so other researchers can use it to test their own ideas for automated pattern mining at scale. That dataset is intended to be used as a standard reference point for this kind of work.

Elias: It’s about using the specific findings on SecPatRec to help guide future work in this area, especially looking at how different LLMs perform and how we can improve the hybrid combination between semantic interpretation and static analysis.

Nadia: The authors acknowledge that while they found good results, a limitation is that they rely on a human-validated ground truth dataset to measure their correctness. That means their accuracy depends on the quality of those annotations.

Priya: That’s a fair caveat; you can't ignore the need for human expertise in creating that ground truth, and it shows that even with this sophisticated AI framework, you still need expert input to validate what the AI is doing against.

Elias: But overall, this paper on A Hybrid Approach for Automated Security Pattern Recovery gives us a way to approach complex software architecture recovery by breaking it down into detection, grounding, and interaction recovery.

Nadia: It shows that combining an LLM's understanding of abstract concepts with concrete evidence from static analysis is a solid way to tackle the challenge of finding these security patterns in real world code.

Anh Minh Nguyen, Tuan Dat Tran, Alzubair Hassan, Liliana Pasquale

University College Dublin

cs.CR, cs.AI, cs.SE

Submitted: 2026-11-04

Updated: 2026-11-04

Comments: 35 pages, 41 figures, 11 tables

Code: https://github.com/anonymousspottedtoad/secpat_interaction_recovery

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 92/100

The gist: Security-by-design advocates integrating security into software development, and security pattern catalogues operationalize this by providing reusable architectural solutions to recurring security…

Key concepts

Security Pattern Catalogues
These are reusable architectural solutions for common security problems in software development. They provide a blueprint for secure designs, but their abstract nature makes it hard to find the actual code implementation they describe.
SecPatRec
A hybrid approach designed to automatically recover security patterns from source code. It uses a Large Language Model (LLM) for semantic understanding of architectural roles and static property graph analysis to find and map these roles within the codebase.
Role Detection Accuracy (RDA)
This metric measures how accurately the system identifies which prescribed security roles are present in the codebase. A high RDA indicates that the system correctly detects all required components of a security pattern.
Interaction Recovery Correctness (IRC)
This metric assesses how well the system recovers the sequence of interactions between identified code entities. It checks if the order and nature of communication between these entities match the original security pattern's prescribed flow.

Terminology

Summary

Security-by-design advocates integrating security into software development, and security pattern catalogues operationalize this by providing reusable architectural solutions to recurring security problems. p. 0

Where the paper says it

ie Abstract—Security-by-design emphasises integrating security into software design and development. Security pattern catalogues operationalise this principle by prescribing reusable architectural solutions to recurring security problems.

A significant challenge in utilizing these patterns is recovering their implementations from source code because security patterns are abstract, span multiple components, and possess numerous valid implementations. p. 0
Where the paper says it

Security pattern catalogues operationalise this principle by prescribing reusable architectural solutions to recurring security problems. However, recovering their implementations from source code remains challenging because security patterns are abstract, span multiple components, and admit many valid implementations.

The paper introduces SecPatRec, a hybrid approach designed for automated security pattern recovery from source code repositories. p. 0

Where the paper says it

However, recovering their implementations from source code remains challenging because security patterns are abstract, span multiple components, and admit many valid implementations. We present SecPatRec, a hybrid approach for automated security pattern recovery from source code repositories.

The core idea treats a security pattern as a reference architecture and frames its recovery as a deductive task: identifying which prescribed roles are present in the codebase, grounding these roles to specific code entities, and subsequently recovering their interaction sequences. p. 0
Where the paper says it

We present SecPatRec, a hybrid approach for automated security pattern recovery from source code repositories. We treat a security pattern as a reference architecture and frame its recovery as a deductive task: determining which prescribed roles are present, grounding them in code entities, and recovering their interaction sequences.

SecPatRec employs an agent utilizing a ReAct-style Large Language Model (LLM), augmented with both semantic and lexical code search capabilities for role detection and grounding. p. 0

Where the paper says it

We treat a security pattern as a reference architecture and frame its recovery as a deductive task: determining which prescribed roles are present, grounding them in code entities, and recovering their interaction sequences. SecPatRec pairs a ReAct-style LLM agent equipped with semantic and lexical code search for role detection and grounding with static code property graph analysis for interaction recovery.

This is paired with static code property graph analysis to recover interactions between the identified entities. p. 2
Where the paper says it

Also, individual code unit classification can not be adopted for the next task of security pattern recovery, which is recovering a sequence of behaviorial interactions between the mapped roles. To address the above challenges, we propose a hybrid approach that combines LLMs with code search methods and static code property graph (CPG) analysis [24] to recover the

The overall pipeline combines the LLM's semantic interpretation of architectural roles with interaction evidence extracted directly from the source code.

The problem is formally defined as recovering a tuple (M, D, S) from a security pattern P = (R, B), where R is the set of Roles and B is an ordered sequence dictating behavioral flow. p. 2

Where the paper says it

• Actions (B): An ordered sequence B = ⟨b1, . , bn⟩ dictating the behavioral flow between those roles.

The recovery involves three tasks: (1) detecting the presence status (D) of each role in the codebase C; (2) mapping each prescribed role r in R to a concrete code symbol e in E(C) —either one defined in C or only referenced by it; and (3) recovering the sequence of interactions (S) between these mapped symbols. p. 2
Where the paper says it

Recovering P from C requires three tasks: (1) detect which roles of P are present in C, by identifying the presence status of each role that composes the pattern, (2) map each prescribed role r ∈ R to a concrete code symbol e ∈ E(C) where E(C) represents the union of all Abstract Syntax Trees (AST) nodes inside the codebase, and (3) recover the sequence of interactions between those mapped symbols. We denote the recovered architecture as a tuple (M, D, S), where: 1) Role mappings (M): A set of pairs m = (r, e) with r ∈ R, each mapping a prescribed role to the code symbol e that implements it, where e is either defined in C, i.

The evaluation relies on a human-validated ground-truth dataset comprising 100 pattern instances across five security patterns (P1: authentication_pwd, P2: verifiable_token_based_authentication, P3: obscure_token-based access control, P4: session-based access control, and P5: limit request rate) spanning 93 open-source Java and Python repositories. Each pattern instance records role-to-code mappings (M), role presence statuses (D), and an interaction sequence (S). p. 1

Where the paper says it

Each pattern is represented by ten instances per language. Each instance records role-to-code mappings, role presence statuses and an interaction sequence.

A three-level evaluation framework measures recovery correctness through Role Detection Accuracy (RDA), Mapping Localization Score (MLS), and Interaction Recovery Correctness (IRC). p. 1
Where the paper says it

Each instance records role-to-code mappings, role presence statuses and an interaction sequence. • A three-level evaluation framework that measures recovery correctness against the ground truth through role detection accuracy (RDA), mapping localization score (MLS) and interaction recovery correctness (IRC).

Empirical results, averaged over four LLMs (including proprietary and open-source models like GPT-6. p. 0

Where the paper says it

We evaluate SecPatRec using four LLMs, including proprietary and open-source models. Averaged over all models and pattern instances, SecPatRec achieves role-detection accuracy of 0.

1, Muse Spark, DeepSeek v4 flash, and GLM 5. p. 6
Where the paper says it

1 Sol, Meta’s Muse Spark, Deepseek-v4. 0-Flash-0731, and GLM-5.

3 flash) across the five patterns and two languages, demonstrate that SecPatRec achieves a mean RDA of 0. p. 1
Where the paper says it

• A three-level evaluation framework that measures recovery correctness against the ground truth through role detection accuracy (RDA), mapping localization score (MLS) and interaction recovery correctness (IRC). • An empirical evaluation using four LLMs, including proprietary and open-source models, across five patterns and two languages.

84, a mean MLS of 0. p. 6
Where the paper says it

We normalize this distance by the height 2H of the containment tree as no path in a tree of height H is longer than 2H edges, then score the tree-induced similarity as 1 − d/2H. The mapping localization score is the mean score over all roles in F, and it is 0 when no role is found by both sides (|F| = 0), as then nothing is mapped correctly: MLS =    1 |F| X r∈F 1 − d(ˆer, er) 2H |F| > 0, 0 |F| = 0.

84, and a mean IRC of 0. p. 8
Where the paper says it

Mean RDA, MLS and IRC against codebase size per LLM: SecPatRec (solid) against the one-shot baseline (dashed), in five size bins of about 20 instances. Instances over the baseline’s input limit score 0 for the baseline.

66. Compared to a one-shot end-to-end LLM baseline, SecPatRec yields higher aggregate recovery scores and lower estimated LLM API costs per run, while maintaining performance even when processing repositories exceeding the baseline's input limits. The hybrid design is effective because the LLM handles code understanding for mapping, while static analysis reads interaction evidence from the code itself. p. 8
Where the paper says it

Most of the IRC error therefore comes from the mapping stage, not from the traversal. This is the main argument for our hybrid design: the LLM handles the part of the task that needs an understanding of the code, while the static analysis reads the interactions from the code itself, without degrading as codebases grow.

The CPG traversal scales effectively with codebase size as RDA and MLS do not decrease, and it proves significantly more cost-effective above 9k to 36k tokens and 6 to 60 times cheaper at 1M tokens compared to the baseline. p. 8
Where the paper says it

28, against 0. 00 for the baseline), so it is cheaper above 9k to 36k tokens and 6 to 60 times cheaper at 1M tokens, while RDA and MLS do not drop as codebases grow.

The researchers conclude that their dataset, metrics, and pipeline serve as a benchmark for mining security patterns at scale. p. 9
Where the paper says it

As the CPG analysis is nearly exact when the roles are mapped exactly, improving role mapping is the key to further improvement. Our dataset, metrics and pipeline can serve as a benchmark and an instrument for mining security patterns at scale.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Hybrid approach for automated security pattern recovery: The system can combine an LLM agent with semantic and lexical code search for role detection and grounding with static code property graph analysis for interaction recovery, leading to higher aggregate recovery scores compared to a one-shot end-to-end LLM baseline.

  2. Role detection accuracy improvement: The system uses a ReAct-style LLM agent equipped with semantic and lexical code search to find code units by what they mean rather than by their structure, which is effective for Java and Python repositories exceeding single prompt limits.

  3. Interaction sequence recovery enhancement: The system utilizes CPG traversal guided by role mappings to recover interaction sequences from the source code, providing structural and behavioral evidence that can support inspection of security pattern implementations.

  4. Cost-effective operation: By using the SecPatRec pipeline, the system achieves lower estimated LLM API costs per run compared to a one-shot baseline, while still processing repositories whose packed source exceeds the baseline’s configured input limit.

  5. Scalability across codebase size: The system's RDA and MLS do not decrease as codebases grow, unlike the baseline, because CPG analysis scales with codebase size, making it cheaper above 9k to 36k tokens compared to the baseline.

  6. LLM selection guidance: The system can recommend GLM for low-cost recovery due to its cost-effectiveness while matching other LLMs on role detection and mapping performance, or GPT-6.1 when exact interaction sequences matter due to its highest IRC score of 0.72 in Java.

  7. Code search method optimization: The ReAct loop treats keyword and semantic search as interchangeable ways to find candidate code entities, meaning removing either tool changes RDA and MLS by less than 0.03, allowing resource-constrained environments to use keyword search alone for saving memory without accuracy loss.

Abstract

Security-by-design emphasises integrating security into software design and development. Security pattern catalogues operationalise this principle by prescribing reusable architectural solutions to recurring security problems. However, recovering their implementations from source code remains challenging because security patterns are abstract, span multiple components, and admit many valid implementations. We present SecPatRec, a hybrid approach for automated security pattern recovery from source code repositories. We treat a security pattern as a reference architecture and frame its recovery as a deductive task: determining which prescribed roles are present, grounding them in code entities, and recovering their interaction sequences. SecPatRec pairs a ReAct-style LLM agent equipped with semantic and lexical code search for role detection and grounding with static code property graph analysis for interaction recovery. We curate a human-validated ground-truth dataset of 100 pattern instances covering five security patterns across 93 open-source Java and Python repositories, and introduce a three-level evaluation framework measuring recovery correctness against this ground truth. We evaluate SecPatRec using four LLMs, including proprietary and open-source models. Averaged over all models and pattern instances, SecPatRec achieves role-detection accuracy of 0.84, role-grounding similarity of 0.84, and an interaction-recovery score of 0.66. Compared to a one-shot end-to-end LLM baseline, SecPatRec achieves higher aggregate recovery scores and lower estimated LLM API costs per run, while processing repositories that exceed the baseline's configured input limit. The recovered role mappings and statically derived interaction sequences provide structural and behavioural evidence that can support inspection of security-pattern implementations.

Sources

Related papers