Security Logs to ATT&CK Insights: Leveraging LLMs for High-Level Threat Understanding and Cognitive Trait Inference
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia.
Elias: And I'm Elias, here with Priya.
Nadia: We're looking at the paper: "Security Logs to ATT&CK Insights".
Elias: Here's the gist of the paper. Understanding adversarial behavior in cybersecurity has traditionally relied on high-level intelligence reports and manual interpretation of attack chains The paper proposes a novel framework that leverages large language models (LLMs) to analyze Suricata IDS logs and infer attacker actions in terms of MITRE ATT&CK techniques Pruning redundant reasoning tokens The core contribution of the work is a structured LLM framework designed to bridge the semantic gap between low-level packet-level logs and high-level strategic intent This framework transforms raw Suricata telemetry into interpretable behavioral representations through a two-stage process involving action segmentation and ATT&CK mapping The methodology involves segmenting logs into discrete operator actions and then applying a retrieval-augmented large language model (RAG-LLM) to map these actions to MITRE ATT&CK techniques How the training signal is built The framework's input data is derived from Operation 418, a controlled cybersecurity experiment where trained professionals acted as red-team participants simulating a realistic corporate network environment The primary source for analysis is the Suricata Intrusion Detection System (IDS), which logs alerts, flow records, and protocol metadata across network segments Action segmentation is achieved by using an LLM to iteratively group related logs into semantically coherent action groups based on a running summary Subsequently, the RAG-LLM interfaces with a structured knowledge base containing descriptions for all ATT&CK tactics and techniques to generate explanations of potential matches Where the gains come from The results compare the Suricata-based inference against a baseline derived from OPNOTES, which are real-time journals where attackers documented their own actions and objectives The evaluation measures overall tactic coverage, participant-level precision, recall, and F1 scores relative to the OPNOTES-derived annotations Findings indicate that the Suricata model achieves high precision across participants when identifying an ATT&CK tactic, suggesting it reliably captures what is visible on the wire However, recall is lower and more variable because network telemetry lacks linguistic or cognitive context Limitations and future directions The study reveals that while the Suricata model reproduces the structure of attacks where traffic signatures exist, it struggles to capture host-resident or cognitively motivated phases like persistence or command-and-control adaptation This gap highlights that visibility shapes inference, as telemetry excels in network-visible phases but fails in capturing intent driven by internal actions Future work plans include incorporating multiple data sources such as host telemetry and authentication traces for a more complete reconstruction of operations Furthermore, the research aims to connect this work to real-time bias detection and bias-aware Theory of Mind architectures capable of anticipating adversary behavior The ultimate goal is deploying the framework in live settings for continuous inference of attacker strategies and cognitive traits This work demonstrates that LLMs can extract meaningful behavioral structure from low-level network telemetry, approaching the interpretive quality of models operating on cognitively rich data The Suricata-based LLM successfully narrows the semantic gap between machine logs and strategic intent through contextual segmentation and ATT&CK mapping By identifying behavioral shifts like tool switching, the framework begins to expose latent cognitive signatures that support reasoning about traits such as risk tolerance The approach confirms that key elements of attacker intent can be reconstructed even without access to human-authored reasoning ACKNOWLEDGMENT This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA) under Reimagining Security with Cyberpsychology Informed Network Defenses (ReSCIND) program contract N66001-24-C-4504 The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S.
Nadia: Government The U.S.
Elias: Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein REFERENCES Al-Sada, B., Sadighian, A., & Oligeri, G.
Nadia: (2025).
Title and authors: Nadia: Let's talk about who wrote this, Stacy Marsella, Sofia Hirschmann, and Nikolos Gurney. They came from USC Institute for Creative Technologies and Northeastern University.
Elias: I think their background is a good mix because they cover the technical side of using LLMs and the real-world testing environment.
Priya: I wonder what their specific expertise was in relation to this Suricata log analysis, because that's a very specific domain.
Nadia: They're focused on exploitation and defense in practice, so they understand how these logs are actually generated and what constitutes a meaningful alert versus noise.
Elias: And I think the cryptographer side of their background helps them think about how the LLM is using those technical inputs to draw inferences about something non-technical, like risk tolerance.
Priya: I'm curious if they have experience with the specific data pipeline they built, because getting raw Suricata data into an LLM input for segmentation is a big engineering challenge.
Nadia: They did it through Operation four hundred eighteen which was this controlled experiment where trained professionals simulated a realistic corporate network over two days <ref:2510.20930#pg3>.
Elias: That environment provided the necessary realism to test the framework under operational constraints, which is crucial for testing if the LLM can actually handle real-world complexity.
Priya: So they built a data pipeline that simulates reality, which means their results are grounded in an environment that's already quite adversarial.
Nadia: And they specifically focused on moving away from relying solely on participant-authored notes, which is a shift in how we think about evidence collection for threat analysis.
Elias: That shift towards automated inference based on telemetry rather than manual annotation is what makes this paper interesting from a modeling standpoint.
Priya: It’s about the mechanism of inference itself, not just the conclusion that it works, right? How does the LLM actually make those cognitive jumps?
The paper's summary: Nadia: The core idea of "Security Logs to ATT andCK Insights: Leveraging LLMs for High-Level Threat Understanding and Cognitive Trait Inference" is using an LLM to analyze Suricata IDS logs to infer attacker actions in terms of MITRE ATT andCK techniques.
Elias: It’s taking low-level network traffic data, like alerts and flow records, and turning it into high-level strategic intent using the ATT andCK framework.
Priya: So instead of just seeing a single alert for a specific exploit, they are trying to infer a broader pattern of behavior across multiple events.
Nadia: They segment the continuous network activity into meaningful behavioral phases and map those phases to probable ATT andCK techniques using textual and temporal evidence.
Elias: And they look beyond the technical mapping itself, exploring how things like tool switching or protocol transitions can signal cognitive traits embedded in attacker decision-making.
Priya: So they are not just looking for 'what' happened, but trying to find out 'why' it happened based on the sequence of events.
Nadia: They are hypothesizing that patterns like those can be signals of cognitive traits such as loss aversion or ambiguity aversion embedded in the attacker's decision-making process.
Elias: So they are essentially training an AI to read network sequences and associate them with psychological concepts derived from other studies on hacker behavior.
Priya: That moves the analysis from simple detection to behavioral interpretation, which is a big step because it tries to model the human element involved in cyber operations.
The paper's improvements: Nadia: The paper highlights that one improvement is this structured LLM framework designed to bridge that semantic gap between low-level packet-level logs and high-level strategic intent.
Elias: That framework involves two stages: action segmentation and then applying a retrieval-augmented LLM to map those actions to MITRE ATT andCK techniques.
Priya: So the improvement is making the process more systematic, less ad hoc, by giving it a clear pipeline instead of just throwing logs at it.
Nadia: Exactly. And they also improved their attack fidelity by comparing their inferences against human-authored notes from Operation four hundred eighteen to get a quantifiable measure of how well it aligns with those labels <ref:2510.20930#pg3>.
Elias: That comparison is interesting because it shows that the model achieves strong precision across participants, meaning when it identifies an ATT andCK tactic, it usually aligns with what the OPNOTES-derived label said.
Priya: So they’re quantifying the accuracy by showing that their inference matches what's already there in human notes for most instances.
Nadia: But they also found that recall is lower and more variable because network telemetry lacks linguistic or cognitive context, which is a key caveat to remember.
Elias: So the improvement isn't perfection; it’s showing where the system excels—high precision on the visible actions versus lower recall because of missing context.
Priya: It shows that we can measure how well telemetry approximates that cognitively informed upper bound on interpretive accuracy from attacker notes, which is a really solid metric.
Conclusion: Nadia: So to wrap up, this work on "Security Logs to ATT andCK Insights: Leveraging LLMs for High-Level Threat Understanding and Cognitive Trait Inference" shows that LLMs can extract meaningful behavioral structure from low-level network telemetry.
Elias: The Suricata-based LLM successfully narrows the semantic gap between machine logs and strategic intent through contextual segmentation and ATT andCK mapping.
Priya: It confirms that key elements of attacker intent can be reconstructed even without access to human-authored reasoning.
Nadia: They’ve managed to expose latent cognitive signatures, like tool switching or protocol transitions, which support reasoning about traits like risk tolerance or persistence.
Elias: This means we can start thinking about how these inferred traits shape the adversary't strategy in a way that goes beyond just spotting malicious signatures.
Priya: It moves the focus toward building systems that can anticipate behavior based on inferred cognitive traits, which is where the real long-term defensive value lies.
Soham Hans, Stacy Marsella, Sofia Hirschmann, Nikolos Gurney
USC Institute for Creative Technologies · Northeastern University
cs.CR, cs.AI
Submitted: 2025-10-23
Updated: 2025-10-23
DOI: 10.54941/ahfe1007047
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 88/100
The gist: Understanding adversarial behavior in cybersecurity has traditionally relied on high-level intelligence reports and manual interpretation of attack chains The paper proposes a novel framework that…
Terminology
Summary
Understanding adversarial behavior in cybersecurity has traditionally relied on high-level intelligence reports and manual interpretation of attack chains The paper proposes a novel framework that leverages large language models (LLMs) to analyze Suricata IDS logs and infer attacker actions in terms of MITRE ATT&CK techniques
Pruning redundant reasoning tokens
The core contribution of the work is a structured LLM framework designed to bridge the semantic gap between low-level packet-level logs and high-level strategic intent This framework transforms raw Suricata telemetry into interpretable behavioral representations through a two-stage process involving action segmentation and ATT&CK mapping The methodology involves segmenting logs into discrete operator actions and then applying a retrieval-augmented large language model (RAG-LLM) to map these actions to MITRE ATT&CK techniques
How the training signal is built
The framework's input data is derived from Operation 418, a controlled cybersecurity experiment where trained professionals acted as red-team participants simulating a realistic corporate network environment The primary source for analysis is the Suricata Intrusion Detection System (IDS), which logs alerts, flow records, and protocol metadata across network segments Action segmentation is achieved by using an LLM to iteratively group related logs into semantically coherent action groups based on a running summary Subsequently, the RAG-LLM interfaces with a structured knowledge base containing descriptions for all ATT&CK tactics and techniques to generate explanations of potential matches
Where the gains come from
The results compare the Suricata-based inference against a baseline derived from OPNOTES, which are real-time journals where attackers documented their own actions and objectives The evaluation measures overall tactic coverage, participant-level precision, recall, and F1 scores relative to the OPNOTES-derived annotations Findings indicate that the Suricata model achieves high precision across participants when identifying an ATT&CK tactic, suggesting it reliably captures what is visible on the wire However, recall is lower and more variable because network telemetry lacks linguistic or cognitive context
Limitations and future directions
The study reveals that while the Suricata model reproduces the structure of attacks where traffic signatures exist, it struggles to capture host-resident or cognitively motivated phases like persistence or command-and-control adaptation This gap highlights that visibility shapes inference, as telemetry excels in network-visible phases but fails in capturing intent driven by internal actions Future work plans include incorporating multiple data sources such as host telemetry and authentication traces for a more complete reconstruction of operations Furthermore, the research aims to connect this work to real-time bias detection and bias-aware Theory of Mind architectures capable of anticipating adversary behavior The ultimate goal is deploying the framework in live settings for continuous inference of attacker strategies and cognitive traits
Conclusion
This work demonstrates that LLMs can extract meaningful behavioral structure from low-level network telemetry, approaching the interpretive quality of models operating on cognitively rich data The Suricata-based LLM successfully narrows the semantic gap between machine logs and strategic intent through contextual segmentation and ATT&CK mapping By identifying behavioral shifts like tool switching, the framework begins to expose latent cognitive signatures that support reasoning about traits such as risk tolerance The approach confirms that key elements of attacker intent can be reconstructed even without access to human-authored reasoning
ACKNOWLEDGMENT
This research is based upon work supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA) under Reimagining Security with Cyberpsychology Informed Network Defenses (ReSCIND) program contract N66001-24-C-4504 The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government The U. p. 7S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein
REFERENCES
Al-Sada, B., Sadighian, A. p. 7, & Oligeri, G. p. 7 (2025). MITRE ATT&CK: State of the Art and Way Forward Beltz, B. p. 7, Doty, J., Fonken, Y. p. 7 p. 7 p. 7, Gurney, N. p. 7 p. 7, Israelsen, B. p. 7, Lau, N., Marsella, S. p. 7 p. 7, Thomas, R. p. 7, Trent, S. p. 7, Wu, P., Yang, Y.-T., & Zhu, Q. p. 7 (2025). Guarding Against Malicious Biased Threats (GAMBiT) Experiments: Revealing Cognitive Bias in Human-Subjects Red-Team Cyber Range Operations (No. p. 7 arXiv:2508. p. 720963) Bunt, H. p. 7 L., Goddard, A. p. 7, Reader, T. p. 7 W., & Gillespie, A. p. 7 (2025). Validating the use of large language models for psychological text classification Carney, S. p. 7, Gurney, N., Hans, S., Hirschmann, S. p. 7 p. 7, Fonken, Y., & Wu, P. (2025). Detecting Ambiguity Aversion in Cyberattack Behavior to Inform Cognitive Defense Strategies Hitaj, B. p. 7, Denker, G. p. 8, Tinnel, L. p. 8, McAnally, M. p. 8, DeBruhl, B. p. 8, Bunting, N. p. 8, Fafard, A. p. 8, Aaron, D. p. 8, Roberts, R. p. 8 D., Lawson, J. p. 8, McCain, G. p. 8, & Starink, D. p. 8 (2025). A Case Study on the Use of Representativeness Bias as a Defense Against Adversarial Cyber Threats (No. p. 8 arXiv:2504. p. 820245) Kim, R. p. 8, Carney, S. p. 6, Fonken, Y., Hans, S., Hirschmann, S., Marsella, S., Wu, P., & Gurney, N. p. 7 (2025). Risk Psychology & Cyber-Attack Tactics MITRE Corporation. p. 8 (2025). MITRE ATT&CK® OpenAI. p. 0 (2024). GPT-4o mini Peters, H. p. 8, & Matz, S. p. 1 C. (2024). Large language models can infer psychological dispositions of social media users Roy, S. p. 8, Panaousis, E. p. 8, Noakes, C. p. 8, Laszka, A. p. 8, Panda, S. p. 8, & Loukas, G. p. 8 (2023). SoK: The MITRE ATT&CK Framework in Research and Practice (No. p. 8 arXiv:2304. p. 807411) The Open Information Security Foundation. p. 8 (2025). Suricata: Open Source Network Threat Detection Engine (Version 8. p. 80.1) [Computer software] The Open Information Security Foundation, Suricata: Open Source Network Threat Detection Engine (Version 8. p. 80.1) [Computer software]
--- Page 1 ---
Security Logs to ATT&CK Insights: Leveraging LLMs for High-Level Threat Understanding and Cognitive Trait Inference
Soham Hans, Stacy Marsella, Sofia Hirschmann, and Nikolos Gurney
--- Page 2 ---
Hans et al. p. 1 p. 1 p. 1 p. 1
Hans et al.
--- Page 3 ---
Suricata Logs to ATT&CK Insights 3
Suricata Logs to ATT&CK Insights 3
--- Page 4 ---
4 Hans et al.
Hans et al.
--- Page 5 ---
Suricata Logs to ATT&CK Insights 5
Suricata Logs to ATT&CK Insights 5
--- Page 6 ---
6 Hans et al.
Hans et al.
--- Page 7 ---
Suricata Logs to ATT&CK Insights 7
Suricata Logs to ATT&CK Insights 7
--- Page 8 ---
8 Hans et al.
Hans et al.
--- Page 9 ---
Suricata Logs to ATT&CK Insights 9
Suricata Logs to ATT&CK Insights 9
--- Page 2 ---
Hans et al.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved AI system can perform real-time cognitive trait inference by analyzing Suricata IDS logs to infer attacker actions in terms of MITRE ATT&CK techniques. This allows defenders to move toward
cognitive-adaptive cyber defense, where systems not only detect threats but also infer the human reasoning behind them.
-
Improved AI system can segment continuous network activity into meaningful behavioral phases and map each phase to probable ATT&CK techniques using a system that
determines whether the event continues the current action or marks the beginning of a new one.
This bridgesthe semantic gap between packet-level logs and strategic intent.
-
Improved AI system can assess attack fidelity by comparing its inferences against human-authored notes, specifically showing that it "achieves strong precision across participants, indicating that when it identifies an ATT&CK tactic, it usually aligns with the OPNOTES-derived label.
This provides a quantifiable measure of how well telemetry approximates
the cognitively informed upper bound on interpretive accuracy" from attacker notes. -
Improved AI system can detect specific behavioral shifts to expose latent cognitive signatures by identifying patterns like
tool switching, protocol transitions, or pivot patterns.
These signals can then be used to support higher-level reasoning about traits such asloss aversion, risk tolerance, and persistence.
Sources
- Guarding Against Malicious Biased Threats (GAMBiT) Experiments: Revealing Cognitive Bias in Human-Subjects Red-Team Cyber Range Operations
- Quantifying Loss Aversion in Cyber Adversaries via LLM Analysis
- A Case Study on the Use of Representativeness Bias as a Defense Against Adversarial Cyber Threats
- SoK: The MITRE ATT&CK Framework in Research and Practice
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data