The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation
summary
The gist
The paper presents the first systematic measurement of procedural sufficiency in public ATT&CK-in-STIX to determine if descriptive threat intelligence provides enough detail to support multi-stage…
In short
The research measured if public threat intelligence (CTI) provides enough detail to actually execute multi-stage cyber attacks. The study found that while CTI describes adversary behavior, it lacks the necessary procedural steps like specific ordering and parameters needed for automated emulation. A new three-stage method was proposed to translate this descriptive intelligence into executable workflows.
Key concepts
- Procedural Sufficiency
- This measures whether threat intelligence contains enough explicit instructions—like exact sequences, required parameters, and preconditions—to allow a system to automatically run an attack simulation. The paper found that public CTI is insufficient because it describes 'what' happens but not precisely 'how' to execute the steps in order.
- ATT&CK-in-STIX
- This refers to using the MITRE ATT&CK framework structured within STIX data formats for threat intelligence. The study analyzed this structure to see if the descriptive behavioral information embedded in these structured feeds is adequate for complex, multi-stage adversary emulation.
- Three-Stage Methodology
- This is a proposed process to convert abstract CTI into runnable attack workflows. Stage 1 structures the data; Stage 2 translates abstract behaviors into minimal executable steps using templates; and Stage 3 integrates these steps into an execution environment based on analyst-provided parameters.
- Procedural Semantics Gap
- This is the core problem identified: the difference between knowing what an adversary does (descriptive intelligence) and having the precise procedural semantics needed to automate that action. Public CTI describes behaviors but fails to provide the concrete ordering, environment bindings, and explicit parameters required for direct execution.
Terminology used across episodes
This episode discusses
- The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation · Paper Radio
- Automated Repeatable Adversary Threat Emulation with Effects Language (EL)
- CTI-HAL: A Human-Annotated Dataset for Cyber Threat Intelligence Analysis
- RACONTEUR: A Knowledgeable, Insightful, and Portable LLM-Powered Shell Command Explainer
- KnowHow: Automatically Applying High-Level CTI Knowledge for Interpretable and Accurate Provenance Analysis
- Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks
- Perry: A High-level Framework for Accelerating Cyber Deception Experimentation
- MultiKG: Multi-Source Threat Intelligence Aggregation for High-Quality Knowledge Graph Representation of Attack Techniques
- From Sands to Mansions: Towards Automated Cyberattack Emulation with Classical Planning and Large Language Models
- ThreatPilot: Attack-Driven Threat Intelligence Extraction
The paper
The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation · Read on arXiv
Aeronautics Institute of Technology · Carnegie Mellon University
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "The Procedural Semantics Gap in Structured CTI".
Elias: Here's the gist of the paper. The paper presents the first systematic measurement of procedural sufficiency in public ATT&CK-in-STIX to determine if descriptive threat intelligence provides enough detail to support multi-stage adversary emulation.
Nadia: This research establishes a reproducible boundary between behavioral information encoded in public CTI and the explicit procedural steps that must still be provided prior to execution Measuring Procedural Sufficiency The core finding is that while structured CTI provides behavioral grounding, it lacks the procedural semantics required for direct multi-stage execution, specifically ordering, concrete parameters, preconditions, and environment bindings The analysis of the ATT&CK Enterprise bundle reveals that campaign objects encode fragmented behavioral segments where only 43.0% of techniques appear in at least one campaign Furthermore, neither clustering nor Longest Common Subsequence (LCS) analysis reveals reusable procedural structure across campaigns This indicates that public Cyber Threat Intelligence (CTI) describes what adversaries do, but not enough of how to automate those behaviors To bridge the gap between descriptive CTI and executable adversary-emulation workflows, the authors introduce a three-stage methodology combining structural CTI modeling, analyst-curated technique translation, and workflow integration in the MITRE Caldera framework Stage 1 performs automated structural conversion of CTI data from STIX to Caldera format while deriving a conservative tactic-ordered list for each campaign Stage 2 introduces a translation from abstract behavioral descriptions to minimal executable steps, often assisted by public templates or language models, which must be curated and validated against the source CTI Stage 3 integrates these steps into an emulation environment using Caldera to execute the translated workflow based on explicit parameters and assumptions supplied by an analyst Structural and Behavioral Analysis The authors quantitatively characterized the MITRE ATT&CK Enterprise bundle using four metrics: coverage, sparsity, overlap, and clustering The analysis showed that campaign heterogeneity is high, with only 43.0% of Enterprise techniques appearing in any campaign Clustering and LCS analyses indicated weak cluster separation, showing no stable, reusable behavioral families across campaigns Despite this lack of shared backbone, all 51 campaigns are distinguishable from one another by small positive witnesses of at most four techniques Intrusion sets account for 70.
Title and authors: Nadia: So we're looking at this paper called "The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation". It’s about figuring out if the threat intelligence we get from public sources, like ATT andCK in STIX format, is actually detailed enough to actually run an attack.
Elias: Yeah, it tackles that core question: does this descriptive intelligence give us enough detail for multi-stage emulation? It sets up a way to systematically measure that procedural sufficiency boundary.
Priya: From my side I’m wondering what they mean by "procedural sufficiency" in the context of emulation. Is it just about having the technique names there, or is it deeper than that?
Nadia: It goes deeper than just having the technique names; it’s about whether you can actually put them into a runnable attack chain. The authors are looking for the specific steps and ordering required for execution.
Elias: Exactly. They find that while structured CTI gives you behavioral grounding, it misses critical procedural elements like ordering, concrete parameters, preconditions, and environment bindings which you need to actually run something.
Priya: That sounds like a big gap between knowing *what* an attacker does and knowing *how* to automate it. So what is the main finding here about the quality of this intelligence?
Nadia: The main finding is that structured CTI has weak reusable procedural structure across different campaigns. They analyzed the ATT andCK Enterprise bundle, and only forty-three point zero percent of techniques appeared in at least one campaign object in their corpus <ref:2512.12078#pg1,the ATT&CK Enterprise bundle>.
Elias: And they looked at clustering and Longest Common Subsequence analysis, but those methods didn't reveal any stable, reusable procedural structure across the campaigns they studied. It shows that public CTI describes what adversaries do, but not enough of how to automate those behaviors.
Priya: So it’s like having a bunch of different recipes for making bread, but none of them share the same fundamental sequence or ingredient requirements for a whole batch?
Nadia: Kind of that. The authors found campaign heterogeneity is high, and despite this variety, all fifty-one campaigns were distinguishable from each other by very small positive witnesses, at most four techniques <ref:2512.12078#pg2>. Intrusion sets accounted for seventy percent of the analysis in their intrusion set findings.
Elias: That reinforces the idea that you can’t just pull a generalized procedure out of public feeds and expect it to work across different scenarios because the underlying structure isn't consistent enough. This distinction is important because they clarify that these claims apply specifically to public ATT andCK-in-STIX bundles, not necessarily proprietary ones.
Priya: So what does this mean for security teams who are trying to build automated defense validation tools? If the CTI isn't procedural enough, how do we actually test if our defenses can stop the actual steps?
Nadia: The paper introduces a three-stage methodology to bridge that gap. Stage one is automated structural conversion of CTI from STIX to Caldera format while deriving a conservative tactic-ordered list for each campaign.
Title and authors: Elias: Then stage two introduces the translation from those abstract behavioral descriptions into minimal executable steps, which they suggest can be aided by public templates or language models, but that translation has to be curated and validated against the source CTI.
Priya: And stage three is where they integrate these steps into an emulation environment using Caldera to execute the translated workflow based on explicit parameters and assumptions supplied by an analyst. That sounds like a lot of manual work upfront.
Nadia: It does require explicit planning, parameter binding, and a declared System Under Test or SUT for that execution substrate thirty-eight forty-seven <ref:2512.12078#pg2,explicit planning, parameter binding, and a declared System Under Test>. The paper argues that this translation into runnable procedures is still necessary even with the structured CTI they are using.
Elias: So the core improvement they propose is moving from just descriptive mapping to creating a system that can translate that description into actual, executable adversary emulation workflows within Caldera. They’re trying to define what procedural detail is missing and how we can supply it manually.
Priya: That shifts the focus from just consuming threat intelligence to actively engineering an emulation pipeline where the gap between description and execution is explicitly addressed by analyst input. It’s about validation, not just mapping.
Nadia: Exactly. We’re moving toward a system that measures procedural sufficiency directly by testing if workflows grounded in CTI and completed through analyst-supplied bindings, parameters, and local execution assumptions can yield internally consistent execution chains in a controlled environment.
Elias: That moves the research from just measuring what's missing to proposing how to bridge that gap using structured modeling and translation layers. It’s a framework for making sense of the fragmentation in public feeds like the ATT andCK Enterprise bundle.
Priya: So, if you only listen to this show, you should understand that the problem isn't just that we don't have enough data; it’s that we lack a standardized way to turn that descriptive data into actionable execution steps.
Nadia: That’s the central message of "The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation". They show us exactly where the gap is and how to start building tools to measure it.
Elias: It highlights that relying solely on structured CTI for automated emulation is insufficient because the procedural semantics are still weak, so you need that explicit planning and parameter binding.
Priya: It’s a necessary step toward making threat intelligence truly useful for automated defense validation by demanding more than just coverage mapping from public sources.
Nadia: That’s what we have on this paper. It sets the stage for how we can start testing if the behavioral information encoded in public CTI is sufficient to support multi-stage adversary emulation, and where it stops providing detail.
The paper's summary: Nadia: So we're talking about this paper now, "The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation." It’s about finding out if the threat intelligence we have from public sources is actually detailed enough to run a real attack.
Elias: Yeah, it sets up a way to systematically measure that procedural sufficiency boundary. It’s trying to find the exact point where descriptive intelligence stops being useful and when you need explicit steps before you can even think about doing anything.
Priya: I'm thinking about what they mean by procedural sufficiency in the context of an attack simulation. Is it just having the technique names there, or is it deeper than that?
Nadia: It goes deeper than just having the technique names; it’s about whether you can actually put those techniques into a runnable attack chain. The authors are looking for the specific ordering, concrete parameters, preconditions, and environment bindings that you need to actually execute something.
Elias: Exactly. They find that while structured CTI gives you behavioral grounding—the 'what'—it lacks the procedural semantics—the 'how'—required for direct multi-stage execution.
Priya: That sounds like a big gap between knowing *what* an attacker does and knowing *how* to automate it. So what is the main finding here about the quality of this intelligence?
Nadia: The main finding is that structured CTI has weak reusable procedural structure across different campaigns. They looked at the ATT andCK Enterprise bundle, and only forty-three point zero percent of techniques appeared in at least one campaign object in their corpus.
Elias: And they looked at clustering and Longest Common Subsequence analysis, but those methods didn't reveal any stable, reusable procedural structure across the campaigns they studied. It shows that public CTI describes what adversaries do, but not enough of how to automate those behaviors.
Priya: So it’s like having a bunch of different recipes for making bread, but none of them share the same fundamental sequence or ingredient requirements for a whole batch?
Nadia: Kind of that. The authors found campaign heterogeneity is high, and despite this variety, all fifty-one campaigns were distinguishable from each other by very small positive witnesses, at most four techniques. Intrusion sets accounted for seventy percent of the analysis in their intrusion set findings.
Elias: That reinforces the idea that you can’t just pull a generalized procedure out of public feeds and expect it to work across different scenarios because the underlying structure isn't consistent enough. This distinction is important because they clarify that these claims apply specifically to public ATT andCK-in-STIX bundles, not necessarily proprietary ones.
Priya: So what does this mean for security teams who are trying to build automated defense validation tools? If the CTI isn't procedural enough, how do we actually test if our defenses can stop the actual steps?
Nadia: The paper introduces a three-stage methodology to bridge that gap. Stage one is automated structural conversion of CTI from STIX to Caldera format while deriving a conservative tactic-ordered list for each campaign.
Elias: Then stage two introduces the translation from those abstract behavioral descriptions into minimal executable steps, which they suggest can be aided by public templates or language models, but that translation has to be curated and validated against the source CTI.
Priya: And stage three is where they integrate these steps into an emulation environment using Caldera to execute the translated workflow based on explicit parameters and assumptions supplied by an analyst. That sounds like a lot of manual work upfront.
Nadia: It does require explicit planning, parameter binding, and a declared System Under Test or SUT for that execution substrate. The paper argues that this translation into runnable procedures is still necessary even with the structured CTI they are using.
Elias: So the core improvement they propose is moving from just descriptive mapping to creating a system that can translate that description into actual, executable adversary emulation workflows within Caldera. They’re trying to define what procedural detail is missing and how we can supply it manually.
Priya: That shifts the focus from just consuming threat intelligence to actively engineering an emulation pipeline where the gap between description and execution is explicitly addressed by analyst input. It’s about validation, not just mapping.
Nadia: Exactly. We’re moving toward a system that measures procedural sufficiency directly by testing if workflows grounded in CTI and completed through analyst-supplied bindings, parameters, and local execution assumptions can yield internally consistent execution chains in a controlled environment.
Elias: That moves the research from just measuring what's missing to proposing how to bridge that gap using structured modeling and translation layers. It’s a framework for making sense of the fragmentation in public feeds like the ATT andCK Enterprise bundle.
Priya: So, if you only listen to this show, you should understand that the problem isn't just that we don't have enough data; it’s that we lack a standardized way to turn that descriptive data into actionable execution steps.
Nadia: That’s what we have on this paper. It sets the stage for how we can start testing if the behavioral information encoded in public CTI is sufficient to support multi-stage adversary emulation, and where it stops providing detail.
Elias: We'll talk more about those three stages in the next segment, focusing on exactly how that translation layer works.
The paper's improvements: Tom: We're moving on to what these authors suggest we do next, which is actually proposing three stages to bridge that gap we just talked about between description and execution.
Nadia: Right, so they’re not just stopping at identifying the problem; they’re giving us a roadmap. Stage one involves automated structural conversion of CTI data from STIX format into Caldera format while deriving a conservative tactic-ordered list for every campaign object.
Elias: That sounds like turning the raw intelligence into something that actually fits into their emulation framework, which is pretty cool because it handles the initial mapping automatically. But what about the second stage?
Priya: What about the second stage? Is that where they do the actual thinking, or is it just more automation? I mean, if we're translating abstract behaviors into steps, how much human curation does that actually require?
Nadia: Stage two introduces a translation from those abstract behavioral descriptions to minimal executable steps. They say this often involves using public templates or language models to do the heavy lifting, but it has to be curated and validated against the source CTI.
Elias: So they’re saying AI can help with that translation, but you still need a human validating that translation because the source intelligence is what's actually driving the attack. That makes sense from a cryptographer's view—the assumptions in the translator matter a lot.
Priya: And stage three? What happens after we have these minimal executable steps? How do we actually test if they work in a real environment, or at least a controlled one?
Nadia: Stage three integrates those steps into an emulation environment using Caldera to execute the translated workflow. But this execution relies on explicit parameters and assumptions supplied by an analyst.
Elias: So you’re saying that even with all this automated translation, the final execution still needs that analyst input—those bindings and local execution assumptions—to make it work. That’s a big caveat they put there, right?
Priya: It means the system isn't fully autonomous yet; it requires a human to provide the specific context for success. So, what does this change for someone who just listens to the show?
Nadia: It changes things by showing us that we can’t just rely on automated mapping anymore. We have to build a pipeline where we explicitly define those procedural gaps ourselves through analyst input and parameter binding.
Elias: Exactly. It moves the goal from passively consuming data to actively engineering a workflow where the gap between what's described and what executes is filled in by explicit, validated steps.
Priya: So the improvement is essentially moving from just measuring sufficiency to creating a system that tests if workflows grounded in CTI can yield internally consistent execution chains under those analyst-supplied constraints.
Nadia: That’s the core idea. It stops being just about how much data we have and starts being about how much procedural detail we can force the intelligence into to actually run an attack simulation.
Elias: And it sets up a clear path for future work, looking at how to make that translation layer even more robust so it doesn't break when the source CTI changes.
Conclusion: Tom: So we're wrapping up this look at "The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation." It boils down to how much descriptive threat intelligence actually lets you build a working attack simulation versus just looking at the names of techniques.
Nadia: Exactly. The central finding is that public structured CTI gives you behavioral grounding, but it lacks the procedural semantics—the ordering and concrete parameters—you need for multi-stage execution.
Elias: Right, they quantified this by showing that clustering and LCS analysis didn't find any stable reusable structures across campaigns, meaning those techniques are just fragmented.
Priya: So what does this actually mean for us on the ground? If the intelligence isn't procedural enough, how does that stop us from actually testing our defenses?
Nadia: It means we can’t just take the CTI and run it; we have to introduce an explicit translation layer. They propose a three-stage methodology to turn that description into something executable.
Elias: The improvement they suggest is basically building that pipeline: first, structural conversion, then translating abstract behavior into minimal steps, and finally integrating those steps into Caldera execution based on analyst input.
Priya: What I see here is that even with all this modeling and AI translation, the final success still depends on the analyst supplying those specific bindings and assumptions for the execution substrate.
Nadia: That’s right. It's not a fully automated system yet; it requires that explicit planning from a human to yield internally consistent execution chains in that controlled environment.
Elias: So, they’re defining exactly what procedural detail is missing and showing us how we can supply it manually to fill that gap. That’s a significant step toward making CTI truly actionable for emulation.
Priya: It shifts the focus from just mapping data points to actively engineering an emulation pipeline where the description and the execution are explicitly linked by analyst context.
Nadia: Exactly. This paper shows us precisely where we need more detail in our threat intelligence sources if we want to build better automated defense validation tools.
Elias: It’s a solid framework for measuring that gap, and it gives us concrete metrics like coverage and sparsity to track where the weakness is most pronounced.
Priya: I just think the limitation they flag is that this entire process still needs that analyst input; if you skip that step, the execution chain breaks.
Nadia: True. So for anyone listening, remember this paper as a blueprint for moving beyond simple coverage mapping toward building executable adversary emulation workflows.
Elias: We'll keep an eye on how the next research builds on this procedural translation layer we just discussed.
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18