STRIATUM-CTF: A Protocol-Driven Agentic Framework for Autonomous CTF Solving

summary

The gist

Large Language Models (LLMs) have demonstrated potential in code generation, yet they struggle with the multi-step…

In short

STRIATUM-CTF is a framework that uses Large Language Models (LLMs) to solve Capture-the-Flag (CTF) challenges by separating reasoning from execution. It introduces the Model Context Protocol (MCP), which acts as a strict validator, ensuring LLM outputs are correctly formatted before tools run. This system significantly reduces errors and hallucinations compared to naive prompting, proving that structured protocols make autonomous cyber operations reliable.

Key concepts

Model Context Protocol (MCP)
A standardized layer that sits between the LLM's reasoning and the security tools being executed. It acts as a strict schema validator, ensuring every command sent to a tool is correctly formatted. This prevents the LLM from generating broken commands or hallucinating invalid options, forcing its output onto valid execution paths.
Neurosymbolic Architecture
A framework that combines neural reasoning (like an LLM planning strategy) with symbolic constraints (like strict rules). The paper uses this to ensure the high-level strategic thinking from the LLM is grounded by deterministic, rule-based checks before any action is taken. This combination makes the agent more reliable than a purely probabilistic system.
Protocol-Mediated Grounding
The process where tool calls are validated by the MCP layer instead of being executed directly. By requiring all tool interactions to pass through this protocol, the system maintains state coherence. It allows the agent to parse structured feedback from tools and self-correct its plan based on verified, concrete results rather than relying solely on potentially drifting context.
POMDP Formulation
Modeling the CTF challenge as a Partially Observable Markov Decision Process. This mathematical approach helps define the environment state (what is hidden) and the agent's actions. The framework uses this to guide its planning, ensuring it considers uncertainty and maintains a coherent history of observations while trying to maximize the reward of capturing the flag.

Terminology used across episodes

This episode discusses

The paper

STRIATUM-CTF: A Protocol-Driven Agentic Framework for Autonomous CTF Solving · Read on arXiv

James Hugglestone, Samuel Jacob Chacko, Dawson Stoller, Ryan Schmidt, Xiuwen Liu

Florida State University

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "STRIATUM-CTF: A Protocol-Driven Agentic Framework for Autonomous CTF Solving".

Elias: Here's the gist of the paper. Large Language Models (LLMs) have demonstrated potential in code generation, yet they struggle with the multi-step, stateful reasoning required for offensive cybersecurity operations The system secures First Place outperforming 21 human teams and demonstrating strong adaptability in a dynamic problem-solving setting This paper introduces STRIATUM-CTF, a modular agentic framework built upon the Model Context Protocol (MCP) to solve CTF problems The Core Architecture STRIATUM-CTF is a neurosymbolic agentic framework that decouples neural reasoning from tool execution using the Model Context Protocol (MCP) for solving CTF problems The architecture consists of three distinct layers: the Reasoning Layer, the Protocol Layer, and the Execution Layer The Reasoning Layer utilizes Claude Sonnet 4.5 as a central planner to generate high-level strategies but is physically isolated from the operating system to prevent direct execution of hallucinated commands The Protocol Layer functions as a schema validator that enforces strict typing on all function calls, effectively collapsing the model’s output manifold onto valid execution paths The Execution Layer consists of containerized security tools like Angr, Ghidra, and GDB wrapped in MCP servers to execute in a secure sandbox Protocol-Mediated Grounding The Model Context Protocol (MCP) is central to reducing hallucinations by standardizing tool interfaces for system introspection, decompilation, and runtime debugging The architecture follows a recursive four-step control loop designed to maintain state coherence across complex attack chains This lifecycle involves Plan, Execute, Parse, and Iterate steps During the execution phase, the agent selects a tool and sends a JSON payload that must pass through the Protocol Layer for validation The system parses structured error streams to self-correct failed attempts while filtering irrelevant data to conserve context tokens Empirical Validation and Performance The system was validated in a live competitive environment where it operated autonomously to identify and exploit vulnerabilities in real-time The results from the experimental runs showed that the Baseline configuration achieved the highest success rate at 86.7% The Minimal condition, with only 55 lines of MCP tool definitions and no guidance documentation, achieved a 77.8% success rate This suggests that neuro-symbolic grounding via MCP is a more token-efficient scaling law than context-based prompting Case studies revealed that the value of structured documentation is challenge-dependent, with the greatest benefits observed for vulnerability classes requiring multi-step reasoning or unconventional approaches Future Directions and Bottlenecks Current architectures face hurdles regarding long-horizon planning and token efficiency, which are addressed by developing a hierarchical context management strategy This strategy involves deploying a lightweight, local LLM to act as a semantic filter to dynamically compress tool descriptions and maintain session state locally Future work will focus on two key architectural enhancements: proposing a domain-specific primitive layer to map high-level reasoning to granular, security-oriented operations Additionally, the team intends to implement a hierarchical planning framework by decomposing complex penetration objectives into manageable subgoals to reduce computational search depth and mitigate error propagation The paper concludes that protocol-mediated grounding is a viable solution to the model hallucination problem STRIATUM-CTF establishes a new baseline for agentic security systems by leveraging general-purpose reasoning models within a strict symbolic constraint layer The empirical results confirm that this neuro-symbolic approach significantly outperforms human baselines in speed and precision, particularly for tasks aligning with semantic tool primitives STRIATUM-CTF marks a critical step toward the next generation of robust, autonomous cyber-operations The system demonstrates the competency of top students with extensive CTF experience The system achieves 215 points in a live competition, exceeding human scores by a factor of 3.6× This work confirms that protocol-mediated grounding is a viable solution to the model hallucination problem The paper establishes STRIATUM-CTF as the next baseline for autonomous cyber-operations Page 1 --- STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving James Hugglestone∗, Samuel Jacob Chacko∗, Dawson Stoller, Ryan Schmidt, Xiuwen Liu Department of Computer Science, Florida State University, Tallahassee, USA Email: jah21e@fsu.edu, sj21j@fsu.edu, dawson.stoller@aol.com, res22f@fsu.edu, xliu@fsu.edu ∗These authors contributed equally.

Nadia: Abstract—Large Language Models (LLMs) have demonstrated potential in code generation, yet they struggle with the multi-step, stateful reasoning required for offensive cybersecurity operations Existing research often relies on static benchmarks that fail to capture the dynamic nature of real-world vulnerabilities In this work, we introduce STRIATUM-CTF (A Search-based Test-time Reasoning Inference Agent for Tactical Utility Maximization in Cybersecurity), a modular agentic framework built upon the Model Context Protocol (MCP) By standardizing tool interfaces for system introspection, decompilation, and runtime debugging, STRIATUM-CTF enables the agent to maintain a coherent context window across extended exploit trajectories We validate this approach not merely on synthetic datasets, but in a live competitive environment Our system participated in a university-hosted Capture-the-Flag (CTF) competition in late 2025, where it operated autonomously to identify and exploit vulnerabilities in real-time STRIATUM-CTF secured First Place, outperforming 21 human teams and demonstrating strong adaptability in a dynamic problem-solving setting We analyze the agent’s decision-making logs to show how MCP-based tool abstraction significantly reduces hallucination compared to naive prompting strategies These results suggest that standardized context protocols are a critical path toward robust autonomous cyber-reasoning systems Index Terms—Autonomous cyber-reasoning, Model context protocol, LLM agents, Tool-augmented generation, Offensive cybersecurity.

Elias: I.

Nadia: INTRODUCTION Offensive security assessments, including penetration testing, red-teaming, and Capture-the-Flag (CTF) competitions remain predominantly manual and labor-intensive endeavors To successfully compromise a target, analysts must execute a complex, non-linear workflow: enumerating vast attack surfaces, interpreting verbose scanner outputs, selecting appropriate exploitation primitives, and iteratively tuning payloads to bypass defenses While powerful tools exist for specific phases (e.g., Nmap, Ghidra, Hashcat), the orchestration of these tools remains a cognitive bottleneck Human operators must act as the “glue,” manually parsing data from one tool to feed into another, a process that is prone to errors and cognitive biases such as confirmation bias

1: , and is increasingly difficult to scale with the rapid cadence of modern software delivery pipelines As system complexity grows, the sheer volume of alerts and the requirement for continuous context switching lead to alert fatigue, degrading the operator’s ability to maintain high-fidelity mental models of the target environment

2: The emergence of Large Language Models (LLMs)

3: offers a potential paradigm shift Their ability to process unstructured text and generate code makes them well-suited to automate the high-level reasoning required to connect these disparate tools However, early attempts to build autonomous security agents have faced significant architectural hurdles

4: Existing methods largely fall into two categories: “copilots” that require constant human prompting, or “naive autonomous agents” that simply pipe model output into a shell

5: Recent evaluations of these naive approaches reveal critical vulnerabilities beyond simple hallucination For instance, Inter-Agent Trust Exploitation

6: has shown that LLMs often blindly execute malicious payloads if they originate from peer agents, lacking the introspection to verify the origin of a command Furthermore, while models like GPT-4o

7: excel at high-level planning, they suffer from a Reasoning-Action Gap

8: , where the model correctly identifies a vulnerability (e.g., SQL injection) but fails to generate the syntactically correct payload to exploit it due to a lack of grounded environment feedback, frequently hallucinating non-existent command-line flags or software packages, a failure mode that introduces downstream supply-chain risks such as slopsquatting

9: This disconnect is deepened in long-horizon tasks, where “context drift” causes the agent to lose track of early reconnaissance data

10: , further observed in recent surveys on LLM hallucinations in cybersecurity

11: leading to loops of redundant, failed attempts To address these limitations, we introduce STRIATUMCTF (A Search-based Test-time Reasoning Inference Agent for Tactical Utility Maximization in Cybersecurity), a neurosymbolic agentic framework that decouples neural reasoning from tool execution using the Model Context Protocol (MCP)

12: By formalizing the interface between the LLM and the execution environment, we transform the model from a passive text generator into an active system operator Internally, the agent is equipped with a curated arsenal of offensive capabilities, including symbolic execution via Angr

13: , static analysis with Ghidra

14: , and runtime debugging through GDB Externally, a containerized environment exposes industry-standard reconnaissance and exploitation utilities such as Nmap, Nuclei, FFUF, and TLSX, allowing the agent to interact with the target system safely and reproducibly Upon receiving a high-level objective, STRIATUM-CTF autonomously formulates a multi-step plan invokes tools via structured MCP calls, parses standard error streams to self-correct failed attempts, and iterates until the objective is achieved or the search space is exhausted Page 2 --- invokes tools via structured MCP calls, parses standard error streams to self-correct failed attempts, and iterates until the objective is achieved or the search space is exhausted This paper makes the following contributions to the field of autonomous cyber-operations: 1) A Protocol-Driven Agentic Architecture: We propose a novel framework that utilizes the Model Context Protocol to standardize LLM interactions with offensive security tools, reducing hallucinations by collapsing the output manifold via strict schema compliance for tool invocation, and enabling robust error recovery through structured feedback loops 2) Integration of Specialized Analysis Primitives: We integrate complex, domain-specific tooling, including symbolic execution (Angr) and dynamic debugging (GDB), within an LLM’s context window, allowing deep binary analysis and exploit generation beyond simple scripting and 3) Validation via Live Competition: We empirically demonstrate the system’s efficacy with a first-place victory in a live, multi-team Capture-the-Flag competition, providing a rigorous case study of autonomous agent performance under time constraints and dynamic uncertainty compared to human baselines The code and data will be made available upon acceptance.

Paper summary: Nadia: So, this paper introduces STRIATUM-CTF, which is this modular agentic framework built on something called the Model Context Protocol or MCP. Basically, they’re tackling the problem that large language models are good at generating code but they struggle with the multi-step reasoning needed for actual offensive security work.

Elias: Yeah, the core idea here is decoupling that neural reasoning from the actual tool execution so you don't just pipe model output straight into a shell, which usually leads to really brittle commands and hallucinations.

Nadia: Right, and they frame it as a system that uses this MCP to standardize how the LLM talks to tools like Angr or Ghidra. It’s about making sure the model only calls things that are actually valid functions, which helps keep the context window coherent over long tasks.

Priya: So, what’s important for us here is how they handle that "context drift" problem you mentioned earlier, where an agent forgets stuff early on in a long exploit chain because it’s just filling up the token space.

Elias: Exactly. The paper claims this architecture uses a recursive four-step loop: Plan, Execute, Parse, and Iterate to maintain that state coherence across complex attack chains.

Nadia: And they’re showing off validation not just on synthetic data but in a live competitive environment where the system secured first place against more than thirty registered teams at Florida State University.

Conclusion: Nadia: So, looking at STRIATUM-CTF, it’s really about moving beyond just asking the LLM for answers and building a structured system where the model has to follow strict rules before it can touch any actual security tools.

Elias: The authors, Hugglestone and Chacko along with Stoller and Schmidt and Liu, they built this framework using that Model Context Protocol to enforce schema compliance on tool calls. That means the LLM output manifold gets collapsed onto a set of valid execution paths before anything runs in the sandbox.

Priya: What this means for us is that instead of hoping the model generates a perfect exploit command, you’ve got this symbolic constraint layer checking it before it even leaves the agent's planning stage. It’s about reducing those hallucinations we see everywhere in security tasks.

Nadia: It suggests that protocol-mediated grounding, as they call it, is a way to make these autonomous agents more predictable and reliable for real-world operations, especially when you need them to maintain state over extended exploit trajectories.

Elias: Yeah, the results show that even a minimal configuration with only fifty five lines of tool definitions and no extra guidance documentation achieved a seventy seven point eight percent success rate on one of their controlled studies. That points toward schema-mediated grounding being a token efficient way to scale this kind of agent.

More episodes

← Home