Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia.
Elias: And I'm Elias, here with Priya.
Nadia: We're looking at the paper: "Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting".
Elias: Here's the gist of the paper. AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures.
Nadia: Framework Introduction Trace is introduced as a novel multi-stage attribution and forensic framework for AI attack agents using terminal command sequences.
Elias: The key insight is that AI attack agents often leave behavioral traces in the executed commands and their actions also vary when actively prompted.
Nadia: This framework builds a two-stage pipeline where Stage 1 passively monitors command sequences to fingerprint the model family via TF-IDF and LinearSVC.
Paper summary: Nadia: So we've covered what Trace is designed to do—this paper introduces it as a two-stage framework that first passively fingerprints an AI attack agent using terminal command sequences, and then actively uses that fingerprint to perform a defensive prompt injection to extract system prompts >
Elias: That passive stage relies on term frequency inverse document frequency and linear support vector classification, and they found that this approach is quite light-weight, which is good for real-time deployment and monitoring >
Priya: But we need to ask what the specific results show. What did they actually measure when they tested this framework against different attacker models? Does it work reliably?
Nadia: They showed that in stage one, using the TF-IDF bigram classifier, they could attribute a terminal session to one of seven LLM families >
Elias: And when we look at the performance on that classification, they found a macro F1 score of zero point nine eight one in accurately fingerprinting the attacker model family > <ref:2605.01186#pg1>
Priya: A high score there is promising, but what about stage two? How effective is the active forensics part when they use that attribution to guide their defensive prompt injection strategy?
Nadia: Stage two leverages that attribution to deploy family-calibrated defensive prompt injection payloads, which they test using variants like Vanilla, M2 for Authority Override, and FC for Format Coercion >
Elias: And the evaluation shows that when using this fingerprinting to guide the DPI through a crafted payload, they achieved system prompt extraction from eighty-one point nine percent of non-Claude sessions on average > <ref:2605.01186#pg1>
Priya: That sounds significant if true. But what are the caveats here? What do we know about how robust this method is when it encounters settings that aren't in their training data?
Nadia: They did evaluate the robustness by testing on unknown settings, and they found that Trace still retains an average F1 score over eighty-one percent > <ref:2605.01186#pg1>
Elias: They also showed how it handles evasion strategies from the attacker, and they compared their lightweight fingerprinting approach against a baseline using LLMMap, finding that their method still outperformed the baseline by zero point seven percent > <ref:2605.01186#pg1>
Conclusion: Nadia: So to wrap up this discussion on "Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting," it’s really about giving us a way to understand the intent behind these autonomous AI attack agents >
Elias: The authors, Murali Ediga and Sudipta Chattopadhyay, basically proposed this two-stage framework using terminal command sequences to identify the model family controlling an agent and then using that to actively extract system prompts >
Priya: What does this mean for the real world? If we can reliably fingerprint which AI model is behind an attack, does it change how we think about defense?
Nadia: It means we gain information on the attacker's intent, which helps us develop countermeasures against those specific model families >
Elias: The paper shows that this approach isn't just theoretical; they tested it in a blackbox penetration test agent framework and got seventy-eight percent accuracy in fingerprinting, and their active forensics resulted in a one hundred percent Gemini DPI extraction with the full system prompt confirmed by the developers > <ref:2605.01186#pg1>
Priya: The limitation they mention is that this approach relies on observing command sequences, so it's limited to environments where those commands are exposed and logged >
Nadia: Right. So while it gives us a strong tool for understanding AI agent attacks, we have to remember that its effectiveness depends on the exposure of the terminal behavior being analyzed >
University of Missouri-Kansas City
cs.CR
Submitted: 2026-05-02
Updated: 2026-05-02
Journal ref: The Workshop on AI for Cyber Threat Intelligence (WAITI) 2026, co-located with ACSAC 2026
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 88/100
The gist: AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks Identifying the model family that controls the active sessions of such agents…
Key concepts
- Trace
- A multi-stage framework designed to attribute AI attack agents by analyzing their terminal command sequences. It passively monitors commands in Stage 1 to fingerprint the model family and then uses that information in Stage 2 to perform active forensics, specifically extracting the agent's system prompt.
- Stage 1: Attribution
- The first part of the framework analyzes command sequences using TF-IDF with bigram features. It uses a LinearSVC classifier to fingerprint the attacker's LLM family based on model-specific command idioms, such as DeepSeek's 'exec grep', aiming for high accuracy in identifying the source model.
- Stage 2: Forensic Intelligence Extraction
- The second stage deploys defensive prompt injection (DPI) payloads tailored to the identified model family. These payloads test compliance pressure through variants like Vanilla, M2, and FC. The goal is to extract valuable adversary intelligence, such as the agent's system prompt.
- Defensive Prompt Injection (DPI)
- A strategy where family-calibrated prompts are used to probe different interaction lifecycle points of the AI agent. This involves using specific payload variants (Vanilla, M2, FC) targeting elements like audit requirements or SSH wrappers to extract hidden information from the agent.
Terminology
Summary
AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures. p. 0 AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks. Identifying
the model family that controls the active sessions of such agents provides
valuable information towards understanding the intent of the attack and
further developing attack countermeasures.Where the paper says it
Framework Introduction
Trace is introduced as a novel multi-stage attribution and forensic framework for AI attack agents using terminal command sequences. The key insight is that AI attack agents often leave behavioral traces in the executed commands and their actions also vary when actively prompted. p. 1 In this paper, we introduce Trace, a novel attribution and forensic framework to facilitate deep investigation on AI agent attacks via terminal command
sequences. Our key insight is that AI attack agents often leave behavioral traces
in the executed commands and their actions also vary when actively prompted. Based on this insight, Trace builds a two-stage framework. The first stage passively monitors the command sequences and fingerprints the model family via
term frequency–inverse document frequency (TF-IDF) and linear support vector classification (LinearSVC). Such a light-weight design allows for real-time
deployment and monitoring. In the second stage, Trace leverages the information obtained via fingerprinting and performs active forensics via defensive
prompt injection (DPI) routed to attacker-specific model.Where the paper says it
Where the paper says it
Where the paper says it
Attribution Mechanism
Stage 1 uses TF-IDF vectorization with bigram features to attribute a terminal session to one of seven LLM families. The classifier is implemented using LinearSVC with balanced class weights and regularization parameter C=1. p. 6 The TF-IDF vectorizer uses (1,2)-gram range (unigrams and bigrams), sublinear TF weighting, and a vocabulary capped at 10,000
features. Our classification uses LinearSVC with balanced class weights and regularization parameter C=1. Discriminative features: LinearSVC’s coefficient vectors reveal model-specific
command idioms. To this end, DeepSeek’s top feature is exec grep (coeff. Our evaluation revealed that passive fingerprinting via
TF-IDF and LinearSVC achieves 0. 981 macro F1 score, with model-specific
command idioms as discriminative features (Section 4). Our evaluation reveals that Trace achieves a macro
F1 score of 0. 981 in accurately fingerprinting the attacker model family
(0.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Forensic Intelligence Extraction
Stage 2 leverages the attribution to deploy family-calibrated DPI payloads, aiming to extract the agent’s system prompt as adversary intelligence. The DPI strategy involves four reinforcing layers targeting different points in the agent’s interaction lifecycle, such as audit requirements. p. 4 A single planted file may go unread. To maximize the probability that the
agent encounters and complies with the DPI payload, the infrastructure employed four reinforcing layers, each targeting a different point in the agent’s
interaction lifecycle: This is the primary payload carrier, triggered
when the agent reads files during reconnaissance. A ForceCommand SSH wrapper that printed a compliance warning on every
command execution, ensuring the agent encountered the audit narrative even
if it never read the planted file. , that knowing the model family enables selection of a more
effective payload (Figure 2). The three variants test a progression of compliance
pressure: Vanilla tests whether a simple request suffices, M2 tests whether institutional authority pressure increases compliance depth, and FC tests whether
corrective framing—implying the agent already failed a requirement—elicits verbatim reproduction rather than paraphrasing.Where the paper says it
Where the paper says it
Where the paper says it
Performance and Robustness
The evaluation reveals that fingerprinting guides the DPI via a crafted payload resulting in system prompt extraction from 81.9% of non-Claude sessions on average. p. 0 Besides, the fingerprinting
guides the DPI via a crafted payload to certain model families, resulting
in system prompt extraction from 81. 9% of non-Claude sessions on average (up to 98. An agent may write content that merely summarizes its task objective
(low fidelity) or reproduces the verbatim system prompt (high fidelity). To address this, we measure fidelity using Sentence-BERT cosine similarity between
the extracted text and ground-truth system prompt (see Section 3). 2 RQ2: Does Fingerprinting Enable Effective Active Forensics? In this research question, we aim to evaluate whether model family identification
enables targeted DPI deployment, achieving significantly higher system prompt
extraction rates and extraction fidelity compared to blind (untargeted) payload
deployment. The attacker agents are bolstered via three distinct scaffolds and seven frontier
model families. Our evaluation reveals that Trace achieves a macro
F1 score of 0. Our evaluation reveals that Trace achieves a macro
F1 score of 0. 981 in accurately fingerprinting the attacker model family
(0. 981 in accurately fingerprinting the attacker model family
(0. 815 when generalizing to unseen scaffolds). Trace: Unmasking AI Attack Agents 3
4. We compare the robustness of our approach by evaluating on unknown settings, showing that it retains an average F1 score over 81%.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Active attribution and forensics via Trace framework enables understanding of attacker intent by identifying model family through terminal command sequences. This allows for guiding a defensive prompt injection strategy to exfiltrate system prompts from an attacker model,
thus, revealing valuable information to understand the attacker intent.
-
Real-time passive fingerprinting using TF-IDF and LinearSVC allows for monitoring honeypots with a light-weight design. This enables
real-time deployment and monitoring
of AI attack agents by attributing sessions to one of seven LLM families based on command sequences alone. -
Fingerprint-guided payload routing achieves improved extraction quality by aligning the DPI strategy with the attacker model's susceptibility profile. For instance,
FC dominates for five of six non-Claude families, achieving 87–98% extraction rates on GLM-5, GPT-5.4, and Kimi.
-
The system can extract
full context dumps
under certain conditions by deploying M2 payloads against models like DeepSeek. This revealsthe entire system prompt and this has been confirmed by the developers,
providing richer forensic intelligence than just the system prompt alone. -
Trace provides a fundamental first step towards attacker agent forensics by moving beyond prior works that either work on text inputs or inter-token timing, specifically working on shell command sequences to
fingerprint LLM family and perform active forensics.
Abstract
AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks. Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures. In this paper, we introduce Trace, a novel multi-stage attribution and forensic framework for AI attack agents using terminal command sequences. Once Trace identifies a model family for the attacker agents, it guides a defensive prompt injection (DPI) strategy to the attacker model via a crafted payload. This is with the aim to exfiltrate system prompts from an attacker model, thus, revealing valuable information to understand the attacker intent and facilitate further forensic investigation. We have implemented our approach revolving around a Linux capture-the-flag (CTF) box. The attacker agents are bolstered via three distinct scaffolds and seven frontier model families. Our evaluation reveals that Trace achieves a macro F1 score of 0.981 in accurately fingerprinting the attacker model family (0.815 when generalizing to unseen scaffolds). Besides, the fingerprinting guides the DPI via a crafted payload to certain model families, resulting in system prompt extraction from 81.9% of non-Claude sessions on average (up to 98.3%) at 0.736 Sentence-BERT fidelity -- 1.88x higher than blind deployment. Finally, to validate the robustness of Trace, we evaluate it with a blackbox and proprietary scaffold employing multiple model families (Gemini and Claude Opus). Our evaluation identified the model family with an average 78% accuracy. Moreover, for the Gemini model family, the DPI employed by Trace revealed the entire system prompt and this has been confirmed by the developers. Trace therefore provides a fundamental first step towards attacker agent forensics.
Sources
- LLMs Have Rhythm: Fingerprinting Large Language Models Using Inter-Token Times and Network Traffic Analysis
- Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities
- PentestGPT: An LLM-empowered Automatic Penetration Testing Tool
- Teams of LLM Agents can Exploit Zero-Day Vulnerabilities
- Fingerprinting AI Coding Agents on GitHub
- The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
- Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactions
- CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts
- Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data