Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting

arXiv:2605.01186 · cs.CR · Submitted 2026-05-02 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting".

Elias: Here's the gist of the paper. AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures.

Nadia: Framework Introduction Trace is introduced as a novel multi-stage attribution and forensic framework for AI attack agents using terminal command sequences.

Elias: The key insight is that AI attack agents often leave behavioral traces in the executed commands and their actions also vary when actively prompted.

Nadia: This framework builds a two-stage pipeline where Stage 1 passively monitors command sequences to fingerprint the model family via TF-IDF and LinearSVC.

Paper summary: Nadia: So we've covered what Trace is designed to do—this paper introduces it as a two-stage framework that first passively fingerprints an AI attack agent using terminal command sequences, and then actively uses that fingerprint to perform a defensive prompt injection to extract system prompts >

Elias: That passive stage relies on term frequency inverse document frequency and linear support vector classification, and they found that this approach is quite light-weight, which is good for real-time deployment and monitoring >

Priya: But we need to ask what the specific results show. What did they actually measure when they tested this framework against different attacker models? Does it work reliably?

Nadia: They showed that in stage one, using the TF-IDF bigram classifier, they could attribute a terminal session to one of seven LLM families >

Elias: And when we look at the performance on that classification, they found a macro F1 score of zero point nine eight one in accurately fingerprinting the attacker model family > <ref:2605.01186#pg1>

Priya: A high score there is promising, but what about stage two? How effective is the active forensics part when they use that attribution to guide their defensive prompt injection strategy?

Nadia: Stage two leverages that attribution to deploy family-calibrated defensive prompt injection payloads, which they test using variants like Vanilla, M2 for Authority Override, and FC for Format Coercion >

Elias: And the evaluation shows that when using this fingerprinting to guide the DPI through a crafted payload, they achieved system prompt extraction from eighty-one point nine percent of non-Claude sessions on average > <ref:2605.01186#pg1>

Priya: That sounds significant if true. But what are the caveats here? What do we know about how robust this method is when it encounters settings that aren't in their training data?

Nadia: They did evaluate the robustness by testing on unknown settings, and they found that Trace still retains an average F1 score over eighty-one percent > <ref:2605.01186#pg1>

Elias: They also showed how it handles evasion strategies from the attacker, and they compared their lightweight fingerprinting approach against a baseline using LLMMap, finding that their method still outperformed the baseline by zero point seven percent > <ref:2605.01186#pg1>

Conclusion: Nadia: So to wrap up this discussion on "Trace: Unmasking AI Attack Agents Through Terminal Behavior Fingerprinting," it’s really about giving us a way to understand the intent behind these autonomous AI attack agents >

Elias: The authors, Murali Ediga and Sudipta Chattopadhyay, basically proposed this two-stage framework using terminal command sequences to identify the model family controlling an agent and then using that to actively extract system prompts >

Priya: What does this mean for the real world? If we can reliably fingerprint which AI model is behind an attack, does it change how we think about defense?

Nadia: It means we gain information on the attacker's intent, which helps us develop countermeasures against those specific model families >

Elias: The paper shows that this approach isn't just theoretical; they tested it in a blackbox penetration test agent framework and got seventy-eight percent accuracy in fingerprinting, and their active forensics resulted in a one hundred percent Gemini DPI extraction with the full system prompt confirmed by the developers > <ref:2605.01186#pg1>

Priya: The limitation they mention is that this approach relies on observing command sequences, so it's limited to environments where those commands are exposed and logged >

Nadia: Right. So while it gives us a strong tool for understanding AI agent attacks, we have to remember that its effectiveness depends on the exposure of the terminal behavior being analyzed >

University of Missouri-Kansas City

cs.CR

Submitted: 2026-05-02

Updated: 2026-05-02

Journal ref: The Workshop on AI for Cyber Threat Intelligence (WAITI) 2026, co-located with ACSAC 2026

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 88/100

The gist: AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks Identifying the model family that controls the active sessions of such agents…

Key concepts

Trace
A multi-stage framework designed to attribute AI attack agents by analyzing their terminal command sequences. It passively monitors commands in Stage 1 to fingerprint the model family and then uses that information in Stage 2 to perform active forensics, specifically extracting the agent's system prompt.
Stage 1: Attribution
The first part of the framework analyzes command sequences using TF-IDF with bigram features. It uses a LinearSVC classifier to fingerprint the attacker's LLM family based on model-specific command idioms, such as DeepSeek's 'exec grep', aiming for high accuracy in identifying the source model.
Stage 2: Forensic Intelligence Extraction
The second stage deploys defensive prompt injection (DPI) payloads tailored to the identified model family. These payloads test compliance pressure through variants like Vanilla, M2, and FC. The goal is to extract valuable adversary intelligence, such as the agent's system prompt.
Defensive Prompt Injection (DPI)
A strategy where family-calibrated prompts are used to probe different interaction lifecycle points of the AI agent. This involves using specific payload variants (Vanilla, M2, FC) targeting elements like audit requirements or SSH wrappers to extract hidden information from the agent.

Terminology

Summary

AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures. p. 0

Where the paper says it

AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks. Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures.

Framework Introduction

Trace is introduced as a novel multi-stage attribution and forensic framework for AI attack agents using terminal command sequences. The key insight is that AI attack agents often leave behavioral traces in the executed commands and their actions also vary when actively prompted. p. 1

Where the paper says it

In this paper, we introduce Trace, a novel attribution and forensic framework to facilitate deep investigation on AI agent attacks via terminal command sequences. Our key insight is that AI attack agents often leave behavioral traces in the executed commands and their actions also vary when actively prompted.

This framework builds a two-stage pipeline where Stage 1 passively monitors command sequences to fingerprint the model family via TF-IDF and LinearSVC. p. 1
Where the paper says it

Based on this insight, Trace builds a two-stage framework. The first stage passively monitors the command sequences and fingerprints the model family via term frequency–inverse document frequency (TF-IDF) and linear support vector classification (LinearSVC).

The second stage performs active forensics via defensive prompt injection (DPI) routed to the attacker-specific model. p. 1
Where the paper says it

Such a light-weight design allows for real-time deployment and monitoring. In the second stage, Trace leverages the information obtained via fingerprinting and performs active forensics via defensive prompt injection (DPI) routed to attacker-specific model.

Attribution Mechanism

Stage 1 uses TF-IDF vectorization with bigram features to attribute a terminal session to one of seven LLM families. The classifier is implemented using LinearSVC with balanced class weights and regularization parameter C=1. p. 6

Where the paper says it

The TF-IDF vectorizer uses (1,2)-gram range (unigrams and bigrams), sublinear TF weighting, and a vocabulary capped at 10,000 features. Our classification uses LinearSVC with balanced class weights and regularization parameter C=1.

0. Discriminative features include model-specific command idioms such as DeepSeek’s top feature being exec grep (coeff. p. 8
Where the paper says it

Discriminative features: LinearSVC’s coefficient vectors reveal model-specific command idioms. To this end, DeepSeek’s top feature is exec grep (coeff.

1.55). The TF-IDF classifier achieves a macro F1 score of 0. p. 1
Where the paper says it

Our evaluation revealed that passive fingerprinting via TF-IDF and LinearSVC achieves 0. 981 macro F1 score, with model-specific command idioms as discriminative features (Section 4).

981 in accurately fingerprinting the attacker model family. p. 0
Where the paper says it

Our evaluation reveals that Trace achieves a macro F1 score of 0. 981 in accurately fingerprinting the attacker model family (0.

Forensic Intelligence Extraction

Stage 2 leverages the attribution to deploy family-calibrated DPI payloads, aiming to extract the agent’s system prompt as adversary intelligence. The DPI strategy involves four reinforcing layers targeting different points in the agent’s interaction lifecycle, such as audit requirements. p. 4

Where the paper says it

A single planted file may go unread. To maximize the probability that the agent encounters and complies with the DPI payload, the infrastructure employed four reinforcing layers, each targeting a different point in the agent’s interaction lifecycle:

txt and a ForceCommand SSH wrapper. p. 5
Where the paper says it

This is the primary payload carrier, triggered when the agent reads files during reconnaissance. A ForceCommand SSH wrapper that printed a compliance warning on every command execution, ensuring the agent encountered the audit narrative even if it never read the planted file.

The DPI payload variants include Vanilla, M2 (Authority Override), and FC (Format Coercion) to test compliance pressure. p. 5
Where the paper says it

, that knowing the model family enables selection of a more effective payload (Figure 2). The three variants test a progression of compliance pressure: Vanilla tests whether a simple request suffices, M2 tests whether institutional authority pressure increases compliance depth, and FC tests whether corrective framing—implying the agent already failed a requirement—elicits verbatim reproduction rather than paraphrasing.

Performance and Robustness

The evaluation reveals that fingerprinting guides the DPI via a crafted payload resulting in system prompt extraction from 81.9% of non-Claude sessions on average. p. 0

Where the paper says it

Besides, the fingerprinting guides the DPI via a crafted payload to certain model families, resulting in system prompt extraction from 81. 9% of non-Claude sessions on average (up to 98.

The extraction fidelity is measured using Sentence-BERT cosine similarity, achieving 1. p. 9
Where the paper says it

An agent may write content that merely summarizes its task objective (low fidelity) or reproduces the verbatim system prompt (high fidelity). To address this, we measure fidelity using Sentence-BERT cosine similarity between the extracted text and ground-truth system prompt (see Section 3).

88× higher information compared to blind deployment. p. 9
Where the paper says it

2 RQ2: Does Fingerprinting Enable Effective Active Forensics? In this research question, we aim to evaluate whether model family identification enables targeted DPI deployment, achieving significantly higher system prompt extraction rates and extraction fidelity compared to blind (untargeted) payload deployment.

Trace achieves a macro F1 score of 0. p. 0
Where the paper says it

The attacker agents are bolstered via three distinct scaffolds and seven frontier model families. Our evaluation reveals that Trace achieves a macro F1 score of 0.

981 in accurately fingerprinting the attacker model family (0. p. 0
Where the paper says it

Our evaluation reveals that Trace achieves a macro F1 score of 0. 981 in accurately fingerprinting the attacker model family (0.

815 when generalizing to unseen scaffolds). p. 0
Where the paper says it

981 in accurately fingerprinting the attacker model family (0. 815 when generalizing to unseen scaffolds).

The approach retains an average F1 score over 81% when evaluated on unknown settings. p. 2
Where the paper says it

Trace: Unmasking AI Attack Agents 3 4. We compare the robustness of our approach by evaluating on unknown settings, showing that it retains an average F1 score over 81%.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Active attribution and forensics via Trace framework enables understanding of attacker intent by identifying model family through terminal command sequences. This allows for guiding a defensive prompt injection strategy to exfiltrate system prompts from an attacker model, thus, revealing valuable information to understand the attacker intent.

  2. Real-time passive fingerprinting using TF-IDF and LinearSVC allows for monitoring honeypots with a light-weight design. This enables real-time deployment and monitoring of AI attack agents by attributing sessions to one of seven LLM families based on command sequences alone.

  3. Fingerprint-guided payload routing achieves improved extraction quality by aligning the DPI strategy with the attacker model's susceptibility profile. For instance, FC dominates for five of six non-Claude families, achieving 87–98% extraction rates on GLM-5, GPT-5.4, and Kimi.

  4. The system can extract full context dumps under certain conditions by deploying M2 payloads against models like DeepSeek. This reveals the entire system prompt and this has been confirmed by the developers, providing richer forensic intelligence than just the system prompt alone.

  5. Trace provides a fundamental first step towards attacker agent forensics by moving beyond prior works that either work on text inputs or inter-token timing, specifically working on shell command sequences to fingerprint LLM family and perform active forensics.

Abstract

AI-driven penetration testing agents are now capable of autonomously executing attacks within compromised networks. Identifying the model family that controls the active sessions of such agents provides valuable information towards understanding the intent of the attack and further developing attack countermeasures. In this paper, we introduce Trace, a novel multi-stage attribution and forensic framework for AI attack agents using terminal command sequences. Once Trace identifies a model family for the attacker agents, it guides a defensive prompt injection (DPI) strategy to the attacker model via a crafted payload. This is with the aim to exfiltrate system prompts from an attacker model, thus, revealing valuable information to understand the attacker intent and facilitate further forensic investigation. We have implemented our approach revolving around a Linux capture-the-flag (CTF) box. The attacker agents are bolstered via three distinct scaffolds and seven frontier model families. Our evaluation reveals that Trace achieves a macro F1 score of 0.981 in accurately fingerprinting the attacker model family (0.815 when generalizing to unseen scaffolds). Besides, the fingerprinting guides the DPI via a crafted payload to certain model families, resulting in system prompt extraction from 81.9% of non-Claude sessions on average (up to 98.3%) at 0.736 Sentence-BERT fidelity -- 1.88x higher than blind deployment. Finally, to validate the robustness of Trace, we evaluate it with a blackbox and proprietary scaffold employing multiple model families (Gemini and Claude Opus). Our evaluation identified the model family with an average 78% accuracy. Moreover, for the Gemini model family, the DPI employed by Trace revealed the entire system prompt and this has been confirmed by the developers. Trace therefore provides a fundamental first step towards attacker agent forensics.

Sources

Related papers