Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents

summary

The gist

AI web agents can be effectively distinguished from humans and traditional crawlers using a multi-layer fingerprint based on both network layer characteristics (e.g., TLS…

In short

The MARK framework distinguishes AI web agents from humans and crawlers using a multi-layer fingerprinting approach that analyzes network characteristics (TLS, HTTP) and browser behavior. By extracting features from timing, TLS, HTTP headers, and user interactions, the system achieves 97% accuracy in identifying agent architectures.

Key concepts

Multi-Layer Fingerprinting
This method involves passively recording traces of web agents across four protocol levels: request timing, TLS settings, HTTP headers, and in-browser actions. By combining data from these different layers, the system creates a robust identification signature that is difficult for agents to change without breaking their core behavior.
Temporal Features
These features analyze the sequence of HTTP requests by measuring timing patterns. Specifically, they calculate the Inter-Request Interval (IRI) distribution and request rate. These temporal patterns are highly distinctive for identifying different types of agents based on how quickly and regularly they make requests.
TLS Fingerprints (JA4)
The TLS extractor analyzes cryptographic parameters used during the secure connection setup, generating twelve feature groups. This includes details about post-quantum key exchange algorithms and JA4 fingerprints. These network-level characteristics are crucial for identifying specific agent architectures, such as Skyvern.
Behavioral Signals
This layer captures detailed client-side events like mouse movements, key presses, and input actions at a high resolution. These behavioral features are particularly effective for differentiating advanced LLM agents like Claude and Gemini by detecting subtle differences in typing patterns or consistency.

Terminology used across episodes

This episode discusses

The paper

Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents · Read on arXiv

University of Massachusetts Amherst

As AI web agents proliferate, combining large language models with autonomous, browser-level control, indiscriminate content scraping by web agents has emerged as a privacy and security challenge. Existing defenses, such as robots.txt and active bot-blocking, are insufficient, as they are widely violated and easily circumvented. In this work, we demonstrate that AI web agents can be effectively distinguished from humans and traditional crawlers using a multi-layer fingerprint based on both network layer characteristics (e.g., TLS, HTTP) and browser interaction behavior. We implement this mechanism as a programmatic logging framework that can be deployed on a live, instrumented domain. By analyzing six prominent agent frameworks (AutoGen, Browser Use, Claude, Gemini, ChatGPT, and Skyvern), we uncover latent structural differences in how these systems assemble HTTP requests, establish TLS/HTTP connections, and execute autonomous browser actions. Feeding these multi-layer features into a decision tree classifier, our framework achieves high-fidelity identification (100% accuracy), successfully isolating distinct agent architectures and differentiating agent traffic from both human browsing baselines and legacy crawlers. Our findings demonstrate that cross-layer agent tracking provides a robust approach to content protection and web security policy enforcement.

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents".

Elias: Here's the gist of the paper. AI web agents can be effectively distinguished from humans and traditional crawlers using a multi-layer fingerprint based on both network layer characteristics (e.g., TLS, HTTP) and browser interaction behavior By analyzing six prominent agent frameworks, the framework achieves high-fidelity identification (97% accuracy), successfully isolating distinct agent architectures and differentiating agent traffic from both human browsing baselines and legacy crawlers Multi-Layer Fingerprinting Methodology The MARK framework passively records and analyzes traces left by web agents at four protocol levels: request timing, TLS, HTTP, and in-browser behavior This system captures every incoming packet’s IP address, timestamp, TLS parameters, and HTTP headers while the front-end records user interactions at 1 ms resolution The framework involves configuring agents with a common web-interaction instruction and collecting raw network and client-side traces as each agent visits URLs and performs controlled UX tasks Feature Extraction Pipeline The extraction process generates feature vectors from different layers, including temporal patterns, TLS fingerprints, HTTP headers, and behavioral signals Temporal features derive from the timestamp sequence of HTTP requests by calculating Inter-Request Interval (IRI) distribution and request rate The TLS extractor parses each record to compute twelve feature groups, including Post-Quantum key exchange algorithms and JA4 fingerprints The HTTP header extractor enumerates seven feature groups, such as the Sec-Fetch rule violation and User-Agent family The behavioral extractor captures browser events like mouse move, keydown, and input, yielding ten groups with total 108 features Discriminative Findings Across Layers Analysis of the collected traces reveals that temporal patterns of requests enable evident identification among different agents Network-level features, such as HTTP/2 settings and JA4 fingerprints, are crucial for distinguishing agents like Skyvern due to its unique TLS/H2 fingerprint Furthermore, HTTP header analysis shows that Browser Use is uniquely identifiable due to impossible Sec-Fetch semantics in 100% of its HTTP requests Behavioral features are powerful, especially for Claude and Gemini, who are separable through subtle signals like typing behavior and consistent low-variance behavior Practical Identification Performance The combination of four cross-layer features in the decision tree classifier achieves a 97% accuracy in identifying agent types with 97% accuracy Temporal features alone achieve 100% accuracy when fully observed, confirming that request timing is highly distinctive, although it is unreliable for early-stage real-time identification The combined feature set recovers high performance (97.1%), demonstrating that the four features are complementary and provide a robust strategy for content protection Limitations and Deployment Considerations A limitation of the framework is that agents like Claude and Gemini are indistinguishable because they present almost identical TLS and HTTP fingerprints, suggesting that identical back-end systems lead to similar behavioral signals The classifier's performance may degrade over time as agent behavior changes due to evolving LLM models For real-time deployment, protocol-level fingerprints preserve more than 60% agent discrimination after three requests, while behavioral features replenish performance after signals accumulate The framework is available at https://github.

Title and authors: Nadia: Now we’re moving into the second part of this discussion, and we’re looking at the title and authors of this work, "Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents." This paper sets up the whole framework by focusing on how to assign an identity to these web agents.

Elias: The authors are Dayeon Kang, Hyejun Jeong, Jade Sheffey, Pubali Datta, and Amir Houmansadr from UMass Amherst. They are researchers focused on security and cryptography in AI systems.

Priya: I’m curious about the title itself. "Whose Agent Are You?" it sounds like it’s asking a very direct question about identity, but what exactly is the scope of that attribution?

Nadia: It’s not just a simple identification; it’s about attribution. They are trying to determine which specific agent framework—like AutoGen, Skyvern, or Gemini—is behind the traffic they are seeing.

Elias: That means they aren't just saying "this is an agent," but rather, "this traffic belongs to Agent X." It’s about uncovering latent structural differences in how these systems assemble their actions.

Priya: So, when we talk about attributing it to a specific framework, are we talking about the code itself or the model that runs on top of it? That distinction feels important for understanding the attack surface.

Nadia: It seems they are looking at both, because they evaluate agents across a range of architectures. They use GPT5-mini as a backbone for frameworks like AutoGen and Skyvern, and then they test GPT5 point five with extended effort for the OpenAI agent, plus computer-use agents from Claude and Gemini.

Elias: That means they are testing how the fundamental model choice interacts with the agent framework structure. It’s a layered approach to seeing what makes an agent unique.

Priya: And this connects back to my question about data trust. If the paper is comparing these different systems, how do they handle the inherent variability of running a complex LLM on top of a fixed framework?

Nadia: The paper does it by using prompts that direct each agent to explore the site and make autonomous decisions. They issue these prompts so every agent has to explore and make autonomous choices on the live website they've been instrumented under.

Elias: That way, you’re not just looking at a static configuration; you’re seeing how each framework interprets that instruction and executes it against the actual page content, which leads into the parsing logic distinction mentioned later.

Priya: So, the method isn't just about looking at a file; it's about watching the agent *do* something on a live site and seeing how its internal design dictates what it does next. That makes sense for privacy research.

The paper's summary: Nadia: So, to summarize the core of this study, they are demonstrating that AI web agents can be effectively distinguished from humans and traditional crawlers by using a multi-layer fingerprint based on network characteristics and browser interaction behavior.

Elias: They implement this as a programmatic logging framework that runs on an instrumented domain. This infrastructure captures every incoming packet’s IP address, the timestamp, TLS parameters, HTTP headers, while the front-end records user interactions at one millisecond resolution.

Priya: It sounds like they are building a very detailed log of everything happening—from the raw network packets right down to the single click a user makes on a page. That level of measurement is what gives them the power to extract these features.

Nadia: Right, and then they generate feature vectors from these layers: temporal patterns from request timestamps like Inter-Request Interval distribution, TLS fingerprints with twelve groups including post-quantum key exchange algorithms, HTTP headers with seven groups, and behavioral signals like mouse moves and keydowns.

Elias: Those features are what allow them to differentiate things. They found that temporal patterns help identify different agents, network features like HTTP two settings and JA4 fingerprints separate specific architectures, and behavioral features are powerful for distinguishing models like Claude and Gemini.

Priya: So the summary is that they found that these distinct layers yield different signals; temporal patterns are good, but network layer details are key for some agents, while subtle typing behavior helps with others.

Nadia: Exactly. The combined four cross-layer features fed into a decision tree classifier achieved ninety seven percent accuracy in identifying agent types across the six frameworks they tested.

Elias: That ninety seven percent performance is solid because it shows that the combination of these layers provides a robust strategy for content protection against these autonomous agents.

Priya: But I want to circle back to what the paper itself says about its limitations regarding those similar models like Claude and Gemini. What’s the catch there?

Nadia: The paper explicitly states that agents like Claude and Gemini can be indistinguishable because they present almost identical TLS and HTTP fingerprints, implying that if their back-end systems are very similar, their behavioral signals will also look very similar.

Elias: They also flag a limitation about the classifier itself: its performance might degrade over time as the agent behavior changes because those models are evolving.

Priya: So they’re acknowledging that this isn't a perfect, static solution for the long haul, which is fair. What about agents that try to look like humans by mimicking real browsing? Does this method handle those evasion attempts well?

The paper's improvements: Nadia: The paper discusses several ways they improved upon the original concept of just using a single fingerprint or just looking at one layer. They emphasized the power of combining multiple layers for a more comprehensive view.

Elias: Yes, they showed that temporal features alone can achieve one hundred percent accuracy when fully observed, confirming that request timing is a highly distinctive signal in isolation. But they also found that when you combine the four features—temporal, TLS, HTTP, and behavioral—the performance jumps to ninety seven point one percent.

Priya: So combining them makes it better than just using any single feature on its own. That’s a clear improvement for any security system trying to be practical.

Nadia: It’s about complementarity; each layer catches what the others miss, which is why that cross-layer approach is so much more robust than relying on just one thing. For example, temporal patterns alone are distinctive, but network features like HTTP two settings are crucial for separating agents like Skyvern.

Elias: And they highlighted specific agent-specific signals they were able to isolate. They pointed out that vision-based web agents can be isolated by detecting unique TLS and H2 fingerprints, such as Skyvern’s "seventeen TLS extensions" and a specific stream five priority weight of one hundred ten.

Priya: That is really interesting detail; linking a specific network parameter to a visual agent type gives us a very concrete way to target different automation tools. It moves beyond just saying "this is an AI agent."

Nadia: And they also showed that identifying bots that mimic human browsing HTTP semantics can be done by detecting cross-header inconsistencies, specifically looking for those Sec-Fetch rule violations and high dynamic header ordering behavior. That’s a direct way to catch those sophisticated attempts to hide their nature.

Elias: So the improvements focus on making the system more specific. Instead of just a general "agent is here," they pinpoint *which* agent architecture is there, based on these fine-grained network and interaction signals.

Priya: That level of specificity makes it useful for practical application because it tells you not just what's happening, but what kind of process is generating the traffic.

Conclusion: Nadia: So we’re wrapping up this look at "Whose Agent Are You? Multi-Layer Fingerprinting and Attribution of Autonomous Web Agents." The paper proves that a multi-layer fingerprinting approach using network characteristics and browser interaction behavior is an effective way to distinguish AI web agents from both humans and traditional crawlers.

Elias: It achieves ninety seven percent accuracy when analyzing six prominent agent frameworks, successfully isolating distinct agent architectures and differentiating their traffic from both human browsing baselines and legacy crawlers.

Priya: It’s a solid result for content protection because it shows a way to move past simple blocking mechanisms toward more nuanced identification based on deep behavioral signals. The ability to differentiate agents that share similar back-end systems is the most important finding here for privacy concerns, I think.

Nadia: Exactly. They showed that by analyzing network layer characteristics like TLS and HTTP and browser actions at one millisecond resolution, you get a very rich dataset for classification. It’s a strong strategy for building better defenses against these autonomous web agents.

Elias: And while acknowledging the limitations, it’s clear that while identical back-end systems lead to similar fingerprints, the cross-layer approach is still powerful enough to find differences in things like typing behavior when those signals accumulate.

Priya: So, this work gives us a concrete tool for understanding agent traffic at a deeper level than just looking at surface-level data. It shows that the nature of an agent’s browsing process leaves traceable footprints across multiple protocols and behaviors.

Nadia: That’s what it is. We’re looking at the full picture—the network layer, the HTTP layer, and the in-browser action—to get a high-fidelity identification of these autonomous agents. That's all for this paper for now.

More episodes

← Home