How Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call Traces
cs.CR, cs.CL
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: 24 pages, 12 figures, 8 tables, including references and appendices
Code: https://github.com/AI45Lab/PrivacyTrace
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language model (LLM)-based agents interact with external tools through protocols such as the Model Context Protocol (MCP)…
Terminology
Abstract
Large language model (LLM)-based agents interact with external tools through protocols such as the Model Context Protocol (MCP), producing tool-call traces that may expose behavioral signals about users. In this work, we systematically study the risk of user-profile inference from MCP tool-call traces, where observers infer user attributes from protocol-valid tool interactions without access to user conversations. We introduce PrivacyTrace, a benchmark containing 1,000 synthetic user profiles and 4,000 agent trajectories across four domains, generated with real MCP tools from 25 servers. Experiments with four mainstream LLM attackers show that tool-call traces enable a 26.25% attack success rate for profile inference, compared with 17.97% from prior-based guessing. Further analysis reveals that tool arguments and returned results provide the main trace-grounded evidence, while linking traces across sessions further amplifies profiling risk. These findings demonstrate that MCP tool-call traces can act as effective behavioral side channels for user profiling, motivating the development of privacy-aware mechanisms for agent-tool interactions. Code and data are available at https://github.com/AI45Lab/PrivacyTrace.
Sources
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- GLM-5: from Vibe Coding to Agentic Engineering
- ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents
- Network-Level Prompt and Trait Leakage in Local Research Agents
- ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
- AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
- Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation
- SPILLage: Agentic Oversharing on the Web
- Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents
- ReAct: Synergizing Reasoning and Acting in Language Models
- IntentMiner: Intent Inversion Attack via Tool Call Analysis in the Model Context Protocol
- PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
- Exposing LLM User Privacy via Traffic Fingerprint Analysis: A Study of Privacy Risks in LLM Agent Interactions
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data