MemHarmBench: Evaluating Cross-Session Prompt Injection in Memory-Persistent AI Agents
cs.AI
Submitted: 2026-11-01
Updated: 2026-11-01
Comments: 24 pages, including appendix
Code: https://github.com/Kirito14IT/MemHarmBench
License: http://creativecommons.org/licenses/by/4.0/
The gist: AI agents often rely on persistent memory to retain user preferences and operational rules across sessions.
Terminology
Abstract
AI agents often rely on persistent memory to retain user preferences and operational rules across sessions. However, this capability introduces a new safety risk: malicious instructions from messages, documents, or Web content can be retained in memory and influence future tasks even after the original attack content is removed. Existing attack success metrics, however, fail to capture who is affected and how attacks translate into downstream harm during otherwise normal task execution. We introduce MemHarmBench, a victim-centric benchmark for evaluating cross-session prompt injection in memory-persistent AI agents. MemHarmBench considers three categories of potential victims: users, workspace/project owners, and external stakeholders. Our evaluation first exposes an agent to malicious instructions and tests whether they are retained in memory. We then remove the attack content and initiate a new session with a legitimate task while preserving the agent's memory. We jointly evaluate whether the agent follows the attacker's intent, completes the legitimate task, and exhibits observable execution anomalies. Across multiple agent systems, we find that cross-session attacks can remain behaviorally stealthy: agents often complete tasks normally and exhibit no detectable anomalies while still making critical decisions aligned with the attacker's intent. By connecting malicious memory retention to downstream decisions and affected parties, MemHarmBench provides a systematic framework for exposing safety risks that conventional attack- and task-success metrics may overlook. We publicly release MemHarmBench at https://github.com/Kirito14IT/MemHarmBench.
Sources
- SecAlign: Defending Against Prompt Injection with Preference Optimization
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents
- Defending Against Indirect Prompt Injection Attacks With Spotlighting
- Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies
- Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
- WebGPT: Browser-assisted question-answering with human feedback
- MemGPT: Towards LLMs as Operating Systems
- Generative Agents: Interactive Simulacra of Human Behavior
- MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
- Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents
- ReAct: Synergizing Reasoning and Acting in Language Models
- Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models
- When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agents
- Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents