LAM: Efficient Lossy Agent Memory Framework With A Retrieval-Score Error Bound
cs.AI
Submitted: 2026-09-26
Updated: 2026-09-26
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Agent memory grows as agents read inputs, reason, and call tools.
Terminology
Abstract
Agent memory grows as agents read inputs, reason, and call tools. Longer histories increase inference cost and eventually exceed the context window. LLM-based summarization reduces this history but adds latency and provides no explicit bound on information loss. We propose LAM, a Lossy Agent Memory system with three components: a deterministic deduplication rule with a substitution bound on retrieval scores - a bound on score perturbation, not a certificate of unchanged ranking; a memory manager that preserves the cached prefix and overlaps compaction with inference; and a performance model that estimates compaction costs before deployment. On 600 agent trajectories, LAM removes 22.47% of observation tokens while retaining 99.984% of the measured gold-patch evidence. At a fixed deletion set, the performance model predicts a 71.4x-91.6x end-to-end speedup from removing records before prefill instead of deleting them from a prefilled context. That benefit comes from the schedule rather than the rule and applies to any prefix-preserving test.
Sources
- SemDeDup: Data-efficient learning at web-scale through semantic deduplication
- KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU
- Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Parallel Context Compaction for Long-Horizon LLM Agent Serving
- LightMem: Lightweight and Efficient Memory-Augmented Generation
- Control Under Compression: Reliability Frontiers for Tool-Using Agents
- Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial
- The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
- Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
- SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
- Remember the Decision, Not the Description: A Rate-Distortion Framework for Agent Memory
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems