Compute Globally, Materialize Locally: The Memory Contract of Sparse Event-KV
cs.AI
Submitted: 2026-07-26
Updated: 2026-10-07
Code: https://github.com/oklen/Compute-Globally-Materialize-Locally
License: http://creativecommons.org/licenses/by/4.0/
The gist: Long-horizon agents increasingly reuse their KV cache as memory: a serving system keeps a subset of cached entries and drops the rest.
Terminology
Abstract
Long-horizon agents increasingly reuse their KV cache as memory: a serving system keeps a subset of cached entries and drops the rest. Eviction and episodic-memory schemes therefore rest on a premise rarely tested directly, that a retained event is still informative once the observations that produced it are gone. We test it by omitting one earlier observation from what is served, across otherwise identical agent histories. Among items sensitive to that observation, the answer overwhelmingly follows the omitted value, though no served span says which value is correct. We call this semantic materialization: a downstream event's cached rows act as an independently servable view of computation whose inputs are gone. It can also be written on purpose. A deliberately phrased, answer-free event raises donor-aligned recovery from 6% to 51% on Qwen3-8B without ever naming the value, whereas passively harvesting natural mentions from long-term dialog yields no detected advantage. What such a row carries is specific and bounded. Compact state survives, larger payloads decay toward chance, and whether a construction writes at all turns on phrasing rather than on meaning alone, so two phrasings the model comprehends equally well can diverge sharply. The result is a memory contract for sparse event-KV serving: what to write, where it lands, and what survives once the source is gone. For anyone who evicts the corollary is that dropping a source event and observing no accuracy loss does not show the source was unnecessary.
Sources
- APEX-MEM: Agentic Semi-Structured Memory with Temporal Reasoning for Long-Term Conversational AI
- Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
- C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
- Human-inspired Episodic Memory for Infinite Context LLMs
- Gemma 2: Improving Open Language Models at a Practical Size
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- EPIC: Efficient Position-Independent Caching for Serving Large Language Models
- EpiCache: Episodic KV Cache Management for Long-Term Conversation on Resource-Constrained Environments
- MEMENTO: Teaching LLMs to Manage Their Own Context
- KV-PRM: Efficient Process Reward Modeling via KV-Cache Transfer for Multi-Agent Test-Time Scaling
- REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation
- Models Take Notes at Prefill: KV Cache Can Be Editable and Composable
- KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
- SnapKV: LLM Knows What You are Looking for Before Generation
- HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching
- Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
- The Llama 3 Herd of Models
- Leyline: KV Cache Directives for Agentic Inference
- Evaluating Very Long-Term Conversational Memory of LLM Agents
- MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems