CADOC: Cache-Aware Dynamic Object Context for Long-Horizon Agents
cs.AI
Submitted: 2026-09-29
Updated: 2026-09-29
License: http://creativecommons.org/licenses/by/4.0/
The gist: For a long-horizon agent, context is the bottleneck: the history is resent with every request, the window caps task length, and reasoning degrades as the history grows.
Abstract
For a long-horizon agent, context is the bottleneck: the history is resent with every request, the window caps task length, and reasoning degrades as the history grows. Replacing structured objects with compact retrieval Cards shortens the prompt and keeps the exact originals retrievable, but editing the history can break prefix-cache reuse, and prior recoverable methods time their edits by forecasts of future reuse or by preset intervals. We propose CADOC (Cache-Aware Dynamic Object Context), an online algorithm that replaces structured objects with compact Cards while preserving exact, on-demand retrieval of their original contents. CADOC schedules replacements in batches by balancing accumulated waiting cost against shared cache-reconstruction cost. Its scheduling rule follows from an economic order quantity trade-off, recovers the optimal integer batch under stationary assumptions. Across evaluation, CADOC consistently achieves the lowest aggregate input cost among the compared configurations, which reduces input cost by approximately 40% on average while maintaining task performance close to full context. CADOC thus provides a cost-derived approach to compressible context management, demonstrating that efficient compression depends not only on shortening prompts but also on scheduling edits to preserve cache reuse.
Sources
- LCM: Lossless Context Management
- Self-GC: Self-Governing Context for Long-Horizon LLM Agents
- ACON: Optimizing Context Compression for Long-horizon LLM Agents
- The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management
- Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks
- The Missing Memory Hierarchy: Demand Paging for LLM Context Windows
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- MemGPT: Towards LLMs as Operating Systems
- LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
- ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization
- ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems