FOCUS: Training-Free Decision-Preserving Context Compression for LLM Agents
cs.AI, cs.CL
Submitted: 2026-09-29
Updated: 2026-09-29
Comments: Preprint. Under Review
License: http://creativecommons.org/licenses/by/4.0/
The gist: LLM agents accumulate interaction histories that grow linearly with task length, causing quadratic inference cost scaling and performance degradation from attention dilution.
Terminology
Abstract
LLM agents accumulate interaction histories that grow linearly with task length, causing quadratic inference cost scaling and performance degradation from attention dilution. Existing context-compression methods learn what to discard offline: by contrastively optimizing guidelines, distilling compressors, or training compression policies. This incurs a substantial cost. Further, the compression policy is learned a priori and is not dynamically conditioned on the evolving test-time trajectories. In this paper we ask a complementary question: Which past interactions causally shape the agent's future decisions? We recast context compression as a causal decision preservation problem over discrete interaction units and introduce FOCUS, a training-free context compression framework that operates entirely at test time. Our method requires no offline data collection or fine-tuning, and is architecture-agnostic, attaching to any closed-API frontier model as a modular compression layer. We evaluate FOCUS on diverse agentic benchmarks including API and tool-calling, QA, web domain and multi-turn dialogue. Our method establishes new state of the art performance, cutting peak context by up to 48% and dependency by 73% while improving task success by up to 8.9 percentage points over uncompressed execution.
Sources
- $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Learning to Contextualize Web Pages for Enhanced Decision Making by LLM Agents
- MemGPT: Towards LLMs as Operating Systems
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
- Scaling Long-Horizon LLM Agent via Context-Folding
- OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation
- ReAct: Synergizing Reasoning and Acting in Language Models
- PAACE: A Plan-Aware Automated Agent Context Engineering Framework
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems