PACE: Keeping the Right Tokens in the KV Cache Is Not Enough for Long-Horizon LLM Agents
cs.LG, cs.CL, cs.DC
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/python/cpython
Terminology
Sources
- TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
- PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
- Extending Context Window of Large Language Models via Positional Interpolation
- LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Mistral 7B
- LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
- Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
- SnapKV: LLM Knows What You are Looking for Before Generation
- Lost in the Middle: How Language Models Use Long Contexts
- CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving
- Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
- MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference
- WAKMan: Workload Aware KVCache Manager
- MemGPT: Towards LLMs as Operating Systems
- KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
- YaRN: Efficient Context Window Extension of Large Language Models
- FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis