Rethinking KV Cache Eviction for Agentic LLM Serving
cs.DC
Submitted: 2026-10-30
Updated: 2026-10-30
Code: https://github.com/uw-syfi/TraceLab
Terminology
Sources
- ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System
- Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
- When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse
- Multi-Segment Attention: Enabling Efficient KV-Cache Management for Faster Large Language Model Serving
- Qwen3 Technical Report
- TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends