A Flat KV Cache Storage Architecture for LLM Serving
cs.DC, cs.OS
Submitted: 2026-10-28
Updated: 2026-11-18
Comments: 10 figures
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
- Tutti: Making SSD-Backed KV Cache Practical for Long-Context LLM Serving
Related papers
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
- AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition