Fairway: Load-Aware KV Cache Reuse for LLM Prefill Scheduling
cs.DC
Submitted: 2026-11-01
Updated: 2026-11-01
Code: https://github.com/docugami/KG-RAG-datasets
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Retrieval-Augmented Generation for Large Language Models: A Survey
- The Llama 3 Herd of Models
- TOPAS: Workflow-Aware Prefix-State Scheduling for Multi-Agent LLM Serving
- HAServe: Horizon-Aware Scheduling and Memory Co-Design for Efficient LLM Serving
- CacheWise: Understanding Workloads and Optimizing KVCache Management for Efficiently Serving LLM Coding Agents
- RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems
- Fast Distributed Inference Serving for Large Language Models
- PEEK: Predictive Queue-Informed KV Cache Management for LLM Serving
- BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends