Nebula: Distributed Position-Independent Caching for LLM Serving
cs.DC
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/redai-studio/Nebula
Terminology
Sources
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
- HYPIC: Position-Independent Caching for Hybrid-Attention LLM Serving
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- MiniPIC: Flexible Position-Independent Caching in <100LOC
- MEPIC: Memory Efficient Position Independent Caching for LLM Serving
- RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems
- TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
- Qwen3 Technical Report
- $A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving
Related papers
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
- AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition