WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse
cs.DC, cs.AI
Submitted: 2026-09-28
Updated: 2026-09-28
Comments: 33 pages, 14 figures, 11 tables
Code: https://github.com/sgl-project/s
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- SiPipe: Bridging the CPU-GPU Utilization Gap for Efficient Pipeline-Parallel LLM Inference
- Marconi: Prefix Caching for the Era of Hybrid LLMs
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- Zero Bubble Pipeline Parallelism
- PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving
- Prompt Cache: Modular Attention Reuse for Low-Latency Inference
- DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
- USP: A Unified Sequence Parallelism Approach for Long Context Generative AI
- Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding
- Fast Transformer Decoding: One Write-Head is All You Need
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Gated Delta Networks: Improving Mamba2 with Delta Rule
- Kimi Linear: An Expressive, Efficient Attention Architecture
- Kimi K3: Open Frontier Intelligence
- GLM-5: from Vibe Coding to Agentic Engineering
- The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
Related papers
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- SAMM: Sharded Automated Market Maker
- InferScale: GPU-Native KV Injection for Personalized LLM Serving
- Vigil: Accountable Liveness against Selective Silence
- Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
- Democratizing MoE inference on commodity GPUs with CoMoE