FluidPD: In-Place Elasticity for SLO-Aware Prefill-Decode Disaggregated LLM Serving
cs.AI
Submitted: 2026-10-02
Updated: 2026-10-02
Code: https://github.com/llm-d/llm-d
Terminology
Sources
- AIBrix: Towards Scalable, Cost-Effective Large Language Model Inference Infrastructure
- Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving
- Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
- P/D-Serve: Serving Disaggregated Large Language Model at Scale
- TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity
- Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
- DOPD: A Dynamic PD-Disaggregation Architecture for Maximizing Goodput in LLM Inference Serving
- Beyond the Buzz: A Pragmatic Take on Inference Disaggregation
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
- Scorpio: Serving Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference
- Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving
- Arrow: Adaptive Scheduling Mechanisms for Disaggregated LLM Inference Architecture
- Pie: Pooling CPU Memory for LLM Inference
- ReAct: Synergizing Reasoning and Acting in Language Models
- FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
- SGLang: Efficient Execution of Structured Language Model Programs
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems