Should You Split Decode? Disaggregation Across Future Models and Accelerators
cs.AR
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/hao-ai-lab/FastAFD
Terminology
Sources
- OmniPilot: An Uncertainty-Aware LLM Inference Advisor for Heterogeneous GPU Clusters
- Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models
- MIST: A Co-Design Framework for Heterogeneous, Multi-Stage LLM Inference
- FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism
- Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V3 Technical Report
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM Serving
- Frontier: Towards Comprehensive and Accurate LLM Inference Simulation
- When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
- GPU-Initiated Networking for NCCL
- Mercury: Ultra-Fast Language Models Based on Diffusion
- Mixtral of Experts
- P/D-Serve: Serving Disaggregated Large Language Model at Scale
- Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
- Kimi K2: Open Agentic Intelligence
- MiniMax Sparse Attention
- OpWeave: Flexible Operator Disaggregation for Heterogeneous LLM Serving
Related papers
- Embedded Arena: Iterative Optimization via Hardware Feedback
- WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
- SparsePixels: Efficient Convolution for Sparse Data on FPGAs
- Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
- AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
- RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design