FastAFD: When Does Attention-FFN Disaggregation Pay on Rack-Scale NVLink?
cs.DC, cs.LG
Submitted: 2026-11-04
Updated: 2026-11-04
Code: https://github.com/hao-ai-lab/FastAFD
Terminology
Sources
- Accelerating Large Language Model Decoding with Speculative Sampling
- Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
- DeepSeek-V3 Technical Report
- Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Fast Inference from Transformers via Speculative Decoding
- Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache
- Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
- Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
- AFD-Ledger: Deployment Provisioning for Attention--FFN Disaggregation
- Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding
- Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
- How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
- Huawei Cloud Model-as-a-Service on the CloudMatrix384 SuperPod
- Qwen3 Technical Report
- MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends