Introducing Cluster Efficiency to LLM Serving: Pooling Economical Accelerators over Fast Interconnects
cs.DC
Submitted: 2026-11-02
Updated: 2026-11-06
Code: https://github.com/tenstorrent/t
Terminology
Sources
- The Llama 3 Herd of Models
- M'elange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
- Splitwise: Efficient generative LLM inference using phase splitting
- Chiplet Cloud: Building AI Supercomputers for Serving Large Generative Language Models
- HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processing
- Parallax: Efficient LLM Inference Service over Decentralized Environment
- HBM Is Not All You Need: Efficient Disaggregated LLM Serving across Memory-heterogeneous Accelerators
- AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
- Qwen3 Technical Report
- TraceLab: Characterizing Coding Agent Workloads for LLM Serving
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends