Capital Efficiency, Isolation, Risk, and Ruin in Generative AI Inference Serving Infrastructure
cs.DC, cs.AI
Submitted: 2026-11-06
Updated: 2026-11-06
Terminology
Sources
- Mixtral of Experts
- Scorpio: Serving Right Requests at the Right Time for Heterogeneous SLOs in LLM Inference
- Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
- MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
- CascadeInfer: Length-Aware Scheduling of LLM Serving with Low Latency and Load Balancing
- JITServe: SLO-aware LLM Serving with Imprecise Request Information
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends