Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving
cs.DC, cs.PF
Submitted: 2026-04-16
Updated: 2026-11-03
Code: https://github.com/TingyangSunJeff/load_balance_llm_simulator
Terminology
Sources
- Placement and Routing Optimization Problem for Service Function Chain: State of Art and Future Opportunities
- Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
Related papers
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
- AgentWeave: Efficient Distributed Agent Serving via Flow Decomposition