JigSLO: Solving the Resource Puzzle for Efficient Multi-LLM Serving
cs.DC, cs.LG
Submitted: 2025-09-08
Updated: 2026-11-17
Terminology
Sources
- MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
- Qwen3 Technical Report
- Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends