Asymptotic Optimality of c mu-Rule for LLM Inference Scheduling under Memory Constraint
math.OC, cs.AI
Submitted: 2026-11-03
Updated: 2026-11-03
Terminology
Sources
- Service-Induced Congestion in Memory-Constrained LLM Serving
- Adaptively Robust LLM Inference Optimization under Prediction Uncertainty
- Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
- General Non-Clairvoyant KV-Cache Scheduling via Regime-Aware Routing
- Competitive Non-Clairvoyant KV-Cache Scheduling for LLM Inference
- The Llama 3 Herd of Models
- Online Scheduling for LLM Inference with KV Cache Constraints
- Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
- Large-Scale LLM Inference with Heterogeneous Workloads: Prefill-Decode Contention and Asymptotically Optimal Control
- A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints
- OpenAI GPT-5 System Card
- GPT-4 Technical Report
- LLM Serving Optimization with Variable Prefill and Decode Lengths
Related papers
- An Accelerated Proximal Gradient Method for Bures-Wasserstein Space
- FOMTools: Development and AI-Assisted Code Generation for First-Order Methods
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Dynamic Service Recommendation with Congestion-Dependent Joining: Near-Optimal and Constant-Factor Approximation Algorithms
- Integrating Learning and Optimization for Open Transport Capacity Pooling under Disruption
- Zeroth-Order Gradient Estimators under Decision-Dependent Noise