Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice
cs.AI
Submitted: 2026-06-21
Updated: 2026-10-07
Code: https://github.com/Li-Beverly-Kong/Geometry-Aware-Online-Scheduling
Terminology
Sources
- GPT-4 Technical Report
- DeepSeek-V3 Technical Report
- Online Scheduling for LLM Inference with KV Cache Constraints
- LLM Serving Optimization with Variable Prefill and Decode Lengths
- ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
- The Llama 3 Herd of Models
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
- Well-Read Students Learn Better: On the Importance of Pre-training Compact Models
- Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank
- Fast Distributed Inference Serving for Large Language Models
- Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
- Don't Stop Me Now: Embedding Based Scheduling for LLMs
- Predicting LLM Output Length via Entropy-Guided Representations
- Adaptively Robust LLM Inference Optimization under Prediction Uncertainty
- Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
- GreenLLM: SLO-Aware Dynamic Frequency Scaling for Energy-Efficient LLM Serving
- Optimizing LLM Inference Throughput via Memory-aware and SLA-constrained Dynamic Batching
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems