LLM Inference Under Bursty Workload Distribution: Modifying the WAIT Algorithm
Anjali Gangadhar Katageria, Shobha Rani, Raghu Nandan Sengupta
cs.LG
Submitted: 2026-08-06
Code: https://github.com/anjalik04/modified_wait
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- Benchmarking Neural Network Training Algorithms
- SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
- PaLM 2 Technical Report
- ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictor
- Scaling Laws for Neural Language Models
- GPT-4 Technical Report
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Efficient LLM Scheduling by Learning to Rank
- Queueing, Predictions, and LLMs: Challenges and Open Problems
- Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
- Fitting procedure for the two-state Batch Markov modulated Poisson process
- Vidur: A Large-Scale Simulation Framework For LLM Inference
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks