What Sets the Serving Speed of Looped Language Models? Measurements and Optimizations Graded by Exactness

arXiv:2611.05150 · cs.DC · Submitted 2026-11-03 · Read on arXiv

cs.DC

Submitted: 2026-11-03

Updated: 2026-11-03

Code: https://github.com/InferMatrixLab/vllm-rlt

Terminology

Sources

Related papers