What Sets the Serving Speed of Looped Language Models? Measurements and Optimizations Graded by Exactness
cs.DC
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/InferMatrixLab/vllm-rlt
Terminology
Sources
- Accelerating Large Language Model Decoding with Speculative Sampling
- LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers
- Training Verifiers to Solve Math Word Problems
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models
- ResidualQuant: KV Cache Quantization for Looped Transformers with 2-Bit Residuals
- LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models
- Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
- FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates
- Scaling Latent Reasoning via Looped Language Models
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends