Not All Loops Cost the Same: Phase-Dependent Depth for Serving Looped Transformers
cs.DC
Submitted: 2026-11-04
Updated: 2026-11-04
Terminology
Sources
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- Accelerating Large Language Model Decoding with Speculative Sampling
- Hybrid Latent Attention for Looped Language Models
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection
- WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models
- FastMoE: A Fast Mixture-of-Expert Training System
- POP: Prefill-Only Pruning for Efficient Large Model Inference
- Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
- One Model to Loop Them All: Elastic Latent Recurrence for Language Models
- Decode-Branch Transformers: Decoupling the Primary Prefill Path from Additional Decode Computation
- Enabling Dynamic Computation in Looped LMs
- Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibility
- Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
- RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models
- SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
- Fast Distributed Inference Serving for Large Language Models
- MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache
Related papers
- Programmable Probabilistic Computer with 1,000,000 p-bits
- BatchGen: An Architecture for Scalable and Efficient Batch Inference
- Seamless Reconfiguration for DAG-BFT (Technical Report)
- ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
- iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
- TIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends