There Is No Free Loop: When Does Looping a Trained Model Pay for Its Inference Compute?
cs.LG, cs.AI
Submitted: 2026-10-30
Updated: 2026-10-30
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Looping re-applies a block of layers so that each token passes through the block more than once.
Terminology
Abstract
Looping re-applies a block of layers so that each token passes through the block more than once. Adding a loop to a trained model is attractive because it leaves the parameter count, and hence the disk size, the memory footprint and the training cost unchanged. Unfortunately, every generated token pays for the loop. Re-applying m of N layers makes each token about (N+m)/N times as expensive. Loops that a model learns during large-scale pretraining justify this price, but whether loops added after pretraining justify this cost remains unclear. We compare a looped model with two simple baselines. We first compare it with the same model trained identically without the loop (equal parameters). Secondly, we compare it with the best unlooped use of the same inference floating-point operations (equal FLOPs). Across Qwen3 and Qwen3.5 models from 0.8B to 9B parameters, only the looped 0.8B model beats both baselines. At 4B and 9B, the looped model does not beat the first baseline, i.e. the same model trained without the loop. This is not because the benchmarks are too easy, since on the same benchmarks the unlooped 9B model still scores up to 13 points higher than the 4B model, showing that there is ample room for improvement. Introducing loops in an already pretrained model therefore only pays for small models with a fixed parameter budget.
Sources
- Improving Test-Time Scaling with Adaptive Looped Transformers
- Closing the Loop: Practical Training Recipes for Looped Language Models
- How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models
- SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
- Loop the Loopies!
- Training-Free Looped Transformers
- Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
- Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
- Scaling Latent Reasoning via Looped Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
- Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
- Training Verifiers to Solve Math Word Problems
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Qwen3 Technical Report
- Adaptive Computation Time for Recurrent Neural Networks
- PonderNet: Learning to Ponder
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- Scaling Laws for Neural Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis