A Mechanistic Analysis of Looped Reasoning Language Models
cs.LG, cs.AI
Submitted: 2026-04-13
Updated: 2026-11-16
Code: https://github.com/karpathy/nanochat
Terminology
Sources
- On the Bottleneck of Graph Neural Networks and its Practical Implications
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning
- Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
- PonderNet: Learning to Ponder
- Why do LLMs attend to the first token?
- gLSTM: Mitigating Over-Squashing by Increasing Storage Capacity
- A Note on Over-Smoothing for Graph Neural Networks
- Training Verifiers to Solve Math Word Problems
- Continuous Thought Machines
- Universal Transformers
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Adaptive Computation Time for Recurrent Neural Networks
- When Attention Sink Emerges in Language Models: An Empirical View
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks
- Less is More: Recursive Reasoning with Tiny Networks
- Advancing the Understanding of Fixed Point Iterations in Deep Neural Networks: A Detailed Analytical Study
- Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
- The Remarkable Robustness of LLMs: Stages of Inference?
- Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis