Spectral Margins and Memory Routing in Looped Transformers
cs.LG, cs.NE
Submitted: 2026-07-12
Updated: 2026-10-30
Comments: 25 pages, 11 figures, 11 tables. v2: substantially revised and retitled (v1: "LayerNorm as Implicit Gain Control in Looped Transformers"). Code available at https://github.com/clojure-finance/mythjure/tree/paper-v2
Code: https://github.com/clojure-finance/mythjure
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Adaptive Computation Time for Recurrent Neural Networks
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- On the Parameterization and Initialization of Diagonal State Space Models
- Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
- Parcae: Scaling Laws For Stable Looped Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis