Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It
cs.AI, cs.CL, cs.LG
Submitted: 2026-09-29
Updated: 2026-09-29
Project page: https://lunamos.github.io/stop-thinking-too-early
License: http://creativecommons.org/licenses/by/4.0/
The gist: Pretrained transformers use little of their depth to follow references in context.
Terminology
Abstract
Pretrained transformers use little of their depth to follow references in context. Thirteen base models reliably follow only 1.4-3.6 lines, and extra pretrained loops add little. A task-trained rank-8 LoRA at one early layer extends this computation with all model weights frozen. Qwen3-8B improves from 15.5% to 99% exact accuracy on 24-line chains; a longer-trained LoRA reaches 50 lines. Ouro-1.4B reaches 60 lines after four loops and at least 160 after eight. The LoRA starts a relay: program lines pass on their chain identity through a short range of middle layers. Frozen heads read progressively further up the chain, and removing parent-line attention stops the relay. A frozen-model measurement locates the last useful intervention layer within tolerance in three of four held-out models. Task-specific LoRAs also improve MuSiQue. Default answers therefore understate the computation accessible through a tiny edit. Code and an interactive demo are available at https://lunamos.github.io/stop-thinking-too-early/
Sources
- Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
- Lessons from Studying Two-Hop Latent Reasoning
- A Mechanistic Analysis of Looped Reasoning Language Models
- How do Language Models Bind Entities in Context?
- Gemma 3 Technical Report
- The Llama 3 Herd of Models
- How Do LLMs Perform Two-Hop Reasoning in Context?
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- Beyond Steering Vector: Flow-based Activation Steering for Inference-Time Intervention
- Entity Tracking in Language Models
- Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts
- Racing Thoughts: Explaining Contextualization Errors in Large Language Models
- The Power of Scale for Parameter-Efficient Prompt Tuning
- Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
- Pointer Sentinel Mixture Models
- A retrieval conditioned rebinding circuit for dynamic entity tracking in large language models
- In-context Learning and Induction Heads
- Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking
- Language Models use Lookbacks to Track Beliefs
- Measuring and Narrowing the Compositionality Gap in Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems