Scaling Inherently Interpretable Language Models
Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen, Isaac Plant, Muawiz Chaudhary, Nathaniel Monson, Saqib Azim, Zhichen Guo, Julius Adebayo
cs.CL, cs.AI
Submitted: 2026-08-06
Updated: 2026-08-11
Code: https://github.com/openai/tiktoken
License: http://creativecommons.org/licenses/by/4.0/
The gist: Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish.
Terminology
Abstract
Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.
Sources
- GPT-4 Technical Report
- Understanding intermediate layers using linear classifier probes
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- Llemma: An Open Language Model For Mathematics
- Qwen Technical Report
- Towards Unifying Interpretability and Control: Evaluation via Intervention
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
- Reasoning Models Don't Always Say What They Think
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Towards A Rigorous Science of Interpretable Machine Learning
- Neuron Populations Exhibit Divergent Selectivity with Scale
- Toy Models of Superposition
- All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously
- The Llama 3 Herd of Models
- Studying Large Language Model Generalization with Influence Functions
- Measuring Massive Multitask Language Understanding
- Scaling Laws for Autoregressive Generative Modeling
- Training Compute-Optimal Large Language Models
- MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
- Missingness Bias in Model Debugging
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering