Learning through Internalization
summary
The gist
We study internalization processes, by which neural-network-based systems absorb an explicit computational procedure into their own weights, and how they facilitate learning <ref:2606.20937#pg2>.
In short
The study explores how neural networks can 'internalize' a computational procedure by first learning it through explicit Chain-of-Thought (CoT) supervision and then reconfiguring their weights to execute it much faster. This method allows simplified transformers to solve hard tasks, like computing parity, directly in one step after removing the initial reasoning steps.
Key concepts
- Internalization
- This is a process where a neural network initially learns a complex procedure over many steps but later adjusts its internal weights to perform the same task much more efficiently. It means moving from a slow, step-by-step implementation to a fast, direct computation.
- Distributional Internalization
- A related concept suggesting success can be achieved by matching the performance of a slow system against a fast system under some underlying input distribution. This framework helps explain how learning can succeed even when direct learning is difficult.
- Semiautomata Internalization
- This investigates if transformers can learn to simulate semiautomata, which involves predicting the next state of a machine after T steps. The research found that internalizing CoT often leads to shortcuts that generalize poorly outside the training distribution.
Terminology used across episodes
This episode discusses
- Learning through Internalization · Paper Radio
- Towards a theory of model distillation
- Skip-Thinking: Chunk-wise Chain-of-Thought Distillation Enable Smaller Language Models to Reason Better and Faster
- Compressed Chain of Thought: Efficient Reasoning Through Dense Representations
- Implicit Chain of Thought Reasoning via Knowledge Distillation
- From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
- Training Large Language Models to Reason in a Continuous Latent Space · Paper Radio
- Distilling the Knowledge in a Neural Network
- Transformers Provably Solve Parity Efficiently with Chain of Thought
- Implicit Reasoning in Transformers is Reasoning through Shortcuts
- Shift is Good: Mismatched Data Mixing Improves Test Performance
- Positive Distribution Shift as a Framework for Understanding Tractable Learning
- Hardness of Learning Fixed Parities with Neural Networks
- Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
- Distilling System 2 into System 1
The paper
Learning through Internalization · Read on arXiv
Nikolaos Tsilivis, Nirmit Joshi, Marko Medvedev, Julia Kempe, Nati Srebro
New York University · Toyota Technological Institute at Chicago · University of Chicago
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Learning through Internalization".
Title and authors: Tom: So, to recap, this paper, "Learning through Internalization," is investigating how transformers absorb an explicit procedure by internalizing chain of thought tokens and then learning to compute the target task directly in one step after those tokens are gone.
Jane: It’s about moving from an explicit reasoning path to an implicit computation path, where the network's weights change to reflect that direct computation.
Lu: The main finding is that for the sparse parity task, a simplified one-layer transformer can provably learn this by first using explicit chain of thought supervision and then internalizing it as those tokens are progressively removed.
Meng: So, they’re showing that direct learning through internalization is possible for tasks that are normally computationally hard to find without that initial supervision.
Tom: And they also looked at how this works for simulating semiautomata, which involves predicting the next state of a semiautomaton after T transition steps given an initial state.
Jane: They found that internalizing chain of thought in these models sometimes leads to shortcut solutions that actually perform worse when we test them on data they haven't seen before.
Lalam: So, while internalization is powerful for learning the task itself, it comes with a trade-off regarding how well the model generalizes to unseen scenarios.
Tom: The paper suggests that this relates to a positive distribution shift framework, where we can use mixtures of chain-of-thought lengths instead of a strict curriculum.
Jane: That’s an interesting idea because it allows us to sidestep the difficulty of direct learning by training on different distribution samples all at once.
Lu: That mixture approach could be more efficient than having to train through a very long, ordered sequence of token removals.
The paper's summary: Tom: One key improvement they suggest is to use a specific next-token prediction loss that specifically targets the correct intermediate chain of thought tokens during each phase of training.
Jane: That means instead of just looking at the final answer, we guide the attention matrix to focus on those relevant coordinates from the hidden parity support as we train.
Lalam: This mechanism should help ensure that even when you remove those explicit reasoning steps later, the model has already built a strong internal representation for that computation.
Tom: And they also look at making models better at simulating semiautomata by designing architectures specifically for that purpose, suggesting using only theta log T layers and an MLP width proportional to the prime power factors of n.
Lu: That’s an architectural suggestion to make the simulation of modular counter semiautomata more efficient by exploiting parallel computation related to the structure of modular arithmetic instead of needing a huge width.
Meng: If we can design these specific architectures, it might mean we don't need massive models just to handle certain types of complex mathematical simulations efficiently.
Jane: It also notes that increasing the number of attention heads actually improves the success rate for internalizing chain of thought for a model called C2 under the inductive curriculum.
Tom: So, in summary, they are proposing both better training loss strategies and more specialized network architectures to enhance this internalization process.
The paper's improvements: Tom: So, wrapping up the "Learning through Internalization" paper, it suggests that we can achieve hard learning tasks like parity by first using explicit chain of thought supervision and then internalizing that generation as tokens are removed.
Jane: The key implication is that this gives us a provable way to move from slow, supervised learning to faster, implicit computation for certain problems.
Lu: The idea of the positive distribution shift framework allows us to use mixtures of chain-of-thought lengths instead of relying on a strict curriculum for training.
Meng: From an engineering standpoint, focusing on wider architectures and specific loss functions seems like a practical way to make these internalization methods more reliable when we try to deploy them.
Lalam: It’s about understanding the mechanics of how the model builds its internal computational path so it can compute things directly later.
Tom: I think the challenge ahead is figuring out which task-specific factors, like prime order semiautomata, make it hardest to internalize and whether we can overcome that difficulty with better training techniques.
Jane: It’s definitely a path forward for understanding how complex reasoning emerges in these systems when they're trained on specific structured data.
Conclusion: Tom: So we’ve been talking about how transformers can learn things they’re actually bad at directly, by first learning them through explicit chain of thought and then just absorbing that computation into their weights.
Jane: That's the core idea of "Learning through Internalization," and it sounds like they showed a way to make models compute things in one step after they've been trained on a slower process.
Lu: The main proof is for the sparse parity task, where this one-layer transformer learns something computationally hard without that initial supervision by internalizing chain of thought tokens as those tokens are progressively removed during training.
Meng: So it means we can teach the model a really tricky math problem by giving it a roadmap first, and then forcing it to figure out the shortcut all on its own later.
Lalam: I see how that works; you give the AI a teacher, and then you make the teacher disappear while keeping your test questions tough. That makes sense.
Tom: Exactly. And they also looked at simulating semiautomata, which is like making an AI predict what happens to a machine after many steps by just knowing the input and where it started.
Jane: They found that sometimes this internalization process actually creates shortcuts that make the model perform worse when it sees new data it hasn't been trained on.
Lu: That’s a big caveat; if the shortcut isn't robust, then you haven't really learned the underlying behavior of the semiautomaton.
Meng: From a production standpoint, we worry about those shortcuts because they might work perfectly in our test set but completely fail when we hit real-world traffic that looks different.
Lalam: I’m fairly certain this paper is quite sure about those generalization issues; they show that the mixture of chain of thought lengths can actually help solve that distribution shift problem.
Tom: So, by using a mixture of chain of thought lengths instead of just one fixed path, you can train on a wider variety of problems at once?
Jane: That’s right. It suggests we don't need to follow a strict sequence for training; we can just present the AI with different reasoning paths simultaneously.
Lu: That mixture approach could be more efficient than forcing the model to go through a very long, ordered curriculum of removing those tokens one by one.
Meng: If it’s more efficient, that’s good because it means less time and fewer training cycles to get a good result on complex tasks.
Lalam: It really seems like a way to make the learning process itself more robust against distribution shifts, which is something we need when building reliable AI.
Tom: So, "Learning through Internalization" gives us a provable path for making models compute directly after they learn slowly.
Jane: It’s an interesting direction because it connects explicit reasoning steps to the model's final internal computation structure.
Lu: It opens up a new way to think about how complex procedures are encoded in neural networks, moving beyond just pattern matching.
Meng: We need to keep watching this because if we can make these methods more reliable for real traffic, it changes how we deploy these kinds of reasoning models at scale.
Lalam: I’m confident the evidence they present on the parity task is solid; it shows the mechanism works exactly as described in their analysis.
Tom: Yeah, it’s a lot to digest because you have both the theory on internalization and the practical implications for how we train these systems.
Jane: It really makes you think about what kind of input distribution we need to provide to get these kinds of powerful internal representations.
More episodes
- 2606.11480-FedRAN: Accurate and Efficient Analytic Federated Continual Learning
- 2606.15160-DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- 2606.24152-Autonomous Video Generation for Self-Evolving World Models
- 2606.26432-Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees
- 2606.27866-FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- 2607.04680-A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- 2607.04919-When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters
- 2607.05025-Uncertainty-aware damage identification in short-span bridges via physics-informed variational autoencoder
- 2607.12304-What Does a Temporal Benchmark Score Measure? Decomposing Channel Use in Video VLM Evaluation
- 2608.02877-Graph-of-Thoughts for LLM Causal Discovery with a Fairness-Pathway Case Study