Emergent Generalization by Representation Learning in Artificial Neural Networks
summary
The gist
Emergent generalisation by representation learning in artificial neural networks shows that extracting low-dimensional representations confers a functional advantage for generalisation in learning…
In short
The study investigates whether learning low-dimensional representations helps neural networks generalize to new data. They found that extracting these compact, emergent representations is linked to enhanced predictability of latent dynamics, suggesting this mechanism underlies biological neural network generalization and supports a causal role for learned representations in cognition.
Key concepts
- Low-Dimensional Representations
- These are the compact, abstract features that an artificial neural network learns from high-dimensional input data. The paper explores how learning these smaller representations allows the system to capture essential patterns, enabling it to perform well on unseen or 'out-of-distribution' test data.
- Causal Emergence ($Ψ$)
- This measure quantifies how predictive a coarse representation is of the system's future state compared to just looking at individual parts. High causal emergence suggests that the learned latent representation has a genuine, causal role in predicting the system's dynamics.
- Information Bottleneck Loss
- This loss function tracks the trade-off between compressing input data and retaining useful information. The paper observes a non-monotonic behavior where this loss decreases during the generalization phase, indicating that compression leads to better out-of-distribution performance.
- Neural Manifolds
- These are the low-dimensional spaces within a neural network where similar or related inputs cluster together. The research suggests that learning these manifolds is key to biological learning, as they provide a structured way for the brain to organize and generalize information.
Terminology used across episodes
This episode discusses
- Emergent Generalization by Representation Learning in Artificial Neural Networks · Paper Radio
- Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience · Paper Radio
- The information bottleneck method
- Opening the Black Box of Deep Neural Networks via Information
- Deep Variational Information Bottleneck
- Invariant Risk Minimization
- Do Compressed Representations Generalize Better?
- Estimating Information Flow in Deep Neural Networks
- Deep Networks Always Grok and Here is Why
- Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
- Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition
- Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation
The paper
Emergent Generalization by Representation Learning in Artificial Neural Networks · Read on arXiv
I-X Centre for AI in Science at Imperial College London · Department of Electrical and Electronic Engineering at Imperial College London
Dimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity. These low-dimensional representations have improved the interpretability of population-level coding. Yet whether such low-dimensional representations are biologically relevant and confer functional advantages in learning systems, or merely reflect neuron-level activity, remains contested in neuroscience. We show that an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalisation in a time-series prediction task. Using information-theoretic measures of causal emergence, we characterise the dynamics of this representation across the memorisation-to-generalisation transition, finding a non-monotonic trajectory which shows an initial decrease, a minimum, and a subsequent rise to a maximum, even as prediction loss falls monotonically. This trajectory scales with task complexity, and the magnitude of emergent structure reliably predicts generalisation performance. Analysis of hippocampal CA1 and medial prefrontal activity in rats learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: We're looking at the paper: "Emergent Generalization by Representation Learning in Artificial Neural Networks".
Marcus: Here's the gist of the paper. Emergent generalisation by representation learning in artificial neural networks shows that extracting low-dimensional representations confers a functional advantage for generalisation in learning systems.
Ines: The ability to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.
Marcus: The Core Hypothesis The study explores whether the ability to learn low-dimensional representations of high-dimensional inputs confers any functional advantages in learning systems.
Ines: The authors argue that the ability to generalise to out-of-distribution data by extracting low-dimensional abstract representations may explain the functional need for neural manifolds in biological neural networks.
Paper summary: Ines: So, we're looking at this paper, "Emergent Generalization by Representation Learning in Artificial Neural Networks," and the core idea is that forcing a neural network to learn a low-dimensional representation through an information bottleneck gives it an advantage when it has to generalize.
Marcus: Exactly. The authors are asking if learning these compact, low-dimensional representations actually helps a system learn new things outside of what it was trained on, and they're framing this as explaining why we see neural manifolds in biology >
Yuki: I'm curious about the biological relevance here; can these abstract representations really map onto how our brains work when we learn something new?
Ines: That’s the big question. The paper suggests that this ability to learn compact, distributed representations confers a functional advantage for generalisation, which supports a causal role for these learned representations in cognition >
Marcus: They set up this idea by looking at how networks transition from memorizing data to compressing it, what they call grokking >
Yuki: So if the network is trying to compress things, does that compression act like a filter for learning new things?
Ines: It does. The authors find that when there's this grokking transition, there's an increase in the emergent structure of the learned latent representation >
Marcus: And they link that emergent structure directly to the network’s ability to generalise to out-of-distribution test data >
Yuki: That sounds like what we see when a population learns a new behavior; it needs to find a robust pattern rather than just memorizing specific instances >
Ines: They use information theory, specifically something called causal emergence denoted by Psi, to measure this structure in the bottleneck layer >
Marcus: And they found that post-training, the normalized Psi measure is negatively correlated with the Information Bottleneck loss >
Yuki: So it seems that a higher value of this causal emergence measure means better generalisation performance on new data?
Ines: That's what they are suggesting. They also see this non-monotonic behavior in the information bottleneck loss, where it first increases during early learning and then decreases during the generalisation phase >
Marcus: It’s interesting because it suggests that generalisation happens when both compressed and emergent latent representations are being learned simultaneously >
Yuki: That implies that the system isn't just throwing away irrelevant noise; it's actively building something predictive about the underlying dynamics >
Ines: And they test this in biological data, looking at mouse recordings from the dorsal CA1 and medial PFC during a spatial learning task >
Marcus: They found that for those brain regions, the measure of causal emergence shows a similar pattern: it initially decreases during early learning and then increases later on >
Yuki: So they are seeing this same emergent neural manifold building process in the animal brains as they do in their artificial network simulations >
Ines: It points toward the idea that learning these emergent neural manifolds might be a key feature of learning in biological neural networks >
Marcus: The authors conclude that this ability to learn low-dimensional representations is tied to enhanced predictability of latent dynamics, which defines the representation as causally emergent >
Yuki: They also noted that a higher value of Psi is observed for more complex tasks, meaning harder prediction jobs require learning more emergent latent representations during training >
Ines: And they found that this generalisation ability is robust across different dynamical systems and only changes based on task complexity and the inherent chaoticity of the system being predicted >
Marcus: They also noted that the ratio of generalisation loss to validation loss stays constant, around one, which suggests the network's generalisation skill isn't tied to learning one specific attractor >
Yuki: It means it learns a common latent space structure rather than just memorizing one specific trajectory >
Ines: The limitation they point out is that the model doesn't perform closed-loop prediction as a generative model, so we don't see how these representations are refined online for that yet >
Marcus: Also, they suggest future work should explore whether this out-of-distribution generalisation can happen without learning a compressed and emergent latent representation >
Yuki: It really opens up the question of how those neural manifolds relate to causal emergence in the temporal domain, which is a deep area for population genetics too >
Conclusion: Ines: So we’re wrapping up this look at "Emergent Generalization by Representation Learning in Artificial Neural Networks." The main point is that forcing a network to learn a compact, low-dimensional representation gives it a functional advantage when it has to generalize.
Marcus: Yeah, they’re arguing that this ability to find those small, distributed patterns helps systems learn new things outside of what they were trained on. It supports the idea that learned representations actually have a causal role in how we think and learn.
Yuki: I’m thinking about what this means for population genetics. If these networks are learning emergent structures, does that translate to how a species learns a new trait across generations?
Ines: It suggests that the process isn't just memorizing specific inputs; it's building something more robust about the underlying structure of the data. The paper shows this happens when there’s a transition from memorization to compression, which they call grokking.
Marcus: And that transition is where the emergent structure in those latent representations really kicks in, which is what makes generalisation possible on new test data. They use information theory to measure this emergence and link it directly to how well the network performs on things it hasn't seen before.
Yuki: So the bigger picture is that these neural manifolds aren't just mathematical shortcuts; they are a way for the system to capture a stable, predictive model of its environment. That’s significant when you look at how complex systems evolve or adapt.
Ines: Exactly. And they found that this generalisation skill stays pretty consistent across different kinds of data, as long as the task complexity is what changes—not just which specific set of training examples we used. It seems like it’s learning a common latent space for solving a class of problems.
Marcus: The numbers showed that prediction accuracy keeps going up when you increase the size of that bottleneck layer, until it matches the dimension of the input data. That suggests an overcomplete low-dimensional basis is helping it find a good solution. But they also flagged that this model doesn't actually perform closed-loop prediction yet, so we don't know how these representations get refined online for that next step.
Yuki: The limitation on closed-loop prediction is a real one. So we’re seeing the structure emerge in training, but applying it dynamically is another puzzle for the researchers to tackle. Where do you think that refinement process fits into evolutionary learning models?
More episodes
- 2610.33881-Sensorimotor behavior from an untrained brain-and-nerve-cord connectome
- 2610.40344-Morphological Features of Murine Pulmonary Arterial Networks for Diagnosing Vascular Disease
- Daily Summary for 2026-11-19
- 2609.10193-Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
- 2610.38236-Generative discovery of chemical reaction networks
- 2507.11176-An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2504.03590-Towards a Unified Framework for Determining Conformational Ensembles of Disordered Proteins
- 2603.15217-A multiscale discrete-to-continuum framework for structured population models