Emergent Generalization by Representation Learning in Artificial Neural Networks

arXiv:2607.10430 · q-bio.NC, cs.IT, cs.LG, cs.NE, math.IT, nlin.CD · Submitted 2026-07-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Emergent Generalization by Representation Learning in Artificial Neural Networks".

Marcus: Here's the gist of the paper. Emergent generalisation by representation learning in artificial neural networks shows that extracting low-dimensional representations confers a functional advantage for generalisation in learning systems.

Ines: The ability to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.

Marcus: The Core Hypothesis The study explores whether the ability to learn low-dimensional representations of high-dimensional inputs confers any functional advantages in learning systems.

Ines: The authors argue that the ability to generalise to out-of-distribution data by extracting low-dimensional abstract representations may explain the functional need for neural manifolds in biological neural networks.

Paper summary: Ines: So, we're looking at this paper, "Emergent Generalization by Representation Learning in Artificial Neural Networks," and the core idea is that forcing a neural network to learn a low-dimensional representation through an information bottleneck gives it an advantage when it has to generalize.

Marcus: Exactly. The authors are asking if learning these compact, low-dimensional representations actually helps a system learn new things outside of what it was trained on, and they're framing this as explaining why we see neural manifolds in biology >

Yuki: I'm curious about the biological relevance here; can these abstract representations really map onto how our brains work when we learn something new?

Ines: That’s the big question. The paper suggests that this ability to learn compact, distributed representations confers a functional advantage for generalisation, which supports a causal role for these learned representations in cognition >

Marcus: They set up this idea by looking at how networks transition from memorizing data to compressing it, what they call grokking >

Yuki: So if the network is trying to compress things, does that compression act like a filter for learning new things?

Ines: It does. The authors find that when there's this grokking transition, there's an increase in the emergent structure of the learned latent representation >

Marcus: And they link that emergent structure directly to the network’s ability to generalise to out-of-distribution test data >

Yuki: That sounds like what we see when a population learns a new behavior; it needs to find a robust pattern rather than just memorizing specific instances >

Ines: They use information theory, specifically something called causal emergence denoted by Psi, to measure this structure in the bottleneck layer >

Marcus: And they found that post-training, the normalized Psi measure is negatively correlated with the Information Bottleneck loss >

Yuki: So it seems that a higher value of this causal emergence measure means better generalisation performance on new data?

Ines: That's what they are suggesting. They also see this non-monotonic behavior in the information bottleneck loss, where it first increases during early learning and then decreases during the generalisation phase >

Marcus: It’s interesting because it suggests that generalisation happens when both compressed and emergent latent representations are being learned simultaneously >

Yuki: That implies that the system isn't just throwing away irrelevant noise; it's actively building something predictive about the underlying dynamics >

Ines: And they test this in biological data, looking at mouse recordings from the dorsal CA1 and medial PFC during a spatial learning task >

Marcus: They found that for those brain regions, the measure of causal emergence shows a similar pattern: it initially decreases during early learning and then increases later on >

Yuki: So they are seeing this same emergent neural manifold building process in the animal brains as they do in their artificial network simulations >

Ines: It points toward the idea that learning these emergent neural manifolds might be a key feature of learning in biological neural networks >

Marcus: The authors conclude that this ability to learn low-dimensional representations is tied to enhanced predictability of latent dynamics, which defines the representation as causally emergent >

Yuki: They also noted that a higher value of Psi is observed for more complex tasks, meaning harder prediction jobs require learning more emergent latent representations during training >

Ines: And they found that this generalisation ability is robust across different dynamical systems and only changes based on task complexity and the inherent chaoticity of the system being predicted >

Marcus: They also noted that the ratio of generalisation loss to validation loss stays constant, around one, which suggests the network's generalisation skill isn't tied to learning one specific attractor >

Yuki: It means it learns a common latent space structure rather than just memorizing one specific trajectory >

Ines: The limitation they point out is that the model doesn't perform closed-loop prediction as a generative model, so we don't see how these representations are refined online for that yet >

Marcus: Also, they suggest future work should explore whether this out-of-distribution generalisation can happen without learning a compressed and emergent latent representation >

Yuki: It really opens up the question of how those neural manifolds relate to causal emergence in the temporal domain, which is a deep area for population genetics too >

Conclusion: Ines: So we’re wrapping up this look at "Emergent Generalization by Representation Learning in Artificial Neural Networks." The main point is that forcing a network to learn a compact, low-dimensional representation gives it a functional advantage when it has to generalize.

Marcus: Yeah, they’re arguing that this ability to find those small, distributed patterns helps systems learn new things outside of what they were trained on. It supports the idea that learned representations actually have a causal role in how we think and learn.

Yuki: I’m thinking about what this means for population genetics. If these networks are learning emergent structures, does that translate to how a species learns a new trait across generations?

Ines: It suggests that the process isn't just memorizing specific inputs; it's building something more robust about the underlying structure of the data. The paper shows this happens when there’s a transition from memorization to compression, which they call grokking.

Marcus: And that transition is where the emergent structure in those latent representations really kicks in, which is what makes generalisation possible on new test data. They use information theory to measure this emergence and link it directly to how well the network performs on things it hasn't seen before.

Yuki: So the bigger picture is that these neural manifolds aren't just mathematical shortcuts; they are a way for the system to capture a stable, predictive model of its environment. That’s significant when you look at how complex systems evolve or adapt.

Ines: Exactly. And they found that this generalisation skill stays pretty consistent across different kinds of data, as long as the task complexity is what changes—not just which specific set of training examples we used. It seems like it’s learning a common latent space for solving a class of problems.

Marcus: The numbers showed that prediction accuracy keeps going up when you increase the size of that bottleneck layer, until it matches the dimension of the input data. That suggests an overcomplete low-dimensional basis is helping it find a good solution. But they also flagged that this model doesn't actually perform closed-loop prediction yet, so we don't know how these representations get refined online for that next step.

Yuki: The limitation on closed-loop prediction is a real one. So we’re seeing the structure emerge in training, but applying it dynamically is another puzzle for the researchers to tackle. Where do you think that refinement process fits into evolutionary learning models?

I-X Centre for AI in Science at Imperial College London · Department of Electrical and Electronic Engineering at Imperial College London

q-bio.NC, cs.IT, cs.LG, cs.NE, math.IT, nlin.CD

Submitted: 2026-07-11

Updated: 2026-11-05

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Emergent generalisation by representation learning in artificial neural networks shows that extracting low-dimensional representations confers a functional advantage for generalisation in learning…

Key concepts

Low-Dimensional Representations
These are the compact, abstract features that an artificial neural network learns from high-dimensional input data. The paper explores how learning these smaller representations allows the system to capture essential patterns, enabling it to perform well on unseen or 'out-of-distribution' test data.
Causal Emergence ($Ψ$)
This measure quantifies how predictive a coarse representation is of the system's future state compared to just looking at individual parts. High causal emergence suggests that the learned latent representation has a genuine, causal role in predicting the system's dynamics.
Information Bottleneck Loss
This loss function tracks the trade-off between compressing input data and retaining useful information. The paper observes a non-monotonic behavior where this loss decreases during the generalization phase, indicating that compression leads to better out-of-distribution performance.
Neural Manifolds
These are the low-dimensional spaces within a neural network where similar or related inputs cluster together. The research suggests that learning these manifolds is key to biological learning, as they provide a structured way for the brain to organize and generalize information.

Terminology

Summary

Emergent generalisation by representation learning in artificial neural networks shows that extracting low-dimensional representations confers a functional advantage for generalisation in learning systems. p. 0

Where the paper says it

Analysis of CA1 hippocampal activity in mice learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.

The ability to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition. p. 0
Where the paper says it

Analysis of CA1 hippocampal activity in mice learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.

The Core Hypothesis

The study explores whether the ability to learn low-dimensional representations of high-dimensional inputs confers any functional advantages in learning systems. The authors argue that the ability to generalise to out-of-distribution data by extracting low-dimensional abstract representations may explain the functional need for neural manifolds in biological neural networks. p. 1

Where the paper says it

In this work, we explore the relationship between causal emergence of learned latent representations and the network’s ability to generalise to out-of-distribution test data. We argue that this ability to generalise to out-of-distribution data by extracting low-dimensional abstract representations may explain the functional need for neural manifolds in biological neural networks.

This is framed by investigating the relationship between causal emergence of learned latent representations and the network’s ability to generalise to out-of-distribution test data. p. 1
Where the paper says it

This suggests that when the optimal low-dimensional representations learned by the network are causally emergent, they may be able to leverage the enhanced predictive power of the low-dimensional representation to generalise to out-of-distribution test data. In this work, we explore the relationship between causal emergence of learned latent representations and the network’s ability to generalise to out-of-distribution test data.

Information Bottleneck and Generalisation

The research investigates the relationship between compression and generalisation, noting that networks transition between a rapid fitting phase (memorisation) and a slow compression phase (generalisation) referred to as grokking. The authors find that the grokking transition is associated with an increase in the emergent structure of the learned latent representation, which in turn is correlated with the network’s ability to generalise to out-of-distribution test data. p. 1

Where the paper says it

However, over-compression in the generalisation phase can enhance discriminability at the expense of transferability31. While we do not address the first two criticisms in this work, we do explore the non-monotonic relationship between compression and generalisation further in the context of next-step prediction and identify that the grokking transition is associated with an increase in the emergent structure of the learned latent representation, which in turn is correlated with the network’s ability to generalise to out-of-distribution test data.

Furthermore, they show that a similar non-monotonic behaviour of the Information Bottleneck loss is observed where it initially increases during early stages of learning and then decreases during the generalisation phase. p. 6
Where the paper says it

D A similar non-monotonic behaviour is observed for the Information Bottleneck loss. The IB loss initially increases during the early stages of learning and then starts decreasing during the generalisation phase.

Causal Emergence Dynamics

The study uses novel measures of causal emergence based on information theory to characterize learned latent representations in the bottleneck layer. The measure of causal emergence, denoted by Ψ, quantifies the degree to which any coarse-grained representation of a system is predictive of its own future state compared to the sum of marginal predictability. p. 3

Where the paper says it

Learned Latent Representations are Causally Emergent Using novel measures of causal emergence based on information theory, we show that the learned latent representations in the bottleneck layer become increasingly causally emergent during learning, and that the network’s ability to generalise to out-of-distribution test data is correlated with the degree to which the latent representation is emergent. The Ψ measure of causal emergence quantifies the degree to which any coarse-grained representation of a system is predictive of its own future state (macro predictability), as compared to the sum of marginal predictability of the individual parts (micro predictability) of the system (see Equation 1).

They find that the normalised Ψ measure post-training is negatively correlated with the Information Bottleneck loss, suggesting that generalisation is associated with the learning of both compressed and emergent latent representations. p. 5
Where the paper says it

However, the Ψ transition preceeds the IB transition during training. Across the trained models, we find that the normalised Ψ measure post-training is negatively correlated with the IB loss, which suggests that generalisation is associated with the learning of both compressed and emergent latent representations (see Figure 4 E).

Biological Validation

To test the findings beyond synthetic data, they analyze data from mice recorded in the dorsal CA1 and medial PFC during a spatial learning task. They find that the measure of causal emergence exhibits a similar non-monotonic behaviour during learning, where it initially decreases during the early stages of learning and then increases in the later stages, which is consistent with their findings in CA1 and the medial PFC. p. 6

Where the paper says it

We then estimate the measure of causal emergence of the bottleneck states in the trained model for each of the 8 training sessions, and track how it changes along with the decoder prediction MSE during learning across sessions. We find that the measure of causal emergence exhibits a similar non-monotonic behaviour during learning, where it initially decreases during the early stages of learning and then increases in the later stages, which is consistent with our findings in the 7/14

This suggests that the learning of emergent neural manifolds may be a key feature of learning in biological neural networks. p. 1
Where the paper says it

By characterising the informationtheoretic properties of the learned latent representation, we show that the learned latent representation becomes causally emergent during training, and explain the network’s ability to generalise to out-of-distribution test data. Finally, we show that the similar non-monotonic behaviour of the causal emergence of latent representations is observed in CA1 and medial PFC of mice during a spatial learning task, which suggests that the learning of emergent neural manifolds may be a key feature of learning in biological neural networks.

Conclusion on Representation Learning

The authors conclude that the ability to learn low-dimensional representations is associated with enhanced predictability of latent dynamics, which characterizes the learned latent representation as causally emergent. They also find that a higher value of Ψ is observed for more complex tasks, indicating that more difficult prediction tasks require the learning of more emergent latent representations during training. p. 5

Where the paper says it

Furthermore, we find that the degree of emergent behaviour exhibited by the trained network increases with the complexity of the task (see Figure 4 C). Indicating that more difficult prediction tasks require the learning of more emergent latent representations during training.

The results suggest that the learning of emergent neural manifolds may be a key feature of learning in biological neural networks. p. 7
Where the paper says it

These results are in line with the previous findings that show a similar non-monotonic behaviour of the neural-latent correlation of the neural manifold during learning in the same dataset41. These results suggest that the learning of emergent latent representations may be a key feature of learning in biological neural networks.

The study opens new questions on the relationship of neural manifolds to causal emergence in the temporal domain. p. 7
Where the paper says it

Recent work has shown that intermediate refinement of latent representations can lead to optimal generalisation43. This study also opens new questions on the relationship of neural manifolds to causal emergence in the temporal domain.

Methods Overview

The methodology involves using a modified reservoir computing architecture with a bottleneck layer trained using backpropagation to learn low-dimensional latent representations. The input data is projected to the reservoir layer using a fixed random projection Win, and the states of the reservoir layer Yt are then projected to the bottleneck layer using a trainable linear transformation Wz. p. 9

Where the paper says it

y(t) = Rx(t) +ε ε ∼ N (0,σ 2 I) (3) Model architecture and training The model architecture used in this study is a modified reservoir computing architecture, which consists of a fixed input and recurrent reservoir layer, followed by a trainable bottleneck layer and a linear readout layer. The input data Xt is projected to the reservoir layer using a fixed random projection Win, and the states of the reservoir layer Yt are then projected to the bottleneck layer using a trainable linear transformation Wz .

The bottleneck layer is trained using backpropagation while the linear readout layer is trained using ridge regression to predict the next time step of the input data. p. 1
Where the paper says it

For the purposes of this study, we use a modified Reservoir Computing Architecture (see Figure 1 C), with a bottleneck layer which is trained using backpropagation. While the linear readout layer is trained using ridge regression to predict the next time step of the input data, the bottleneck layer is trained to learn a low-dimensional predictive latent representation of the input data.

Information-theoretic measures are used to quantify causal emergence and information bottleneck, based on mutual information. p. 9
Where the paper says it

Information theoretic measures We use the information-theoretic measures of causal emergence and information bottleneck to characterise the learned latent representations in the bottleneck layer. Both of these measures are based on mutual information, which quantifies the amount of information shared between two random variables.

The study also compares the similarity between the learned latent representation Vt and the true latent states of the dynamical system using Canonical Correlation Analysis (CCA). p. 3
Where the paper says it

We can extract the learned low-dimensional representation by performing a PCA and projecting the bottleneck states to a three-dimensional space. We compare the similarity between the low-dimensional representation learned by the bottleneck layer and the true latent states of the dynamical system using Canonical Correlation Analysis (CCA).

Data Generation Details

Six different three-dimensional dynamical systems were simulated to generate input data, including chaotic Lorenz, Rössler, and Sprott B attractors for training and chaotic Chen, Sprott A, and quasi-periodic Lissajous attractors for testing. The complexity of the task is modulated by subsampling the simulated attractor timeseries to keep exactly Ntau equally spaced timesteps before the autocorrelation drops below 1/e. p. 9

Where the paper says it

dx(t) dt = f(x(t)) (2) In order to systematically modulate the complexity of the task, we downsampled the simulated attractor (integration timestep = 0. 001) timeseries to keep exactly Ntau equally spaced timesteps of the attractor timeseries before the autocorrelation of the timeseries drops below 1/e.

The high-dimensional input data was then projected to a 10-dimensional space using a random orthogonal projection R, with some added Gaussian noise ε. p. 9
Where the paper says it

The subsampled attractor timeseries were then projected to a 10-dimensional space using a random orthogonal projection R, with some added Gaussian noise ε. The high-dimensional input data was then used to train and test the network.

The reservoir dynamics are defined by the recurrent dynamics Yt = htanh(WresYt−1 +WinXt) + (1−h)Yt−1. p. 9
Where the paper says it

The validation and test sets are generated independently from the training set, and consist of random rotations of the training attractors and held-out test attractors, respectively. The reservoir dynamics are defined by the following recurrent dynamics, Yt = htanh(WresYt−1 +WinXt) + (1−h)Yt−1 (4) Here, h is the leak rate of the reservoir, which controls the timescale of the reservoir dynamics.

The reservoir size was set to 500 across all experiments. p. 9
Where the paper says it

The reservoir weights Wres are initialised from a normal distribution. We use the reservoir size of 500 across all experiments, and the hyperparameters of the reservoir, including the leak rate, spectral radius, sparsity, the input scaling and the input bias, are optimised using the Tree-structured Parzen Estimator (TPE) algorithm implemented in Optuna.

The hyperparameters of the reservoir were optimized using the Tree-structured Parzen Estimator (TPE) algorithm implemented in Optuna. p. 9
Where the paper says it

The reservoir weights Wres are initialised from a normal distribution. We use the reservoir size of 500 across all experiments, and the hyperparameters of the reservoir, including the leak rate, spectral radius, sparsity, the input scaling and the input bias, are optimised using the Tree-structured Parzen Estimator (TPE) algorithm implemented in Optuna.

The study also used spiking neural data recorded from dorsal CA1 and medial PFC of mice during a W-maze spatial learning task. p. 5
Where the paper says it

Emergent Representations in spatial learning tasks in mice To test the validity of our findings beyond synthetic data and artificial neural networks, we test whether the non-monotonic behaviour of the measure of causal emergence during learning is also observed in biological neural networks. We analyse the data from a recent study which recorded the activity of neurons from the dorsal CA1 and the medial PFC of mice during a W-maze spatial learning task40.

The processed firing rates were used as input to a simple network with a bottleneck layer and linear readout decoder. p. 5
Where the paper says it

The mice are trained for over 8 training sessions, and the behavioural measures (position and velocity) and performance measures (correct and incorrect trials) are recorded along with the neural activity from the CA1 and the medial PFC (see Figure 5 B). We use the preprocessed firing rates of the neurons from one of the regions (CA1 or medial PFC) as input to a simple network with a linear bottleneck encoder and ridge readout decoder, and train the network to 6/14

The statistical analysis involved fitting a linear mixed effects model to predict the generalisation loss from macro and micro predictability. p. 5
Where the paper says it

Therefore, we first normalise the macro and micro predictability by that of the true latent attractor. We then use a linear mixed effects model to account for the random effect of the attractor on the prediction loss.

The quadratic coefficient of the fitted model was compared between the measure of causal emergence and the MSE in both regions. p. 8
Where the paper says it

The quadratic coefficient is significantly larger for the Ψ measure as compared to the MSE in both regions. F,I Using the fitted parameters of the linear model, the estimated minima of the Ψ measure significantly precede the minima of the MSE in both PFC (F) and CA1 (I).

The study used a permutation test to compare the minima of the Ψ measure and the MSE. p. 7
Where the paper says it

We find that the quadratic coefficient is significantly larger for the emergence measure as compared to the MSE in both CA1 and the medial PFC, which suggests that the emergence measure shows a significant non-monotonic increase as compared to the MSE (see Figures 5 E,H). Consequently, we also observe the minima of the emergence measure to significantly precede the minima of the MSE in both regions (see Figures 5 F,I).

The quadratic coefficient was significantly larger for the emergence measure as compared to the MSE in both CA1 and the medial PFC. p. 7
Where the paper says it

To confirm the difference in the non-monotonic behaviour between the emergence and the MSE, we fit a quadratic linear model to the measures for each subject and each brain region. We find that the quadratic coefficient is significantly larger for the emergence measure as compared to the MSE in both CA1 and the medial PFC, which suggests that the emergence measure shows a significant non-monotonic increase as compared to the MSE (see Figures 5 E,H).

The minima of the emergence measure significantly precede the minima of the MSE in both regions. p. 7
Where the paper says it

We find that the quadratic coefficient is significantly larger for the emergence measure as compared to the MSE in both CA1 and the medial PFC, which suggests that the emergence measure shows a significant non-monotonic increase as compared to the MSE (see Figures 5 E,H). Consequently, we also observe the minima of the emergence measure to significantly precede the minima of the MSE in both regions (see Figures 5 F,I).

Key Findings on Generalisation

The network without a bottleneck layer is able to learn the training data but is not able to generalise to random rotations of the training attractors or to the held out dynamical systems. The size of the bottleneck layer controls generalisation, and prediction accuracy decreases with increasing task complexity using subsampling Ntau points before autocorrelation decay. p. 3

Where the paper says it

This suggests that the bottleneck layer and representation learning are a necessary feature for enabling generalisation. The network’s prediction accuracy decreases with increasing task complexity using subsampling Ntau points before the autocorrelation decay.

The next step prediction loss monotonically decreases for both unseen rotations of training attractors as well as for the held out test systems when the size of the bottleneck layer is increased, reaching a minimum when it matches the dimensionality of the input. p. 3
Where the paper says it

This requires the network to learn a rotation-invariant representation of the latent states, which is key for out-of-distribution generalisation (see Figure 2 C). As we increase the size of the bottleneck layer, the next step prediction loss monotonically decreases for both unseen rotations of training attractors as well as for the held out dynamical systems, and reaches a minimum when the size of the bottleneck layer matches the dimensionality of the input (see Figure 2 D).

This suggests that an overcomplete low-dimensional basis allows it to learn a general solution in a common latent space. p. 3
Where the paper says it

Any further increase in the size of the bottleneck layer causes a slight increase in the next step prediction loss. The true dimensionality of the bottleneck layer is still low rank and matches the dimensionality of the latent dynamical system, which suggests that the bottleneck learns an overcomplete low-dimensional basis of the dynamics, which allows it to learn a general solution in a common latent space.

Functional Advantages

The study demonstrates that the ability to generalise using low-dimensional latent representation is associated with the enhanced predictability of the latent dynamics, which characterizes the learned latent representation as causally emergent. The authors show that this ability to generalise is robust across different dynamical systems and is modulated only by task complexity and the inherent chaoticity of the dynamical system being predicted. p. 7

Where the paper says it

The performance is robust across different dynamical systems and is modulated only by task complexity and the inherent chaoticity of the dynamical system being predicted. Crucially, we show that this ability to generalise using low-dimensional latent representation is associated with the enhanced predictability of the latent dynamics, which characterises the learned latent representation as causally emergent.

The performance is robust across different dynamical systems and is modulated only by task complexity and the inherent chaoticity of the latent dynamical system being predicted. p. 7
Where the paper says it

Crucially, the model shows zero-shot generalisation to held-out dynamical systems, which suggests that the network is able to learn a general solution in a common latent space. The performance is robust across different dynamical systems and is modulated only by task complexity and the inherent chaoticity of the dynamical system being predicted.

The ratio of generalisation loss to validation loss is constant and close to 1, which suggests that the generalisation ability of the network is robust and not dependent on learning a specific attractor. p. 3
Where the paper says it

When comparing the Pearson correlation between validation loss LV and generalisation loss LG across different trained models for different task complexities, we find that the correlation is low and positive for high task complexities, and low and negative for low task complexities (see Figure 3 C). Furthermore, when looking at the ratio of generalisation loss to validation loss, across different trained models and task complexities, we find the ratio is constant and close to 1, which suggests that the generalisation ability of the network is robust and not dependent on learning a specific attractor (see Figure 3 D).

Limitations and Future Work

The model achieves zero-shot generalisation to held-out dynamical systems, suggesting it learns a general solution in a common latent space. However, the model is unable to perform closed-loop prediction as a generative model. p. 7

Where the paper says it

Future work should explore whether such out-of-distribution generalisation can be achieved without learning a compressed and emergent latent representation. Finally, although the model achieves zero-shot generalisation to held-out dynamical systems, it is unable to perform closed-loop prediction as a generative model.

Further work is needed on how the latent representations can be refined online to enable generative predictions. p. 7
Where the paper says it

Finally, although the model achieves zero-shot generalisation to held-out dynamical systems, it is unable to perform closed-loop prediction as a generative model. Further work is needed on how the latent representations can be refined online to enable generative predictions.

The study also opens new questions on the relationship of neural manifolds to causal emergence in the temporal domain. p. 7
Where the paper says it

Recent work has shown that intermediate refinement of latent representations can lead to optimal generalisation43. This study also opens new questions on the relationship of neural manifolds to causal emergence in the temporal domain.

Future work should explore whether such out-of-distribution generalisation can be achieved without learning a compressed and emergent latent representation. p. 7
Where the paper says it

Furthermore, while we show that the learned latent representations are causally emergent, it is unclear if this is a necessary condition for generalisation. Future work should explore whether such out-of-distribution generalisation can be achieved without learning a compressed and emergent latent representation.

The paper is available as arXiv:2607. p. 0
Where the paper says it

This debate is continually evolving, and new methods and techniques are being developed to address these concerns, which will further help to clarify the role of neural manifolds in neuroscience. arXiv:2607.

10430v1 [q-bio. p. 0
Where the paper says it

arXiv:2607. 10430v1 [q-bio.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improve generalization to out-of-distribution data by implementing an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation, as this is necessary for rotational and out-ofdistribution generalisation in a time-series prediction task. The improved system can achieve zero-shot generalisation to held-out dynamical systems by learning a general solution in a common latent space.

  2. Enhance temporal prediction accuracy by utilizing learned causal emergence of latent representations, as this is correlated with the network's ability to generalize. The system can leverage the fact that the learned latent representation becomes causally emergent during training, which is linked to improved performance on unseen data.

  3. Optimize learning dynamics by incorporating a measure of causal emergence into the training objective, specifically using the Psi measure, as it shows a non-monotonic behaviour during learning. This allows the system to learn more emergent latent representations during the grokking transition and correlate this with increased generalisation performance.

Abstract

Dimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity. These low-dimensional representations have improved the interpretability of population-level coding. Yet whether such low-dimensional representations are biologically relevant and confer functional advantages in learning systems, or merely reflect neuron-level activity, remains contested in neuroscience. We show that an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalisation in a time-series prediction task. Using information-theoretic measures of causal emergence, we characterise the dynamics of this representation across the memorisation-to-generalisation transition, finding a non-monotonic trajectory which shows an initial decrease, a minimum, and a subsequent rise to a maximum, even as prediction loss falls monotonically. This trajectory scales with task complexity, and the magnitude of emergent structure reliably predicts generalisation performance. Analysis of hippocampal CA1 and medial prefrontal activity in rats learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.

Sources

Related papers