Schema-based active inference supports rapid generalization of experience and frontal cortical coding of abstract structure

summary

The gist

Schemas – abstract relational structures that capture the commonalities across experiences – are thought to underlie humans’ and animals’ ability to rapidly generalize knowledge…

In short

This work introduces Schema-based Hierarchical Active Inference (S-HAI), a computational framework combining predictive processing and active inference with schemas. The system demonstrates rapid knowledge generalization in spatial tasks by mapping abstract task structures onto physical locations. It shows how the agent learns to reuse or create new mappings through a mixture of grounding likelihoods, reproducing relevant neural codes in the rodent medial prefrontal cortex.

Key concepts

Schemas
Schemas are abstract relational structures formed from experience that capture commonalities across different events. They allow humans and animals to organize memories and rapidly apply existing knowledge to new, similar situations without relearning everything from scratch.
Active Inference
Active inference is a cognitive framework where agents minimize 'variational free energy.' Agents use a generative model of their beliefs about causes, states, and actions to predict outcomes. They learn by updating this model to reduce prediction errors and select actions that minimize expected future free energy.
Grounding Likelihood
This mechanism links the high-level abstract schema (like a task structure) to specific physical locations in the environment. It allows the agent to rapidly map an abstract goal structure onto various novel physical settings, enabling flexible generalization and deciding when to reuse old mappings or create new ones.

Terminology used across episodes

This episode discusses

The paper

Schema-based active inference supports rapid generalization of experience and frontal cortical coding of abstract structure · Read on arXiv

Toon Van de Maele, Tim Verbelen, Dileep George, Giovanni Pezzulo

VERSES Research Lab · Google Paradigms of Intelligence Institute of Cognitive Sciences and Technologies National Research Council

Schemas -- abstract relational structures that capture the commonalities across experiences -- are thought to underlie humans' and animals' ability to rapidly generalize knowledge, rebind new experiences to existing structures, and flexibly adapt behavior across contexts. Despite their central role in cognition, the computational principles and neural mechanisms supporting schema formation and use remain elusive. Here, we introduce schema-based hierarchical active inference (S-HAI), a novel computational framework that combines predictive processing and active inference with schema-based mechanisms. In S-HAI, a higher-level generative model encodes abstract task structure, while a lower-level model encodes spatial navigation, with the two levels linked by a grounding likelihood that maps abstract goals to physical locations. Through a series of simulations, we show that S-HAI generates candidate analogues to key behavioral signatures of rapid schema-based generalization in spatial navigation tasks, including the ability to flexibly remap abstract schemas onto novel contexts, resolve goal ambiguity, and balance reuse versus accommodation of novel mappings. Crucially, S-HAI also qualitatively recapitulates prominent neural codes reported in rodent medial prefrontal cortex during a schema-dependent navigation and decision task, including task-invariant goal-progress cells, goal-and-spatially conjunctive cells, and place-like codes at the lower level. Taken together, these results provide a candidate mechanistic account of schema-based learning and inference that bridges behavior, neural data, and theory. More broadly, our findings suggest that schema formation and generalization may arise from predictive processing principles, potentially implemented hierarchically across cortical and hippocampal circuits.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Schema-based active inference supports rapid generalization of experience and frontal cortical coding of abstract structure".

Marcus: Here's the gist of the paper. Schemas – abstract relational structures that capture the commonalities across experiences – are thought to underlie humans’ and animals’ ability to rapidly generalize knowledge, rebind new experiences to existing structures, and flexibly adapt behavior across contexts Schemas are defined as relational knowledge structures that capture abstracted commonalities across multiple experiences They enable individuals to organize and interpret experiences in memory and to generalize to novel situations that share underlying structure, even when differing in sensory details (Piaget, 1952; Bartlett, 1932) The crucial insight is that schemas are formed through experience, encoding inferred relational task structure while abstracting away from low-level (sensory) details Schemas not only organize experiences into a rich set of relations but also serve as templates that enable the rapid assimilation of new experiences This type of learning—or assimilation—requires only mapping the low-level sensory details of a novel experience onto the abstract relational structure of an existing schema, enabling the fast (ideally, one-shot) reuse and generalization of knowledge to new contexts Furthermore, besides assimilation—fitting new information into existing schemas—there is a second process—called accommodation—which involves creating new schemas or modifying existing ones when new information does not fit The S-HAI Computational Framework Schema-based hierarchical active inference (S-HAI) is introduced as a novel computational framework that combines predictive processing and active inference with schema-based mechanisms In S-HAI, a higher-level generative model encodes abstract task structure, while a lower-level model encodes spatial navigation, with the two levels linked by a grounding likelihood that maps abstract goals to physical locations The agent is organized hierarchically: the higher level (Level 2) is responsible for schema learning and navigation in an abstract task space, while the lower level (Level 1) encodes spatial information and supports navigation in physical space Behavioral Validation of Generalization Through a series of simulations, S-HAI reproduces key behavioral signatures of rapid schema-based generalization in spatial navigation tasks, including the ability to flexibly remap abstract schemas onto novel contexts, resolve goal ambiguity, and balance reuse versus accommodation of novel mappings The first simulation shows that schema-based inference in the S-HAI agent enables fast generalization in the ABCD task of El-Gaby et al.

Ines: (2024) The second simulation demonstrates the efficacy of schemabased inference in a more demanding ABCB task, in which two goals can occupy the same location, as in spatial alternation tasks (Jadhav et al., 2012) Both S-HAI agents required fewer steps per trial (Figure 2c) and accumulated more reward (Figure 2d) than the HAI-20 agent and the Random baseline Neural Signatures in Cortical Coding S-HAI reproduces prominent neural codes reported in rodent medial prefrontal cortex during a schema-dependent navigation and decision task, including task-invariant goal-progress cells, goal-and-spatially conjunctive cells, and place-like codes at the lower level The model simulates neural activity that tracks progress toward goals when considering the agent’s belief over the inductive cost, which reveals distinct populations of “goal-progress cells” tuned to early, mid, or late phases of progress toward any goal Furthermore, Level 1 spatial activations encode the agent’s position within the environment in a way analogous to hippocampal place cells (O’Keefe and Dostrovsky, 1971) Mechanisms for Flexibility and Reuse The grounding likelihood affords schema-based generalization by enabling the agent to rapidly map its abstract schema (e.g., ABCD) to each novel spatial configuration of the goals rather than relearning from scratch on each trial The S-HAI MoGL agent maintains a mixture of grounding likelihoods that expands over time using a truncated Dirichlet process, allowing it to flexibly create new grounding likelihoods when encountering novel problems or reuse existing ones when encountering previously seen problems This mechanism demonstrates the ability to decide when to reuse an existing mapping or create a novel one, capturing at least basic signatures of assimilation and accommodation processes as conceived by Piaget (1952) Future Directions and Limitations The current S-HAI agent has several limitations that can be addressed in future research, including the fact that it currently implements only a single schema, whereas the SHAI framework allows for an extension to maintaining a mixture over multiple schemas The model also focuses primarily on schema learning and inference in the frontal cortex, and future work could extend S-HAI to provide a system-level model addressing schema-based processes beyond the frontal cortex, encompassing other relevant brain regions such as the hippocampus and entorhinal cortex Further research could investigate how schema-based mechanisms can be reused to support navigation in abstract, conceptual spaces The model also generates novel empirical predictions concerning the behavior and neural representations that might be observed in animals performing the ABCD task Active inference is a framework that describes cognitive processes and brain dynamics in living organisms, in terms of the minimization of an information-theoretic functional: variational free energy (Parr et al., 2022) Agents are endowed with a generative model: a probabilistic model that encodes the internal belief over the causal relationship between hidden states, actions, and subsequent observed outcomes During perception, the generative model is inverted to find the hidden causes that give rise to the observed outcomes by minimizing the free energy with respect to the approximate posterior’s variational parameters θ During learning, the agent’s generative model is updated through its experiences by optimizing the variational free energy objective with respect to the model parameters Finally, during action selection, the agent selects a policy π that minimizes the expected free energy G, i.e., the expectation of the free energy under future observations, under each policy augmented with an inductive prior over policies H Formal Description of S-HAI The S-HAI agent comprises two coupled Partially Observable Markov Decision Processes (POMDPs), one per level, which interact through top-down and bottom-up message passing (Van de Maele et al.

Title and authors: Ines: So we're looking at this paper today, "Schema-based active inference supports rapid generalization of experience and frontal cortical coding of abstract structure." It tackles how abstract structures help us learn new things quickly.

Marcus: Right. This work introduces a computational framework called S-HAI that mixes predictive processing and active inference with schema mechanisms to model this learning process.

Ines: Basically, they’re suggesting that schemas are these abstract relational structures that capture commonalities across experiences, which allows us to rapidly generalize knowledge and rebind new experiences to existing structures.

Marcus: That means the core idea is that we don't have to relearn everything from scratch every time; we just map the sensory details of a new experience onto an existing structure.

Yuki: From a population genetics standpoint, I'm interested in what this says about how our species learns complex spatial or sequential tasks across different environments.

Ines: Exactly. The paper uses this S-HAI framework to show how it can reproduce key behavioral signatures of rapid schema-based generalization in spatial navigation tasks.

Marcus: They ran simulations on things like the ABCD task, which requires reaching four goal locations in a specific order, and they found the agent could generalize that structure quickly to novel environments where only the locations change.

Yuki: That’s interesting because it suggests that this ability to reuse a structure isn't just about memorizing paths; it’s about understanding the underlying relational rules of how those locations connect.

Ines: And they showed that this inference in S-HAI allows for fast generalization, and they also looked at more complex tasks, like the ABCB task where two goals can occupy the same location.

Marcus: They found that both these simulated agents used fewer steps per trial and accumulated more reward compared to other models, which suggests this schema-based approach is computationally efficient.

Yuki: It’s interesting that they linked this to how rodents in the medial frontal cortex learn that abstract sequential structure for tasks like the ABCD task, suggesting a potential neural correlate here.

Ines: They found specific neural codes there, like goal-progress cells and place-like codes at the lower level, which seem to map physical locations in a way similar to hippocampal place cells.

Marcus: And the model simulates activity tracking progress toward goals based on how much information you have about the inductive cost of that progress.

Yuki: That connection between abstract goal beliefs and physical location encoding is what makes this particularly compelling for understanding higher-order cognition in animals.

Ines: The mechanism behind this grounding is important, because it lets the agent rapidly map its abstract schema to each novel spatial configuration instead of relearning from scratch.

Marcus: They even have a component called S-HAI MoGL that maintains a mixture of grounding likelihoods over time using a truncated Dirichlet process, which lets the agent decide when to reuse an existing mapping or create a new one.

Title and authors: Yuki: So it's not just one fixed schema; it’s flexible enough to learn when it needs to adapt its underlying understanding of how those abstract structures connect.

Ines: That ability to decide between assimilation—fitting new info into old schemas—and accommodation—creating new ones when things don't fit—is what the paper calls the dual process of learning and development, referencing Piaget in one thousand nine hundred fifty-two <ref:2601.18946#pg1>.

Marcus: It shows that this framework isn't just about fitting things in; it’s about dynamically managing that trade-off between reusing old knowledge and building new structures.

Yuki: That flexibility seems crucial for any organism navigating a constantly changing physical world, which is exactly what we see in natural behavior.

Ines: The paper also mentions how this works computationally, describing S-HAI as two coupled Partially Observable Markov Decision Processes, one per level, interacting through top-down and bottom-up message passing.

Marcus: That hierarchical structure separates the learning and navigation into distinct levels: the higher level learns the abstract task structure in an abstract space.

Yuki: And the lower level handles the actual spatial information in physical space, which then connects to that high-level goal understanding through a grounding likelihood.

Ines: The implication is that this hierarchical setup provides a clear computational blueprint for how we might model complex cognitive abilities like planning and navigation.

Marcus: For someone looking at genomics data, the cohort and batch effects are always huge concerns, but this paper tries to isolate the mechanism of schema generalization rather than just claiming a statistical association.

Yuki: I wonder if the findings from this work can be extended to other species or even more abstract conceptual spaces we study in population genetics.

Ines: The paper does point out some limitations though. Currently, the S-HAI agent only implements a single schema, whereas the broader SHAI framework allows for maintaining a mixture over multiple schemas.

Marcus: They also focused mainly on schema learning in the frontal cortex, and future work will need to extend this model to other brain regions like the hippocampus or entorhinal cortex.

Yuki: I hope they do that because understanding how these abstract structures are encoded across different brain areas is key to understanding how these cognitive abilities develop over time.

Ines: And finally, they generated some novel empirical predictions about what behavior and neural representations we might actually observe in animals performing the ABCD task.

Marcus: So to wrap up this look at "Schema-based active inference supports rapid generalization of experience and frontal cortical coding of abstract structure," it shows a computational model that successfully reproduces fast generalization in spatial tasks, goal ambiguity resolution, and specific cortical neural codes.

Yuki: It provides a clear way to see how abstract relational structures are learned and then used flexibly across different contexts.

Ines: And the paper suggests that this framework has real implications for building more powerful models of cognition beyond just trial-and-error learning.

The paper's summary: Ines: So, to recap, this paper is about how using these abstract relational structures—schemas—helps an agent learn new things fast and what that looks like in the brain's frontal cortex.

Marcus: Right. They’re taking active inference, which is a way of modeling how brains make decisions based on minimizing some kind of energy functional, and hooking it up with schemas to see if we can explain rapid generalization.

Yuki: From a population genetics view, I want to know what the authors are actually saying about the biological plausibility of this whole schema thing, not just the simulation results.

Ines: They argue that these schemas aren't just arbitrary rules; they’re what let us take sensory input—like seeing a new layout—and instantly map it onto an existing structure we already have.

Marcus: The big implication for us in the cohort analysis is that this suggests our brains might be doing this constant re-binding, not just memorizing raw data points.

Yuki: So if schemas are these templates, what does that imply about how variation accumulates across a species? Does it mean we’re just shuffling existing templates around?

Ines: The paper shows that the agent has two main processes: assimilation, which is fitting new things into old schemas quickly, and accommodation, which is when you actually have to build a new schema when the new information just doesn't fit.

Marcus: That distinction between assimilation and accommodation is where I get interested statistically; it explains why some learning takes one shot versus needing many more trials.

Yuki: And the paper suggests this dynamic process of fitting and building might be what allows for flexible behavior in environments that change constantly, like natural habitats.

Ines: Exactly. And they show that this whole system, S-HAI, can reproduce neural codes in rodents—like specific cells in the medial prefrontal cortex—that track goal progress based on these abstract beliefs.

Marcus: Linking those high-level goals to low-level spatial locations through a grounding likelihood is a neat way to bridge the gap between what’s happening conceptually and what’s happening physically in the brain.

Yuki: It sounds like they’re proposing a mechanism for how abstract concepts, like "going to that specific spot," become physically represented in the cortex.

Ines: And they show how this mapping can be reused, or remapped, depending on whether you reuse an old schema or need to create a new one when things get really weird.

Marcus: It’s interesting that they also mentioned the limitations; for instance, they only tested one schema in their main model, so we don't know how robust this is when the agent has to manage multiple different types of knowledge simultaneously.

Yuki: So what we’re seeing here is a strong computational blueprint for how we might model cognitive abilities that require both abstract planning and physical movement across varying conditions.

The paper's improvements: Tom: So, we’re looking at what the authors suggest to fix or improve this S-HAI model today on how schemas work in active inference for generalization and neural coding.

Ines: Basically, they point out that while their current agent only handles one schema at a time, the full SHAI framework lets you maintain a mixture over multiple schemas.

Marcus: That means instead of just having one fixed rule set for learning, the system can dynamically decide which schema to use when it encounters a new problem.

Yuki: That flexibility in managing different knowledge structures across tasks sounds like something that might be more representative of how complex animals actually navigate their world over time.

Ines: Right. And they also suggest moving the focus beyond just the frontal cortex to include other brain regions, like the hippocampus and entorhinal cortex, for a more complete picture.

Marcus: That makes sense from a data science side; if we only look at one part of the brain for cohort analysis, we might miss where the real generalization mechanism is happening.

Yuki: And if you can model how those abstract structures map onto physical space across different brain areas, that gives us a much better framework for understanding evolutionary changes in navigation skills.

Ines: The authors also suggest extending this to handle abstract spaces, meaning testing if this schema-based inference works when the goals aren't just physical locations but conceptual ones.

Marcus: That would be a big leap; moving from "go to location A" to "achieve goal B" in a way that respects the underlying relational structure of the task itself.

Yuki: Because if it can handle abstract spaces, then we might start seeing these schema-based mechanisms as universal tools for any kind of complex problem-solving, not just spatial ones.

Ines: So the paper’s suggestion is to make the model more scalable by increasing its capacity to hold and switch between different relational knowledge structures.

Marcus: It takes the theoretical framework from being a single mechanism that fits things into one structure, to a system that can actively manage its own knowledge base.

Yuki: And if we can see how these schema mixtures evolve across generations in the genetic data, it could tell us how cognitive flexibility is inherited.

Ines: So the paper’s direction is toward building a model that isn't just good at one task, but can handle a whole library of learned ways to organize experience.

Marcus: It shifts the focus from proving one specific result to building a general computational architecture for how cognitive flexibility might arise in living organisms.

Conclusion: Tom: So we’re wrapping up this look at "Schema-based active inference supports rapid generalization of experience and frontal cortical coding of abstract structure." It really boils down to how these abstract structures allow for fast learning and what that looks like in the brain's frontal cortex.

Ines: Exactly. The main point is that schemas are not just labels; they are functional templates that let an agent map new sensory experiences onto old knowledge quickly, which we see reflected in simulation results on tasks like ABCD and ABCB.

Marcus: And from a genomics standpoint, the cohort size matters here; you have to look at how these effects scale up across different populations to really believe this isn't just a fluke in one specific group.

Yuki: I wonder if the evidence presented supports broader conclusions about cognitive flexibility in species that have been around for millions of years.

Ines: The paper shows the mechanism is grounded through a grounding likelihood, which allows the agent to decide when to reuse an old mapping or build a new one, which is essentially modeling assimilation and accommodation.

Marcus: That decision-making process over time, using that mixture of likelihoods like S-HAI MoGL, is what makes it more realistic than just having one static rule set.

Yuki: It’s fascinating to think about how that continuous updating of the knowledge structure might be encoded in the genetic variation we study.

Ines: And they link this to specific neural activity, like goal-progress cells in the rodent medial prefrontal cortex, showing exactly where these abstract goals are being represented physically.

Marcus: That connection between the high-level goal and low-level location encoding through that grounding likelihood is a strong piece of evidence for how cognition works in real neurons.

Yuki: If we can map those physical codes back to the abstract schemas, then we get a way to test if these learned structures are truly universal across different species.

Ines: So what this means for us right now is that we have a computational framework that provides clear predictions about what neural activity should look like when an agent is actually generalizing knowledge.

Marcus: It gives us a new language to talk about cognitive processes beyond just trial-and-error learning, focusing on the underlying relational structure.

Yuki: We’ll keep watching how these structural concepts play out in our genetic samples and see if they support this dynamic updating idea over evolutionary time.

More episodes

← Home