Renormalising Generative Models for Active Inference: Foundations, Derivations, and Verification

This paper provides a self-contained, derivation-oriented account of Renormalising Generative Models (RGMs) for active inference, together with an open, verified implementation. RGMs address the scaling problem of discrete active-inference models by composing discrete generative models across spatial and temporal scales, coarse-graining lower-level states and paths into higher-level causes for objects, events, and action.

The paper reconstructs the hierarchical generative model, explaining how the hierarchy is built, how beliefs and actions are updated within it, and how information is passed between levels. It defines a hidden-state factor f with categorical values, and a path value uf = h that selects a transition slice. Each level keeps the same interpretation of hidden states, paths, likelihoods, transitions, and empirical-prior mappings. Lower-level episodes are summarised upward; higher-level beliefs return downward as empirical priors.

The local updates use standard variational-message-passing and Dirichlet–categorical conjugacy for discrete graphical models. For a Dirichlet-distributed categorical tensor X with posterior counts x̃, the paper defines φ(x̃)iλ:= E[log Xiλ] = ψ(x̃iλ)−ψ(Σi′ x̃i′λ), covering likelihoods A, transitions B, hierarchical state mappings D, and hierarchical path mappings E.

For static hierarchical coupling, a two-level RGM replaces the ordinary prior over a lower state by a higher-level cause. The two state updates are q(s(1)) = σ(m↓D + m↑A) and q(s(2)) = σ(log π + m↑D), with m↑A = O ⊙ φ(ã), m↓D = φ(d̃) ⊙ ŝ(2), and m↑D = ŝ(1) ⊙ φ(d̃). The vertical mapping learns by the expected-count rule d̃ = d + ŝ(1) ⊗ ŝ(2).

For temporal dynamics, a path value uτ = h selects a transition slice p(sτ+1 = i sτ = j, uτ = h, B) = Bijh. The posterior state combines likelihood, forward, and backward transition messages: q(sτ) = σ(m↑A,τ + m→B,τ + m←B,τ), where m↑A,τ = Oτ ⊙ φ(ã), m→B,τ = φ(b̃) ⊙ ŝτ−1 ⊙ ûτ−1, and m←B,τ = φ(b̃⊤) ⊙ ŝτ+1 ⊙ ûτ. Transition learning adds path-indexed state-change counts b̃ = b + Στ=1T ŝτ ⊗ ŝτ−1 ⊗ ûτ−1. The mapping E is the path analogue of D: p(u0 = h sparent = j, E) = Ehj.

The paper distinguishes between fixed-point equations and the reference algorithm. The fixed-point account uses expected-log messages, while the reference implementation uses a filtering form. For state factor f, it first predicts the current state from the previous state and path beliefs: s̄fτ,i = Σj,h µ(b̃f)ijh ŝfτ−1,j ûfτ−1,h. The current observation then corrects this prior. The main contrast is that the fixed-point update averages log probabilities, while the reference filter takes logs of posterior-predictive probabilities.

Planning evaluates candidate future paths using expected free energy. A policy row Vk specifies one path value for each controllable factor. The score is G(k) ≈ Σg DKL[ôg,kτ+1 pgpref] + Σg Σspa(g) Qk(spa(g)) H g amb(spa(g)) − N(k), where the first term is risk, the second is ambiguity, and N(k) collects epistemic value. For a single controllable path, πG(h) = normh(πhu e−αGh). With several controllable factors, the posterior over joint policies is q(k) = normk(π0(k)e−αG(k)).

Retrospective path inference asks which path best explains an inferred transition. The reference implementation uses a posterior-predictive transition likelihood: LL(h) = log(Σi,j ŝτ,i µ(b̃)ijh ŝτ−1,j), ûτ−1 = σ(LL + mprior).

Recursive inference across levels is inverted recursively: a parent state provides empirical priors for a child segment, and the child returns posterior summaries as evidence for the parent. The reference implementation forms ordinary parent-induced priors using posterior means: D parent f(i) = Σj µ(d̃f)ij ŝτ,j and E parent f(h) = Σj µ(ẽf)hj ŝτ,j. These are multiplied into local child priors: D eff f(i) ∝ D local f(i) D parent f(i) and E eff f(h) ∝ E local f(h) E parent f(h).

Parameter learning uses conjugate expected-count forms. The RGM-specific sufficient statistics are ∆ag = Oτg ⊗f∈pa(g) ŝfτ, ∆bf = ŝfτ ⊗ ŝfτ−1 ⊗ ûfτ−1, ∆df = ŝf0 ⊗p∈pa(f) ŝp, and ∆ef = ûf0 ⊗p∈pa(f) ŝp. Active learning treats some candidate updates as decisions using a binary update gate r ∈ 0, 1, scored as q(r) = σ(−β G(a(r))). The score G(a(r)) = −Ir(O; S) − Σo Pr(o) log ppref(o), where the first term favors informative likelihoods and the second favors candidates whose induced outcomes agree with preferences. The practical rule is ãτ+1 = ãτ + q(r = 1)∆aτ, with an optional memory scale η.

Fast structure learning addresses how a hierarchy can be built from a high-dimensional sequence. The constructive RG step learns a level-n model M(n) and produces coarser data for the next level: Rn: Y(n) ↦ (M(n), Y(n+1)). The steps are: grouping variables by dependence using a mutual-information matrix I and a spectral relaxation; compressing local patterns into hidden states where recurring local patterns become state values; learning local likelihoods and transitions by accumulating counts; and passing summaries upward, where each retained group sends a state belief and a path belief to the next level. The vertical D and E links arise from reading higher-level likelihoods downward as empirical priors.

The paper includes an implementation and verification section. The translation reproduces three demonstrations from the SPM library: Drone VI (active vision and planning in a dynamic three-dimensional scene), Atari III (a hierarchical model of a simplified Pong-like game from pixel-level observations), and a third demonstration. Oracle testing treated the MATLAB implementation as the executable oracle, and the reported verification run passed all tests. The paper notes that the reference routines are a tractable online inversion and planning procedure, not a literal implementation of every fixed-point equation. Several differences recur: the fixed points use expected logs such as φ(ã), whereas the routines take logs of posterior means such as log E[A]; interior-time fixed points combine likelihood, forward, and backward messages, whereas the routines predict forward and correct on the current observation; fast structure learning is a constructive approximation; and the constructor passes hard state and path summaries upward.

The paper concludes that RGMs scale discrete active inference by turning lower-level episodes into higher-level causes and returning those causes as empirical priors. The contributions are: a reconstruction of the hierarchical generative model in which every coupling is derived as a local variational message; an explicit separation of these fixed-point equations from the reference algorithm; and an oracle-verified Python translation of the three capstone demonstrations.

Improvements to AI Systems Based on This Paper:

1. Hierarchical Temporal Abstraction for Long-Horizon Tasks

• Implement multi-scale generative models where lower-level state transitions (e.g., pixel-level movements) are summarized into higher-level event causes (e.g., object approaching).

• The improved AI can plan over hours/days by reasoning about abstract events rather than raw sensory steps, reducing computational cost and enabling sparse-reward tasks.

2. Recursive Belief Propagation for Partial Observability

• Use the RGM’s bidirectional message passing (upward summaries, downward empirical priors) to maintain coherent beliefs across levels.

• The AI can resolve ambiguity at low levels using high-level context (e.g., inferring a hidden object’s trajectory from scene-level priors) and refine high-level hypotheses with low-level evidence.

3. Efficient Policy Selection via Expected Free Energy Decomposition

• Adopt the planning score that separates risk (KL divergence from preferences), ambiguity (expected entropy), and epistemic value.

• The AI can actively choose actions that reduce uncertainty about ambiguous states while balancing goal-directed behavior, improving exploration in novel environments.

4. Fast Structure Learning for Scalable Perception

• Use the constructive RG step (mutual-information grouping + spectral relaxation) to automatically build hierarchical representations from raw high-dimensional data.

• The AI can discover reusable spatial/temporal patterns (e.g., recurring object interactions) without manual feature engineering, enabling rapid adaptation to new domains.

5. Online Parameter Learning with Conjugate Counts

• Apply the expected-count update rules (e.g., Δb = ŝ⊗ŝ⊗û) for Dirichlet-categorical parameters.

• The AI can continuously learn transition dynamics and likelihoods from streaming data, with a memory scale η to forget outdated statistics—critical for non-stationary environments.

6. Active Learning via Update Gating

• Use the binary gate q(r) = σ(−βG(a(r))) to decide whether to incorporate new observations into model parameters.

• The AI can selectively ignore redundant or uninformative data, reducing catastrophic forgetting and improving sample efficiency in lifelong learning.

7. Oracle-Verified Implementation for Reproducible AI

• Adopt the open, verified Python translation (tested against MATLAB oracle) as a baseline for hierarchical active inference.

• Developers can deploy a drop-in module for robotics, video game AI, or autonomous agents that require hierarchical reasoning, with guaranteed correctness of inference and learning steps.

8. Cross-Level Policy Generalization

• Leverage the path mapping E (higher-level causes → lower-level action sequences) to reuse learned high-level policies across different low-level contexts.

• The AI can transfer skills (e.g., navigate to goal) learned in one environment to another with different low-level dynamics, by re-parameterizing the lower-level transition slices.

9. Robustness to Model Mismatch

• Use the fixed-point vs. filtering distinction to choose between exact Bayesian inference (for offline analysis) and tractable online filtering (for real-time control).

• The AI can switch between modes based on computational budget, maintaining performance under latency constraints.

10. Interpretable Decision-Making

• Because the hierarchy maps to objects/events/actions, the AI’s internal beliefs (e.g., level-2 state = object A moving left) are human-readable.

• The AI can explain its reasoning in terms of abstract causes, improving trust and debuggability in human-AI collaboration.