A Variational Latent Equilibrium for Learning in Neuronal Circuits
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: We're looking at the paper: "A Variational Latent Equilibrium for Learning in Neuronal Circuits".
Marcus: Here's the gist of the paper. Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns<ref:2603.09600#pg2> This work proposes a general formalism to approximate backpropagation through time in a controlled, biologically plausible manner<ref:2603.09600#pg2>.
Ines: Theoretical Framework The approach builds on previous methods of local, timecontinuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action<ref:2603.09600#pg2>.
Marcus: The starting point is a prospective energy function of neuronal states from which real-time error dynamics are calculated for time-continuous networks<ref:2603.09600#pg2>.
Ines: In the general case, this provides a simple derivation of the adjoint method result for neuronal networks, which is the time-continuous equivalent to BPTT<ref:2603.09600#pg2>.
Title and authors: Ines: So we're looking at a paper called "A Variational Latent Equilibrium for Learning in Neuronal Circuits." It’s from Simon Brandt and his team, and it tackles the idea that backpropagation through time, or BPTT, is hard to implement biologically because it’s not local.
Marcus: Yeah, exactly. They propose this general formalism to approximate BPTT in a way that respects how real brains actually work—locally in space and time. It builds on older ideas about credit assignment using energy conservation and extremal action principles.
Yuki: From a population genetics standpoint, what I find interesting is how they try to make this continuous dynamic approach work for discrete neuronal systems. They start with a prospective energy function of the neuron states to calculate real-time error dynamics in time-continuous networks.
Ines: Right, and they show that this general approach gives you the adjoint method result, which is the time-continuous equivalent of BPTT, but they make it much simpler and more transparent than usual derivations of the adjoint method.
Marcus: And then to get that biological plausibility, they look at how explicit approximations in their formalism lead to something called GLE dynamics. That's where things get a bit more specific about what the neurons are doing over time.
Yuki: I wonder how these continuous error dynamics translate back into the actual learning rules for synapses and neurons themselves, because that’s where we usually see the biological mechanism in action.
Ines: They do that by showing how to turn it into a fully local set of equations for neuron and synapse dynamics with just a few modifications. It moves the problem from being global to being very localized within the circuit.
Marcus: To make sure this works locally, they have some approximations in place. They approximate the discounted future operator using something called look-ahead, and the look-back operator with a low-pass filter to keep things temporally local in that error stream.
Yuki: So they are trying to manage that temporal dependency—the future and the past—by applying these specific filters so it doesn't just become a complicated global calculation.
Ines: And then, to fix the issues with causality and spatial locality, they propose learning backward weights. These weights are learned to compensate for some gain offset in those GLE equations that comes up when you use the look-ahead and look-back approximations.
Marcus: They minimize a specific quantity called E B i j, which is essentially minimizing that error propagation distortion by training these backward weights, and they show this leads to better convergence to the exact adjoint method solution while keeping things local.
Yuki: That’s a clever way to handle the approximation errors by introducing a learned correction term into the system dynamics. It suggests that learning these backward weights is actually beneficial for learning forward weights in more complex tasks, which they tested experimentally.
Title and authors: Ines: They did test three scenarios for learning B—learning B directly, using B equals W T, and using constant B equals minus W T—and they showed that just learning the backward weights helps when you are also trying to learn the forward weights for harder problems.
Marcus: It’s a nice result because it suggests that this method can solve temporal problems that purely spatial methods just can't handle, which is a big deal for modeling real neural systems.
Yuki: So the overall implication is that this Variational Latent Equilibrium approach provides a systematic way to approximate backpropagation through time in a manner that is local both in space and time.
Ines: Exactly. They show it's an approximation of the adjoint method and BPTT, but one that’s constrained to be local, which is what we need for biological plausibility.
Marcus: And the conclusion they draw is that learning these backward weights makes the learning process faster and better for complex signals overall.
Yuki: It seems like a framework that could be a real blueprint for building brain models and applying neuromorphic computing because it grounds the math in energy conservation.
Ines: Yeah, it’s a systematic way to handle those tricky spatiotemporal tasks that require different frequency ranges of information to learn.
Marcus: So, we’ve seen how they use variational methods to derive dynamics from an energy function and then use learned weights to correct the approximations in the GLE dynamics.
Yuki: It connects the abstract mathematics of extremal action directly to what happens at a circuit level in terms of error propagation.
Ines: That’s what this paper, "A Variational Latent Equilibrium for Learning in Neuronal Circuits," is all about—approximating BPTT with local dynamics by finding a variational latent equilibrium.
Marcus: So we’ve covered how they set up the energy function, how they derive the error dynamics through the Euler-Lagrange equations, and how they use learned backward weights to fix the approximations in GLE.
Yuki: It seems like their key contribution is showing that this path recovers the adjoint method results in a much more intuitive way than standard derivations.
Ines: And it opens up a path for designing models that are physically motivated by energy principles while still being capable of learning complex temporal patterns effectively.
Marcus: We need to remember that they have to make some explicit approximations, like using look-ahead and low-pass filters, but they show how to correct those errors systematically with learned weights.
Yuki: That correction mechanism is what really makes the leap from a theoretical approximation to something that can actually be used in a neural circuit simulation.
Ines: So the paper suggests we can have enhanced stability and more accurate learning for tasks that involve diverse frequency ranges of signals.
Marcus: It’s certainly a way to get better results without having to deal with the full computational cost of standard backpropagation through time.
Yuki: We'll keep an eye on how this framework is applied in larger systems, because translating these local dynamics to massive networks is always going to be the next big hurdle.
The paper's summary: Ines: So, what we’re looking at now is how they wrap up this whole Variational Latent Equilibrium concept for learning in neural circuits. Basically, they show that by finding the right energy state—the latent equilibrium—you can approximate backpropagation through time in a way that stays local both in space and time.
Marcus: That’s the big picture, yeah. It means you don't have to run these massive global calculations every single time you train a deep neural network or a spiking network. You get this local approximation of backpropagation through time without losing the core physics of how the system evolves over time.
Yuki: From my side, it’s interesting because they build this on principles like energy conservation and extremal action, which is pretty fundamental stuff for biology. It suggests that learning might be fundamentally tied to minimizing some kind of action or energy functional in a network.
Ines: Exactly. They start with a prospective energy function of the neuron states and derive the real-time error dynamics from there. They show this general derivation leads directly to what we call the adjoint method result, which is basically backpropagation through time for continuous networks.
Marcus: But they make it much simpler than standard derivations of the adjoint method by turning that into a set of local equations for neuron and synapse dynamics with just a few modifications. That’s where you get that biological plausibility we’re after.
Yuki: And then they show how to handle the temporal parts—the future and the past—by approximating them. They use things like a look-ahead operator for the future error and a low-pass filter for the past error to keep that stream temporally local.
Ines: That’s where it gets tricky, though. To fix those approximations, they propose learning backward weights that compensate for some gain offset in those dynamics. This is done by minimizing a specific quantity that ties the forward and backward dynamics together locally.
Marcus: So what this means for us on the ground is that you train these network weights to learn how to correct the error propagation distortion so the network converges faster to the actual adjoint method solution while still respecting spatial and temporal locality.
Yuki: And they did some testing, too, showing that learning these backward weights is actually helpful for learning forward weights in more complicated tasks. It can solve temporal problems that purely spatial methods just can't touch.
Ines: So the implication here is a systematic way to design models of the brain and neuromorphic applications by grounding them in energy principles while still being able to learn complex spatiotemporal patterns effectively.
Marcus: Yeah, it gives us a blueprint for how these systems might actually learn in real biological circuits instead of just treating them as black boxes where you can throw any learning algorithm at them.
Yuki: It’s a neat way to bridge the gap between abstract mathematics and actual neuronal dynamics, focusing on those diverse frequency ranges that require different learning mechanisms.
The paper's improvements: Tom: So we’re looking at how they suggest improving this Variational Latent Equilibrium approach for learning in neural circuits. They focus on making those approximations more robust and ensuring they respect causality better than before.
Ines: They suggest that the core improvement is explicitly training those backward weights, not just treating them as a fixed parameter. This allows the network to learn how to correct for the gain offset in those GLE equations directly from the data.
Marcus: That’s significant because it moves the system from just being an approximation to being actively corrected by learning. It means that instead of guessing how much correction is needed, you let the learning algorithm figure out that value through minimizing that error quantity.
Yuki: That ties back into what we were talking about—the sample size and cohort effects—if the learning mechanism itself can adapt to those environmental or network-specific offsets, it might be more reliable across different biological contexts.
Ines: Right. They showed that this learned correction helps the network converge faster to the exact adjoint method solution while maintaining that local spatial and temporal structure we need for biology.
Marcus: The numbers show that learning B directly, instead of using a fixed version like B equals W T, actually yields better results when you’re tackling more complex tasks. This suggests a statistical benefit from letting the weights evolve during training.
Yuki: So what this changes for the wider population genetics view is that this isn't just about fitting a static model to data; it’s about having an adaptive mechanism that learns how to handle noise and local dynamics across the network structure.
Ines: Exactly. And they proved this works even when you are learning forward weights for tasks that are purely temporal, which is something spatial models struggle with.
Marcus: The caveat here is that this method still relies on those initial approximations of the future and past operators—the look-ahead and the low-pass filter—so it’s not a perfect backpropagation through time implementation, just a much more practical one.
Yuki: So while it's not perfect, it gives us a powerful tool to model how complex biological systems manage temporal dependencies without needing impossibly large computational overhead.
Conclusion: Ines: So to wrap up, this paper on "A Variational Latent Equilibrium for Learning in Neuronal Circuits" shows us how to get a local approximation of backpropagation through time that’s grounded in energy conservation principles.
Marcus: It’s about using variational methods to derive the dynamics from an energy function and then using learned backward weights to correct the approximations in those dynamics, making learning faster for complex signals.
Yuki: From a population genetics standpoint, it suggests that the underlying mechanisms for temporal pattern recognition might be driven by these local energy minimization principles across different network scales.
Ines: And what it changes is how we model biological learning; instead of just applying global optimization techniques, we can use this variational approach to design systems that naturally respect spatial and temporal constraints.
Marcus: The main caveat remains that it’s an approximation because they have to use specific filters for the future and past operators, but the experimental validation suggests those corrections are actually quite effective.
Yuki: It’s still a model, obviously, but it provides a way to test hypotheses about how different frequency ranges of information might be processed locally in a neural system.
Ines: Yeah. It gives us a systematic tool for building brain models that aren't just fancy mathematical boxes but have some physical grounding in how energy flows through the network.
Marcus: So, this framework isn't just theoretical math; it’s something that could actually be used to build more stable and accurate AI systems that mimic biological learning processes better.
Yuki: It opens up a path for understanding temporal coding in organisms by linking those mathematical dynamics back to observable evolutionary strategies.
Simon Brandt, Paul Haider, Walter Senn, Federico Benitez, +Mihai A. Petrovici
Department of Physiology, University of Bern
q-bio.NC, cs.AI, cs.NE, cs.SY, eess.SY, physics.bio-ph
Submitted: 2026-03-10
Updated: 2026-11-13
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 69/100
The gist: Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns<ref:2603.09600#pg2> This work proposes a general formalism to approximate backpropagation through…
Key concepts
- Variational Latent Equilibrium (VLE)
- VLE is a method where finding the extremum of an energy function corresponds to neuronal dynamics obeying Euler-Lagrange equations. This approach simplifies error propagation links compared to standard methods, making it easier to derive how errors flow through the network.
- Adjoint Method (AM) / BPTT
- The adjoint method is a standard technique used for backpropagation through time (BPTT) in neural networks. VLE aims to provide a simpler, time-continuous equivalent of this method, allowing for credit assignment in a way that respects the continuous nature of neuronal signals.
- Temporal Locality Approximations
- To ensure the error stream is temporally local, the paper approximates future operators using a look-ahead mechanism and past operators using a low-pass filter. These specific approximations are necessary to transform the general theory into usable equations for real-time network dynamics.
Terminology
Summary
Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns<ref:2603. p. 0 benitez@unibe. ch
Abstract
Brains remain unrivaled in their ability to recognize and
generate complex spatiotemporal patterns. This is
prominently the case for backpropagation through time
(BPTT), the go-to algorithm for learning complex temporal dependencies. In this work we propose a general
formalism to approximate BPTT in a controlled, biologically plausible manner.Where the paper says it
Where the paper says it
Theoretical Framework
The approach builds on previous methods of local, timecontinuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action<ref:2603.09600#pg2>. The starting point is a prospective energy function of neuronal states from which real-time error dynamics are calculated for time-continuous networks<ref:2603. p. 0 Our approach builds on, unifies
and extends several previous approaches to local, timecontinuous, phase-free spatiotemporal credit assignment
based on principles of energy conservation and extremal
action. Our starting point is a prospective energy function
of neuronal states, from which we calculate real-time error
dynamics for time-continuous neuronal networks. Our starting point is a prospective energy function
of neuronal states, from which we calculate real-time error
dynamics for time-continuous neuronal networks. In the
general case, this provides a simple and straightforward
derivation of the adjoint method result for neuronal networks, the time-continuous equivalent to BPTT. In the
general case, this provides a simple and straightforward
derivation of the adjoint method result for neuronal networks, the time-continuous equivalent to BPTT. With a
few modifications, we can turn this into a fully local (in
space and time) set of equations for neuron and synapse
dynamics.Where the paper says it
Where the paper says it
Where the paper says it
Variational Latent Equilibrium (VLE)
The method is called variational latent equilibrium (VLE)<ref:2603.09600#pg2>. Finding the extremum of E is equivalent to having neuronal dynamics obey the corresponding Euler-Lagrange equations, which yield the error dynamics<ref:2603. p. 2 Because of these reasons, we call
this approach variational latent equilibrium (VLE). Finding the extremum of E is equivalent to having neuronal dynamics obey the corresponding Euler-Lagrange
equations, which yield the error dynamics (see Methods
for details)
ei = ˜ε
m
i with εi = φ
′
i
X
k
ˆWkiek
r
, (8)
which is exactly the result of the AM for this type of
system [1]. Following standard methodology, we require neuronal dynamics to keep the integrated energy stationary:
δuE = 0 (7)
This variational approach can be viewed as a form of equilibrium in the state of lowest integrated energy rather than
just the lowest instantaneous energy. Furthermore, as we
show in more detail below, the outcome of the method
links directly to (G)LE. Finding the extremum of E is equivalent to having neuronal dynamics obey the corresponding Euler-Lagrange
equations, which yield the error dynamics (see Methods
for details)
ei = ˜ε
m
i with εi = φ
′
i
X
k
ˆWkiek
r
, (8)
which is exactly the result of the AM for this type of
system [1]. Note that this is derived in a much simplified way compared to standard treatments, thanks to the
specific form taken by dynamical variables when modeling
prospective behavior.Where the paper says it
Where the paper says it
Where the paper says it
Approximations for Plausibility
To achieve temporal locality in the error stream, the discounted future operator is approximated by the look-ahead, and the look-back is approximated by a low-pass filter<ref:2603.09600#pg2>. These approximations yield GLE equations<ref:2603. p. 2 Setting aside this special case, to achieve temporal locality in the error stream we approximate the discounted
future operator by the look-ahead; additionally, we can
approximate the look-back by the low-pass filter to make
error dynamics even more neuron-like, and which has the
additional advantage of partially compensating for the approximation made to the forward signal. With these approximations, we get
ei ≃ ϵ˘
m
i with ϵi ≃ φ
′
i
X
k
Wkiek
r
(10)
and recover the GLE equations. VLE uses this kind of mechanism to also deal with the
causality (temporal) issue. By learning backward weights
that compensate the gain offset in GLE, the network can
better converge to the exact AM solution, while preserving
spatial and temporal locality. To achieve the above, we simply use local gradient descent
to minimize
E
B
ij =
Wjiˆϵ˘j
r
r −Bijˆϵ˘j
mm
2
(15)
⇒ B˙
ij ∼ −
∂EB
ij
∂Bij
=
Wjiˆϵ˘j
r
r −Bijˆϵ˘j
mm
ˆϵ˘j
mm , (16)
where both look-aheads and look-backs are causal and
thus local in time (see Methods for a detailed derivation). We used εj = ˆϵ˘j
r
r
to compensate for the difference of the
low-pass filter in VLE and the look-back operator in AM.Where the paper says it
Where the paper says it
Where the paper says it
Learning Dynamics
The learning of the backward weights involves a local gradient descent to minimize E B ij, yielding B˙ij ∼ −∂EBij/∂Bij=<ref:2603.09600#pg2>. This mechanism allows the network to better converge to the exact AM solution while preserving spatial and temporal locality<ref:2603. p. 8 VLE uses this kind of mechanism to also deal with the
causality (temporal) issue. By learning backward weights
that compensate the gain offset in GLE, the network can
better converge to the exact AM solution, while preserving
spatial and temporal locality.Where the paper says it
Experimental Validation
The experiments validate three scenarios for learning B, including learning B, using B = WT, and constant B = -W T<ref:2603.09600#pg2>. Learning B is shown to be beneficial for the learning of the forward weights in more complex tasks<ref:2603. p. 4 does not converge to a good solution. While we claim that learning B is beneficial for the learning of the forward weights, in this simple task, B = W T
converges to the teacher weights faster. For
spatiotemporal tasks that require the processing of signals
with a diverse frequency range, VLE enables enhanced
stability and more accurate learning. Our experiments
show that VLE can solve non-trivial temporal problems,
which cannot be solved by purely spatial methods.Where the paper says it
Where the paper says it
Comparison and Implications
VLE improves on GLE by correcting for the approximation of the discounted future error through backward weight training<ref:2603.09600#pg2>. The model is similar to NLA in taking inspiration from physics but diverges by allowing neurons in the backwards errors path to be prospective<ref:2603. p. 6 Thus, for a fully biologically
plausible implementation of BPTT, the backward learning rule of VLE becomes a necessity. Our model is similar to NLA in taking inspiration from
physics but diverges from it in important ways. This backward
learning can be combined with methods developed to solve
the problem of weight transport, most notably the PAL
framework. All these ingredients can be assembled into a
cortical microcircuit that implements biologically plausible learning, and constitutes a blueprint for brain models
and for neuromorphic applications.Where the paper says it
Where the paper says it
Conclusions
In this work, VLE is presented as a systematic approximation of the AM and BPTT that is local both in space and time<ref:2603.09600#pg2>. The results show that learning the backwards weights leads to faster and better learning for complex signals<ref:2603. p. 6 Our experiments
show that VLE can solve non-trivial temporal problems,
which cannot be solved by purely spatial methods. For
tasks that require the processing of complex signals, learning the backwards weights, and thus providing a neuron
specific learning rate that is modulated through the gain
correction, leads to faster and better learning. Our work improves on previous results by improving temporal error transmission in the backward pathway. For
spatiotemporal tasks that require the processing of signals
with a diverse frequency range, VLE enables enhanced
stability and more accurate learning.Where the paper says it
Where the paper says it
Methods
The network dynamics are solved using forward Euler integration to define the membrane potential update rule<ref:2603.09600#pg2>. The update rule for B then becomes Bij (t + dt) = Bij (t) + ηB(Wjiˆϵ˘j r r)(t −Bijˆϵ˘j mm (t))ˆϵ˘j mm (t)<ref:2603. p. 9 ˆϵ˘i
r
r= ϵi − (τ
r
i
)
2
ϵ¨i
, requires second order temporal
derivatives. The update rule for B then becomes
Bij (t + dt) = Bij (t) + ηB(Wjiˆϵ˘j
r
r
(t) − Bijˆϵ˘j
mm (t))ˆϵ˘j
mm (t),
(43)
with ηB =
dt
τ
B
again being a learning rate and
ˆϵ˘j
r
r
(t) = ϵj (t) − (τ
r
j
)
2
ϵj (t + dt) − 2ϵj (t) + ϵj (t − dt)
dt
2
,
(44)
using the instantaneous input integrated in Eqn. ˆϵ˘i
r
r= ϵi − (τ
r
i
)
2
ϵ¨i
, requires second order temporal
derivatives. The update rule for B then becomes
Bij (t + dt) = Bij (t) + ηB(Wjiˆϵ˘j
r
r
(t) − Bijˆϵ˘j
mm (t))ˆϵ˘j
mm (t),
(43)
with ηB =
dt
τ
B
again being a learning rate and
ˆϵ˘j
r
r
(t) = ϵj (t) − (τ
r
j
)
2
ϵj (t + dt) − 2ϵj (t) + ϵj (t − dt)
dt
2
,
(44)
using the instantaneous input integrated in Eqn.Where the paper says it
Where the paper says it
Temporal XOR Task
In the Temporal XOR task, the network learns a continuous and temporal version of the XOR task by learning r in 0(t − ∆0) − r in 1(t − ∆1)<ref:2603.09600#pg2>. Learning B leads to a significantly faster convergence of the training loss compared to using B = WT<ref:2603. p. 5 Before training, the output of the student is very different
from the target and converges throughout the training as
shown in Fig. Learning B leads to a significantly
faster convergence of the training loss compared to using
B = WT
. Learning B leads to a significantly
faster convergence of the training loss compared to using
B = WT
. Since the sharp edges caused by the absolute
value in the target lead to high frequency components
in the error, the network used to learn this task benefits
from the synapse specific learning of the backward weights
that accounts for offsets in the gain that arise when using
B = WT
.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved AI systems can perform spatiotemporal deep learning in a controlled, biologically plausible manner by approximating backpropagation through time using Variational Latent Equilibrium (VLE). This framework builds on previous approaches to local, timecontinuous, phase-free credit assignment based on principles of energy conservation and extremal action.
-
The improved system can derive neuronal dynamics for systems of neurons equipped with temporal processing capabilities from a first-principles approach in terms of variational calculus. This derivation recovers the results of the adjoint method while being
much simpler and conceptually more transparent than standard derivations.
-
The system can be designed to learn forward and backward weights to achieve a local spatiotemporal approximation of BPTT. By learning backward weights that compensate for the gain offset in GLE, the network can better converge to the exact adjoint method solution while respecting spatial and temporal locality.
Abstract
Brains remain unrivaled in their ability to recognize and generate complex spatiotemporal patterns. While AI is able to reproduce some of these capabilities, deep learning algorithms remain largely at odds with our current understanding of brain circuitry and dynamics. This is prominently the case for backpropagation through time (BPTT), the go-to algorithm for learning complex temporal dependencies. In this work we propose a general formalism to approximate BPTT in a controlled, biologically plausible manner. Our approach builds on, unifies and extends several previous approaches to local, time-continuous, phase-free spatiotemporal credit assignment based on principles of energy conservation and extremal action. Our starting point is a prospective energy function of neuronal states, from which we calculate real-time error dynamics for time-continuous neuronal networks. With this, as a first contribution we provide a simple and straightforward derivation of the adjoint method dynamical equations, the time-continuous equivalent to BPTT. Importantly, additional approximations are necessary to avoid unphysical temporal non-localities. One such approximation was recently proposed in Generalized Latent Equilibrium (GLE), but enforcing temporal locality of the backpropagating dynamics induces an error in the gain that becomes problematic when processing complex signals. Thus, our second contribution is to design a learning rule for the backward weights to find a fully local (in space and time) set of equations for neuron and synapse dynamics that corrects for this gain difference. With this inclusion of backward weight learning, our theory provides a rigorous framework for spatiotemporal deep learning in the brain, while simultaneously advancing a blueprint for physical circuits capable of carrying out these computations.
Sources
- Spike-based causal inference for weight alignment
- Spike-based alignment learning solves the weight transport problem
Related papers
- Self-Supervised Learning of Local Predictive Directions as a Neural Computation
- Synchronized Hearts, Desynchronized Brains: A Nonlinear Dynamics Perspective for Function and Failure
- Message interaction dynamics covary with brain volume across mammalian connectomes
- Planning as Dynamics Relaxation: Hippocampal Recurrent Network Realizes Optimal Goal-Directed Navigation
- The Platonic brain bridge hypothesis: brain networks as architectural priors for multimodal large language models
- BrainPFN: Amortized Brain-Behaviour Prediction via Prior-Data Fitted Networks