Learning Heat-based Equations in Self-similar variables

arXiv:2602.00872 · cs.LG, math-ph, math.MP · Submitted 2026-01-31 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Learning Heat-based Equations in Self-similar variables".

Tom: The gist: Self-similar variables provide a mathematically motivated inductive bias for learning the long-time dynamics of heat-based equations, consistently delivering substantially more accurate and stable extrapolation beyond the training window.

Jane: First, who's behind it and why it matters.

Title and authors: Tom: So, we’re looking at a paper called "Learning Heat-based Equations in Self-similar variables". It sounds super technical, but basically, they're trying to teach AI models how to learn the long-time behavior of heat equations by switching the way they look at time and space.

Jane: That’s right. They’re talking about using self-similar variables as a way to make sure the AI learns what happens far into the future, not just what happens right now when you train it on a computer.

Lu: It's an interesting approach because classical math already shows that these equations have this predictable, self-similar long-time behavior, which means they look the same in a certain way over time.

Meng: So the main idea here is to use those known mathematical structures as a guide for training neural operators so the models don't just get stuck on short-term patterns.

Lalam: It gives the AI a better map of what stability looks like, instead of just memorizing a few snapshots in time.

Tom: Exactly. The paper is exploring this by setting up an SSV training framework that works with standard neural-operator training methods for things like the two-dimensional Navier–Stokes equations and the one-dimensional viscous Burgers equation.

The paper's summary: Jane: What they’re summarizing is how they take those equations, like Navier–Stokes or Burgers, and transform them into these self-similar variables using a change of variables. This makes the heat kernel look like a stationary profile, which simplifies things a lot.

Tom: And that transformation is key because it means that solutions that settle down to this self-similar profile become localized in those new coordinates with rapidly decaying tails. That’s what they call Proposition one (Long-time Behavior) > <ref:2602.00872#pg2>

Lu: So the point is, if you learn the solution in these self-similar variables, a fixed sampling window in those new coordinates can capture the main part of the solution at large times, and you can ignore those exponentially small tails >

Meng: That’s a big deal for practical AI because it means we don't have to sample an infinitely long time horizon to get an accurate picture of what the system is actually doing over the long run.

Lalam: It suggests that learning in physical coordinates, where time keeps running away, is inefficient compared to learning in these specialized self-similar coordinates >

Tom: Right. So they set out to develop a precise SSV formulation that fits into neural-operator training and then prove how much better it is than just training the model directly in physical coordinates on both 2D Navier–Stokes and 1D Burgers equations <ref:2602.00872#pg3>.

The paper's improvements: Jane: The main improvement they highlight is that SSV-trained networks consistently deliver substantially more accurate and stable extrapolation beyond the training window, even when comparing them to simpler network architectures like standard multi-layer perceptrons or factorized fully connected networks.

Tom: That’s the core finding. They show that this self-similar representation provides a mathematically motivated inductive bias that helps the learning process generalize much better than just using physical coordinates >

Lu: And it's not just about accuracy; they found that SSV-trained models more reliably reproduce qualitative features intrinsic to the underlying dynamics, like long-time physical patterns and those coarsening or merging behaviors when they occur >

Meng: So what this means for engineering is that if you want a model to predict long-term physical phenomena, using these self-similar coordinates will help it capture those important visual trends that might otherwise be missed >

Lalam: It shows the gain comes from working in a coordinate system intrinsically adapted to the parabolic scaling and self-similar dynamics of the equations, rather than just tweaking the network weights in physical space >

Tom: Precisely. They’re showing that this advantage holds across different architectures, meaning it’s not dependent on how you build your neural network, which is really reassuring for practitioners.

Conclusion: Jane: To wrap up, the paper on "Learning Heat-based Equations in Self-similar variables" shows that using self-similar variables gives us a robust advantage when learning heat-based equations because it aligns the representation with the intrinsic scaling structure of the dynamics >

Tom: It’s about moving away from physical coordinates when we want to understand long-time evolution, and they proved this works for both 2D Navier–Stokes and 1D Burgers equations using two different network structures > <ref:2602.00872#pg3>

Lu: The results consistently point to a clear practical gain in terms of stability and accuracy when extrapolating beyond the training window, especially for capturing those complex physical patterns >

Meng: For someone building a model, this strategy offers a practical route to improved rollout stability and better long-time prediction for heat-based systems because it addresses the mismatch between training data and the physics of long times >

Lalam: It gives us a way to improve how AI learns these kinds of complex physical processes by using mathematical structure instead of just relying on brute force time steps >

Tom: So, that’s what this paper is all about. We’re seeing a clear path forward for making neural operators better at predicting the long-term behavior of heat equations using self-similar variables, and we're ready for whatever comes next in this field.

University of Wisconsin–Madison · University of Arizona

cs.LG, math-ph, math.MP

Submitted: 2026-01-31

Updated: 2026-02-03

Journal ref: ICLR 2026 Workshop on AI and Partial Differential Equations

Code: https://github.com/Kagami-319/SSV-Learning

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist: Self-similar variables provide a mathematically motivated inductive bias for learning the long-time dynamics of heat-based equations, consistently delivering substantially more accurate and

Key concepts

Self-Similar Variables (SSV): These are specific mathematical changes of variables applied to a differential equation, like Navier–Stokes, that align with its known scaling properties. The goal is to transform the complex dynamics into a simpler form where the long-time behavior becomes more apparent and easier for a neural network to learn.
Neural-Operator Training: This is a standard way of training neural networks for solving differential equations. Instead of learning every single point, the network learns an operator that maps functions from one space to another, making it efficient for simulating continuous physical systems like heat flow.
Extrapolation Performance: This refers to how well a trained model performs when asked to predict outcomes far beyond the data it was trained on. The paper shows that SSV-trained models excel here, providing more accurate and stable results than models trained in standard physical coordinates, which is crucial for long-time prediction.
Coordinate Transformation: This involves changing the way we look at a problem by redefining the variables used to describe it. For example, in Navier–Stokes, using variables like $\xi = x/\sqrt{t} + 1$ instead of just $(x, t)$ helps capture the self-similar scaling inherent in how heat diffuses over time.

Summary

The gist: Self-similar variables provide a mathematically motivated inductive bias for learning the long-time dynamics of heat-based equations, consistently delivering substantially more accurate and stable extrapolation beyond the training window.

SSV Training Framework and Methodology

The study develops an SSV training framework compatible with standard neural-operator training. This approach exploits the intrinsic scaling structure of governing dynamics by working in variables that better reflect long-time behavior. The framework is instantiated on two systems: the two-dimensional incompressible Navier–Stokes equations and the one-dimensional viscous Burgers equation. The goal is to develop a precise self-similar-variable (SSV) formulation for heat-type equations that is compatible with neural-operator training.

Network Architectures Compared

The comparison across architectures demonstrates that the advantage of SSV training is model-independent. Two network architectures are used for comparison:

Standard Multi-Layer Perceptron (MLP): This model takes a concatenated coordinate z ∈ R m (either (x, t) or (ξ, τ)) and applies a single fully connected network fθ: R m → R.
Factorized Fully Connected Network (FCN): Inspired by branch–trunk constructions, the FCN output is defined by Nθ(s, z) = bθ + Σ K=1 to K Bθ,k(s) Tθ,k(z).

Comparison of Extrapolation Performance

Across both systems and both architectures, SSV-trained networks consistently deliver substantially more accurate and stable extrapolation beyond the training window. Furthermore, SSV-trained models more reliably reproduce qualitative features intrinsic to the underlying dynamics—such as long-time physical patterns and, when present, coarsening/merging behaviors—whereas these effects are often degraded or missed by models trained directly in physical coordinates. This indicates the main gain comes from working in a coordinate system intrinsically adapted to the parabolic scaling and self-similar dynamics of the underlying equations.

Application to Navier-Stokes and Burgers Equations

For the 2D incompressible Navier–Stokes equation, self-similar variables are defined by the change of variables ξ = x/√t + 1, τ = log(t + 1), which transforms the problem into a new equation. For the one-dimensional viscous Burgers equation, the corresponding self-similar unknown ω is introduced such that u(x, t) = (t + 1)−1/2ω(ξ, τ), which solves the SSV Burgers equation. Proposition 2 shows that for integrable initial data with nonzero mass, the long-time asymptotics of viscous Burgers are described by GM0 0 we define Dt,C:= A x ∈ R 2: x ≤ C√t + 1, which is the expanding disk obtained from DC by the map x = √t + 1 ξ>. The comparison protocol involves minimizing time-averaged windowed mean-squared errors Lssv(C) and Lphy(C).

Conclusion and Broader Implications

The results consistently indicate a robust advantage of SSV training. This improvement is driven by an appropriate representation that aligns with the self-similar structure of heat-based evolution. This strategy extends to other equations with self-similar asymptotics such as the porous medium equation. Looking ahead, directions include extending the framework to broader classes of systems with known scaling laws and developing adaptive windowing strategies. This provides a practical route to improved rollout stability and long-time prediction for heat-based systems. The same strategy extends to other equations with self-similar asymptotics such as the porous medium equation ∂tu = ∆u m, m ∈ N, which admits self-similar spreading solutions (see [20]) <ref:2602.008733,the same strategy extends to other equations with self-similar asymptotics such as the porous medium equation ∂tu = ∆u m, m ∈ N, which admits self-similar spreading solutions (see [20])>.

Improvements for AI systems

The improved system can perform substantially more accurate and stable extrapolation beyond the training window for heat-based equations by utilizing self-similar coordinates, as this approach provides a mathematically motivated inductive bias for learning the long-time dynamics of heat-based equations.
The improved system will be able to capture qualitative long-time trends and more reliably reproduce qualitative features that are intrinsic to the underlying dynamics–such as long-time physical patterns and, when present, coarsening/merging behaviors, which are often missed by models trained directly in physical coordinates.
The improved system can achieve this by developing an SSV training framework compatible with standard neural-operator training and using a representation that aligns with the intrinsic scaling structure of the governing dynamics.