Benchmarking Stochastic Interpolants for Modeling Physical Systems

arXiv:2509.26282 · cs.LG · Submitted 2025-09-30 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Benchmarking Stochastic Interpolants for Modeling Physical Systems".

Jane: Here's the gist of the paper. Generative models need to balance deterministic accuracy, spectral consistency, and probabilistic calibration and stochastic interpolants can potentially fulfill these requirements by adjusting their sampling.

Tom: Stochastic Interpolants as a Baseline Stochastic interpolants are defined as a class of generative models that admit arbitrary source and target distributions.

Jane: Given samples x0 from a source distribution ρ0 and samples x1 from a target distribution ρ1, a stochastic interpolant is defined as a stochastic process xt such that xt = I(t, x0, x1) + γ(t)z, t ∈

0, 1: .

Tom: The interpolant I satisfies the boundary conditions I(0, x0, x1) = x0 and I(1, x0, x1) = x1.

Title and authors: Jane: So, to recap what we just heard about "Benchmarking Stochastic Interpolants for Modeling Physical Systems," we’re seeing researchers look at how these new models can be used to map physical states directly instead of relying on simple noise transport. The authors are testing them across several challenging physical problems, including flow and convection, to see if this direct learning approach offers a better path than the standard Gaussian diffusion methods.

Tom: Right. And the core idea they are pushing is that by learning a stochastic process between two states, you can potentially achieve higher quality samples with fewer sampling steps compared to just evolving noise backward or forward through a fixed process. They’re looking at how controlling the amount of added noise influences this efficiency and accuracy, which is pretty important for real-world computation.

Lu: The paper highlights that these methods can map between arbitrary source and target distributions, which means they aren't locked into a specific type of data transformation like just image translation; they can handle different physical mappings. This flexibility is what makes them interesting because it’s not just about one kind of task, but a broader capability.

Meng: Flexibility sounds good in a lab setting, but when I think about running these on real traffic or climate data, the computational cost of training a model that learns this complex drift function might be prohibitive at scale. How much does learning that drift b add to the inference time?

Lalam: The authors do show how you can learn this drift using an empirical loss function, L b, which estimates the time derivative of the stochastic interpolant based on samples taken along the path from source to target. This loss helps train a network that guides the process correctly during inference, allowing you to integrate samples using probability flow ODEs or SDEs like dXODE t = b(t, Xt)dt (four) <ref:2509.26282#pg3>.

The paper's summary: Jane: So, we’ve covered the setup. The main summary of this paper is that stochastic interpolants offer a way to directly learn a path between two physical states without relying solely on the standard Gaussian transport method. They introduce this interpolation x t = I(t, x zero x one) + gamma(t)z, where I connects the source and target distributions exactly at time one <ref:2509.26282#pg3>.

Tom: The key takeaway for me is that they found these interpolants can achieve lower variance mean square error when applied to specific problems. For instance, on Kolmogorov Flow, which deals with turbulent fluid dynamics, they achieved the lowest variance mean square error among all tested methods. That's a solid number behind it.

Lu: And they also looked at Rayleigh-Benard Convection and found a trade-off there; lower pointwise error often comes with higher spectral error, meaning you might get good results locally but lose the overall structure when you look at the whole system across different scales. That’s a crucial caveat to keep in mind.

Meng: A trade-off is always an issue when we move from theory to application, right? If the spectral error gets too big, does that mean the model isn't useful for long-term predictions where global consistency matters?

Lalam: They also looked at weather forecasting and found that these stochastic interpolants tend to perform well on medium-range forecasts, specifically up to ten days. This suggests their ability to handle longer temporal dependencies is stronger than what standard methods might offer in those scenarios.

The paper's improvements: Tom: The paper suggests a few key ways we can improve this approach. One big thing they point out is that pixel space models, which are based on the image directly, tend to have higher errors compared to latent space models, so sticking with latent space representations might be more stable overall.

Jane: That makes sense from a stability standpoint. And another improvement they highlight is that the effect of compression ratio on rollout error isn't as bad as some people thought; even with aggressive compression, it doesn't kill the performance much for generative models using this framework.

Lu: They also showed that when you look at how we sample these interpolants, ODE-based samplers actually require fewer steps to get good results compared to other methods. That’s a practical improvement because fewer steps mean faster inference times, which is something engineers always want.

Meng: Fewer sampling steps are definitely good for deployment speed. But what about the training side? If we have to learn that drift function b using that empirical loss L b, does that make the training process itself computationally expensive or unstable?

Lalam: The paper notes a practical issue with the loss function itself; it can have high variance around the endpoints, specifically at time zero and time one, if the noise coefficient gamma dot(t) becomes singular. That means numerical errors can affect how accurately we learn that drift estimate during training.

Conclusion: Jane: So, wrapping up this look at "Benchmarking Stochastic Interpolants for Modeling Physical Systems," the paper shows that these interpolants are a viable alternative to pure noise transport for physical systems because they can adapt their sampling to achieve better accuracy across different physical tasks. They’re showing promise in areas like Kolmogorov flow and medium-range weather forecasts.

Tom: The main implication is that we can use these models not just for pretty pictures, but as more accurate surrogates for complex dynamics, provided we manage the trade-off between pointwise error and spectral consistency carefully across different physical systems.

Lu: For me, it’s about the representation learning aspect; if latent space models prove to be more stable than pixel space ones, that opens up a lot of doors for modeling things where global structure matters. We need to keep exploring how these learned stochastic processes can generalize beyond the specific datasets they were trained on.

Meng: From an engineering standpoint, we’d want to see clearer benchmarks on real-world traffic data or climate models that actually run continuously, because relying only on idealized datasets doesn't tell us what breaks when it hits production scale.

Lalam: And from a production perspective, the paper confirms that optimizing the sampler is key; using ODE samplers instead of other methods allows us to get good performance with fewer steps, which translates directly into faster inference times for deploying these models.

Anthony Zhou, Amir Barati Farimani, Alexander Wikner, Pedram Hassanzadeh, Amaury Lancelin

Carnegie Mellon University · University of Chicago · LMD/IPSL, CNRS, ENS, Universite PSL é Ecole Polytechnique, Institut Polytechnique de Paris, Reseau de Transport d’ Electricité (RTE)

cs.LG

Submitted: 2025-09-30

Updated: 2026-11-09

Comments: Code and data is available at http://github.com/anthonyzhou-1/interpolant_pdes

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: Generative models need to balance deterministic accuracy, spectral consistency, and probabilistic calibration and stochastic interpolants can potentially fulfill these requirements by adjusting their…

Key concepts

Stochastic Interpolants
These are generative models defined as a process $x_t$ that smoothly connects two different distributions ($ ho_0$ and $ ho_1$). They use an interpolation formula $x_t = I(t, x_0, x_1) + ext{noise}$ to map samples from the source distribution to the target distribution exactly in finite time.
Training Frameworks
The model learns a drift function $b_ heta$ using an empirical loss. This learned drift is crucial for inference because it allows for efficient integration of samples using probability flow ODEs or SDEs, which are mathematical tools used to evolve the state from the source distribution to the target distribution.
Performance Trade-offs
When modeling physical systems, like Rayleigh-Benard Convection, there is often a trade-off. Lower errors in specific points might come at the expense of higher spectral errors (errors across different frequencies). Stochastic interpolants show this balance depending on the application.
Latent vs. Pixel Space Models
Ablation studies suggest that models operating in latent space tend to be more stable and perform better than those operating directly in pixel space, which is where the input data resides. This implies that representing data in a compressed, lower-dimensional latent space can lead to more reliable generative results.

Terminology

Summary

Generative models need to balance deterministic accuracy, spectral consistency, and probabilistic calibration and stochastic interpolants can potentially fulfill these requirements by adjusting their sampling. p. 0

Where the paper says it

This allows for generative models that can use fewer sampling steps and produce more accurate predictions than models relying on transporting Gaussian noise. Our experiments suggest that generative models need to balance deterministic accuracy, spectral consistency, and probabilistic calibration, and that stochastic interpolants can potentially fulfill these requirements by adjusting their sampling.

Stochastic Interpolants as a Baseline

Stochastic interpolants are defined as a class of generative models that admit arbitrary source and target distributions. Given samples x0 from a source distribution ρ0 and samples x1 from a target distribution ρ1, a stochastic interpolant is defined as a stochastic process xt such that xt = I(t, x0, x1) + γ(t)z, t ∈ [0, 1]. p. 1

Where the paper says it

, 2023; Zhang et al. Given samples x0 from a source distribution ρ0 and samples x1 from a target distribution ρ1, a stochastic interpolant is defined as a stochastic process xt such that: xt = I(t, x0, x1) + γ(t)z, t ∈ [0, 1] (1) The interpolant I satisfies the boundary conditions I(0, x0, x1) = x0 and I(1, x0, x1) = x1. p. 1

Where the paper says it

, 2023; Zhang et al. Given samples x0 from a source distribution ρ0 and samples x1 from a target distribution ρ1, a stochastic interpolant is defined as a stochastic process xt such that: xt = I(t, x0, x1) + γ(t)z, t ∈ [0, 1] (1) The interpolant I satisfies the boundary conditions I(0, x0, x1) = x0 and I(1, x0, x1) = x1.

The interpolant I satisfies the boundary conditions I(0, x0, x1) = x0 and I(1, x0, x1) = x1. Furthermore, the interpolant xt maps between the densities ρ0, ρ1 exactly and in finite time. p. 1
Where the paper says it

Firstly, the stochastic interpolant produces samples x0 ∼ ρ0 at t = 0 and x1 ∼ ρ1 at t = 1 by construction. Furthermore, the interpolant xt maps between the densities ρ0, ρ1 exactly and in finite time, which is not true for typical DDPMs in PDE and climate domains.

Lastly, a stochastic interpolant can be realized by either an ODE or an SDE. p. 1
Where the paper says it

Furthermore, the interpolant xt maps between the densities ρ0, ρ1 exactly and in finite time, which is not true for typical DDPMs in PDE and climate domains. Lastly, a stochastic interpolant can be realized by either an ODE or an SDE, which can produce samples xt at any time t ∈ [0, 1].

Training and Sampling Frameworks

The implementation of spatially linear interpolants is given by xt = α(t)x0 + β(t)x1 + γ(t)z. The coefficients α(t), β(t), γ(t) are chosen to satisfy boundary conditions. p. 1

Where the paper says it

Implementation Stochastic interpolants have many instantiations, as well as different training and sampling procedures. We consider spatially linear interpolants, where x0 is sampled from a current solution u(t) and x1 is sampled from a future solution u(t + 1): xt = α(t)x0 + β(t)x1 + γ(t)z (2) The coefficients α(t), β(t), γ(t) are chosen to satisfy boundary conditions.

The drift bθ of the stochastic interpolant can be learned with a network bθ by minimizing the empirical loss Lb[bθ] = 1/N Σ X N i=1 bθ(ti, xi ti)2 − bθ(ti, xi ti)(∂tI(ti, xi 0, xi 1) + ˙γ(t)zi). p. 1
Where the paper says it

We consider spatially linear interpolants, where x0 is sampled from a current solution u(t) and x1 is sampled from a future solution u(t + 1): xt = α(t)x0 + β(t)x1 + γ(t)z (2) The coefficients α(t), β(t), γ(t) are chosen to satisfy boundary conditions. Since these coefficients are specified, we can learn the drift b of the stochastic interpolant with a network bθ by minimizing 2

This drift estimate is useful during inference for integrating samples using the probability flow ODE or SDE: dXODE t = bθ(t, Xt)dt and dXSDE t = bθ(t, Xt)dt + γ˙(t)√t dWt (4). p. 2
Where the paper says it

Intuitively, bθ aims to estimate the time derivative of the stochastic interpolant. This is useful during inference, where samples x0 are integrated using drift estimates bθ according to the probability flow ODE or SDE: dXODE t = bθ(t, Xt)dt, dXSDE t = bθ(t, Xt)dt + γ˙(t) √ t dWt (4) where Wt := √ tz is a Wiener process on t ∈ [0, 1].

Performance on Physical Systems

Models are benchmarked across diverse physical domains and tasks, including Kolmogorov Flow, Rayleigh-Benard Convection, and weather forecasting. For Kolmogorov Flow, stochastic interpolants achieve the lowest VRMSE. p. 5

Where the paper says it

This can be qualitatively seen in Figure 8. Deterministically sampled stochastic interpolants achieve the lowest VRMSE, yet exhibit large spectral errors.

For Rayleigh-Benard Convection, a trade-off is observed where lower pointwise error usually comes at the cost of higher spectral error. p. 5
Where the paper says it

(2025), where it is unstable across the trajectory. For generative models, Rayleigh-Benard convection reveals an interesting ´ trade-off: lower pointwise error usually comes at the cost of higher spectral error.

In weather forecasting, stochastic interpolants tend to do well on medium-range forecasts up to 10 days. p. 6
Where the paper says it

accuracy of generative models but also for understanding their ability to approximate underlying distributions and to capture uncertainty. After training, models are evaluated on medium-range weather forecasting for up to 10 days.

Ablation and Insights

Ablation studies show that pixel space models have higher errors than latent space models, suggesting that latent space models could be more stable. Furthermore, the effect of compression ratio on rollout error is modest for generative models with more aggressive compression. p. 3

Where the paper says it

KF and RBC autoencoders use a compression ratio of 64× and the PlaSim autoencoder uses a compression ratio of 32×. Architectures We use a diffusion transformer (DiT) (Peebles & Xie, 2023) as the backbone for all latent-space generative models.

The choice of sampler and number of sampling steps for stochastic interpolant models shows that ODE-based samplers require fewer steps to obtain good performance. p. 1
Where the paper says it

, blurry images, masked images) to samples from the target distribution (e. , sharp images, in-painted images) can require fewer sampling steps and produce higher quality samples (Albergo et al.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Stochastic interpolants can be used as a competitive baseline for physical emulation by directly learning a stochastic process between current and future states, which allows generative models to leverage the proximity of successive physical distributions.

  2. Improved AI systems can perform faster or more accurate predictions in autoregressive tasks like those in PDEs and climate by using stochastic interpolants trained to map current to future states, as this approach is believed to be wasteful when relying solely on transporting Gaussian noise.

  3. Generative models can be improved by balancing deterministic accuracy, spectral consistency, and probabilistic calibration through the use of stochastic interpolants adjusting their sampling, potentially fulfilling these requirements by adjusting their sampling.

  4. The system can achieve better long-term climate emulators by leveraging the hypothesis that complex, global weather systems evolve at multiple timescales, allowing stochastic interpolants to benefit from related subsequent states separated by 6 hours during sampling.

  5. Models can be optimized for turbulent systems by introducing more stochasticity in the sampling process, as this can help resolve high-frequency features and mitigate the tendency of deterministic models to produce overly smoothed predictions when minimizing pointwise error.

Abstract

Generative models have recently emerged as powerful surrogates for physical systems, demonstrating increased accuracy, stability, and/or statistical fidelity. However, most approaches rely on iteratively denoising a Gaussian, a choice that may not be the most effective for autoregressive prediction tasks, where the current and future states are often closely related. We therefore benchmark generative models based on stochastic interpolants, which can learn maps between arbitrary densities, against standard Gaussian-denoising frameworks across a range of fluid and climate systems. Following prior work, we consider a transport map for dynamical systems that learns an interpolant between a point mass at the current state and a conditional distribution of future states. We find that stochastic interpolants can match or outperform Gaussian-denoising baselines in pointwise accuracy, spectral fidelity, and probabilistic skill. Furthermore, we identify a tradeoff where deterministic sampling can improve pointwise accuracy but smooth predictions, whereas stochastic sampling recovers spectral content and calibration but requires more function evaluations, as well as ensemble averaging to recover pointwise accuracy. As a whole, these results establish stochastic interpolants as a competitive baseline for generative physical emulation, and also indicate that the training loss, sampler, and transport map should be chosen with the physical system and target metric in mind.

Sources

Related papers