Stochstic Sampling for Generative Diffusion Models: From Euler-Maruyama to Higher-Order Schemes
math.NA, cs.NA
Submitted: 2026-01-26
Updated: 2026-10-29
Code: https://github.com/emanuelpfarr/SSGDM
License: http://creativecommons.org/licenses/by/4.0/
The gist: We develop a convergence analysis for generative diffusion models that simultaneously accounts for the three principal error sources in stochastic sampling: initialization, score matching…
Terminology
Abstract
We develop a convergence analysis for generative diffusion models that simultaneously accounts for the three principal error sources in stochastic sampling: initialization, score matching, and discretization of the reverse-time stochastic differential equation (SDE). Our central tool is the notion of a iterative-wise general strong scheme, a broad class of discretization methods for the reverse dynamics defined via explicit, index-wise tolerances on their Itô--Taylor coefficients. This notion refines Kloeden and Platen's general strong schemes into an iterate-wise formulation whose per-step conditions imply the classical remainder bound. We prove a 2-Wasserstein convergence theorem that applies to this entire class, reducing the analysis of any concrete sampler to a finite checklist, and covers general forward processes with time-dependent, spatially linear drift and spatially independent diffusion coefficient. We focus on dissipative reverse-time SDEs, which allows us to relax the T-dependence of the discretization and score-matching errors from exponential to polynomial. We instantiate our theorem for Euler--Maruyama, the exponential integrator, and, as our main application, derivative-free stochastic Runge--Kutta schemes of strong order 1.5. To our knowledge, these are the first full convergence guarantees for solvers whose strong order strictly exceeds that of Euler--Maruyama. We further derive the resulting iteration complexity and a parameter-selection rule for the terminal time, score accuracy, and step size. Numerical experiments on Gaussian toy models and CIFAR-10 are consistent with the predicted convergence orders, and on ImageNet, out-of-the-box higher-order SDE solvers can yield substantial speed-ups, requiring at least four times fewer score evaluations than Euler--Maruyama. Code available at: https://github.com/emanuelpfarr/SSGDM.
Sources
- Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance
- On the Importance of Noise Scheduling for Diffusion Models
- Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models
- Gotta Go Fast When Generating Data with Score-Based Models
- Non-asymptotic error bounds for probability flow ODEs under weak log-concavity
- A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models
- Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models
Related papers
- Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
- A Neural-preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions
- Advanced Linear Algebra with Applications - Part I (Numerical linear algebra for PDEs, machine learning, and data assimilation)
- Operator learning meets inverse problems: A probabilistic perspective
- Second-order consistency for learning chaotic dynamics via randomized Jacobian matching
- Windowed thinning and query complexity for the bouncy particle and Zigzag samplers