Sampling from Neural Operator Priors to Solve Partial Differential Equations

arXiv:2610.16396 · cs.LG · Submitted 2026-10-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Sampling from Neural Operator Priors to Solve Partial Differential Equations".

Jane: The gist The authors combine pBS guided sampling with a UNO diffusion prior to solve PDE inverse problems via function-space,

Tom: First, who's behind it and why it matters.

Paper summary: Jane: So we’ve looked at the paper "Sampling from Neural Operator Priors to Solve Partial Differential Equations." What's the main idea here for someone just listening right now?

Tom: Basically, they took a neural operator prior and combined it with particle-guided sampling to tackle solving partial differential equations in function space. It’s about using learned AI knowledge of functions as a guide while running a sequential Monte Carlo method.

Lu: The core idea is that they use this neural operator as a prior to understand how functions change, and then they use pseudo-bootstrap sequential Monte Carlo, or pBS, to navigate the complex PDE problem effectively. It’s linking representation learning with classical numerical solvers in a way that was hard to see before.

Meng: For me as someone who ships models to production, the implication is that if this works well on these benchmarks, it means we can actually solve inverse problems with sparse data that we’re currently struggling with in production. Or is this just a clever experiment for a paper?

Lalam: Based on what the authors show, they achieve the best error versus time trade-off across all their tests. They use this method to recover a likelihood-weighted version of the posterior, which they say is empirically justified by how well it performs.

Tom: The title itself says "Sampling from Neural Operator Priors." That just means we are using those learned neural operator models as a guide while we run our particle samplers to find the solution. It’s about using what the AI has already learned about functions to speed up finding the answer.

Jane: And who wrote this paper? Did any of you recognize those names, or is it just a bunch of researchers from different places putting these pieces together? I want to know if this is coming from a specific lab that’s pushing something new.

Lu: It involves folks working on function-space diffusion models and linking them with sequential Monte Carlo methods. It connects representation learning with classical numerical solvers in a way that was previously hard to see clearly.

Meng: I’m curious about the limitations mentioned. The authors flag that inverse problems are still ill-conditioned, no matter how many particles you use. That means even if the AI guidance is perfect, the math of the original problem itself might be tricky to solve accurately.

Tom: Exactly what I’m saying. So while they got great results on the error versus time graph, it doesn't magically fix a fundamentally ill-posed problem just by having a better sampler. It shows how much guidance helps when things are hard, but it doesn't erase the problem itself.

Jane: So to wrap up this whole discussion on "Sampling from Neural Operator Priors to Solve Partial Differential Equations," the main point is that combining diffusion priors with particle guidance gives us a way to tackle these function space problems efficiently, even if there are still some underlying mathematical challenges with ill-posed data.

Conclusion: Tom: So, we’re wrapping up our look at this paper on "Sampling from Neural Operator Priors to Solve Partial Differential Equations." Basically, they’ve taken a neural operator prior and combined it with particle-guided sampling to solve those tough partial differential equations in function space.

Jane: That sounds like a lot of complex machinery for someone just listening on their commute. Can you break down what that actually means for someone who doesn't read research papers? What’s the core idea behind this whole approach?

Lu: The core idea is using a neural operator, which is basically an AI model trained to understand how functions change, as a prior. Then they use pseudo-bootstrap sequential Monte Carlo sampling—pBS—to guide that sampling process through the complex PDE problem.

Meng: I’m thinking about deployment right now. If this works well on these benchmarks, does it mean we can actually solve inverse problems with sparse data that we’re currently struggling with in production? Or is this just a clever experiment for a paper?

Tom: The title itself says "Sampling from Neural Operator Priors." That just means we’re using those learned neural operator models as a guide while we run our particle samplers to find the solution. It’s about using what the AI has already learned about functions to speed up finding the answer.

Jane: So, who are these authors? Did any of you recognize those names, or is it just a bunch of researchers from different places putting these pieces together? I want to know if this is coming from a specific lab that’s pushing something new.

Meng: I’m curious about the limitations mentioned in the paper. The authors flag that inverse problems remain ill-conditioned regardless of how many particles you use. That means even if the AI guidance is perfect, the math of the original problem itself might be tricky to solve accurately.

Tom: Exactly what I’m saying. So while they got great results on that error versus time graph, it doesn't magically fix a fundamentally ill-posed problem just by having a better sampler. It shows how much guidance helps when things are hard, but it doesn't erase the problem itself.

Andrew Millard, Joshua Murphy, Zheng Zhao

Linköping University · Trinity College Dublin

cs.LG

Submitted: 2026-10-12

Updated: 2026-10-12

Project page: https://ml4physicalsciences.github.io/2024/files/NeurIPS_ML4PS_2024_253.pdf

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist The authors combine pBS guided sampling with a UNO diffusion prior to solve PDE inverse problems via function-space, particle-guided methods Problem Formulation and Background The paper

Key concepts

U-shaped Neural Operator (UNO): A U-shaped Neural Operator acts as a sophisticated denoiser within a function-space diffusion model. It helps refine the noisy data distribution by learning how to map intermediate states back toward the true solution, making the diffusion process more effective for solving PDEs.
Pseudo-Bootstrap Sequential Monte Carlo (pBS): pBS is a technique used to re-weight particles in sequential Monte Carlo methods. It adjusts particle weights based on the ratio of approximate likelihoods at consecutive steps, helping the sampler maintain a target distribution that is useful for guiding the diffusion process.
Function-Space Diffusion Model: This approach uses a neural operator as a prior to model the solution of PDEs directly in function space. Instead of solving for point values, it learns a distribution over functions, allowing it to handle complex problems where classical solvers struggle with high dimensions or nonlinearity.

Summary

The gist The authors combine pBS guided sampling with a UNO diffusion prior to solve PDE inverse problems via function-space, particle-guided methods

Problem Formulation and Background

The paper addresses solving partial differential equations (PDEs) in function space by pairing pseudo-bootstrap sequential Monte Carlo (pBS) with a U-shaped Neural Operator (UNO) diffusion prior Classical numerical solvers scale poorly in dimension, parameter count, and nonlinearity, leading to alternatives like PINNs and Physics-Informed Neural Operators that penalize PDE residuals during training The authors use the U-shaped Neural Operator [UNO; Rahman et al., 2023] as a backbone denoiser of a function-space diffusion model, forming the neural operator diffusion prior They consider joint states x = (a, u) in the separable Hilbert space H = L 2(omega) × L 2(omega), following the EDM formulation The data distribution is corrupted by Gaussian noise with standard deviation σt ∈ [0, σmax], giving the reference measure pref(x):= N (0, σ 2 maxC), where C: H → H is a symmetric, positive-definite, trace-class covariance operator They solve PDEs using a diffusion model pretrained on the joint distribution of coefficients and solutions [Huang et al., 2024] The guidance term requires the intermediate likelihood pθ(y xt, σt) = p(y x0) pθ(x0 xt) dx0, which is intractable This is approximated at the denoiser’s reconstruction, p˜θ(y xt, σt):= p(y x0 = Dθ(xt, σt)), which is exact at t = 0

Methodology and Sampler Adaptation

The authors modify the multistep DPM++(2M) solver of Lu et al. [2025] to accept guidance, creating G-DPM++ This update combines the first-order DPM++ update with a second-order correction and a guidance term, where the guidance term is ζk ∇xk log ˜pθ(y xk, σk) The proposal density of G-DPM++ is intractable, necessitating the use of pBS to maintain its use within a particle-guided sampler The pseudo-bootstrap (pBS) framework re-weights particles by the ratio of approximate likelihoods at consecutive steps, giving a target related to but distinct from the diffusion posterior recovered by exact schemes The pBS weight update is given by Gk−1(xk, xk−1) = p˜θ(y xk−1, σk−1) / q EM(xk−1 xk) q GEMθ(xk−1 xk, y)

Proposal Comparison and Performance

Table 3 shows that G-DPM++ pBS achieves the best error–versus-time trade-off on every benchmark and step count tested Specifically, G-DPM++ pBS achieves a Combined wins status in 51.7% of the 30 forward/inverse problem slots The effect of pBS is significant, as wrapping each proposal in pBS improves accuracy across most benchmarks and step counts For instance, G-DPM++ with pBS is the strongest method in 23 of the 30 inverse-problem settings The gain is largest where the proposal is under-resolved, such as for NS-Bounded at K = 25, where pBS reduces relative error from 0.2765 to 0.1498 through SMC alone

Practical Considerations and Limitations

The pBS framework is well suited for settings where the target solution is deterministic, as it allows approximate solutions to be fast and accurate enough for parameter exploration A limitation noted in the forward problem is that at high K, DPS frequently surpasses SMC-augmented methods because particle diversity offers little benefit when the proposal is under-resolved Furthermore, inverse problems remain ill-conditioned regardless of K, suggesting that a pointwise gradient might be unreliable The residual likelihood term ω weights the PDE residual against observation terms β and γ, making the reliance on the governing equation explicit

Conclusion

The authors conclude that G-DPM++ pBS achieves superior performance in terms of error–time trade-off across all tested benchmarks and step counts The method successfully combines function-space diffusion with particle guidance to address the challenges of solving PDEs via sparse observations The pBS target recovers a likelihood-reweighted version of the posterior induced by the guided sampler, which is justified empirically despite appearing biased relative to the true posterior The work demonstrates that this approach provides a robust and efficient method for solving PDE inverse problems in function space using particle-guided diffusion.

Improvements for AI systems

No longer relying on fixed-resolution grid approximations for function spaces; this allows models to generalize across different discretizations by employing a U-shaped Neural Operator (UNO) prior acting directly on function space.
Implementing G-DPM++ with pseudo-bootstrap sequential Monte Carlo (pBS) provides the best error–time trade-off at every step count tested and achieves the lowest error on most inverse-problem settings, with the largest gains when few sampling steps are used.
The system can perform function space diffusion guided by PDE residuals and sparse observations by employing a guidance term that incorporates both observation likelihoods and PDE residual likelihood, explicitly weighting them as described in Equation (1).
The proposed G-DPM++ solver reuses the cached estimate rather than re-evaluating them, it is comparable in cost per step to first-order GEM while achieving second-order accuracy, significantly improving computational efficiency over previous methods like Heun's integrator.
The pBS framework corrects point-estimate guidance bias via particles even without a tractable proposal density, leading to a 46% reduction in relative error for NS-Bounded at K=25 when compared to the proposal-only method.
The model can reconstruct coefficient fields from sparse observations with high accuracy, as shown by achieving best E and best E×t per row in bold across all benchmarks (Table 1).