A Convex Approximation Framework for Neural Likelihood-Based Bayesian Inverse Problems
Fabian Schneider, Tapio Helin, Leila Taghizadeh
stat.ML, cs.LG, math.PR, stat.ME
Submitted: 2026-07-07
Comments: 22 pages, 3 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Many problems in science and engineering are difficult to model accurately, either due to unknown physical mechanisms, poorly quantified measurement uncertainty, or prohibitive computational costs of
Terminology
Abstract
Many problems in science and engineering are difficult to model accurately, either due to unknown physical mechanisms, poorly quantified measurement uncertainty, or prohibitive computational costs of high-fidelity simulations. These challenges limit the applicability of classical probabilistic inference methods such as Markov chain Monte Carlo, especially in high-dimensional Bayesian inverse problems. As data from scientific experiments become increasingly available, machine learning methods offer a flexible alternative to explicit parametric modelling. We study neural likelihood approximation, where the goal is to learn the likelihood function directly from data without explicit knowledge of the underlying data-generating process. A common approach trains likelihood surrogates by minimizing the Kullback-Leibler divergence between the true posterior and an approximate posterior, which is equivalent to minimizing the expected negative log-likelihood. This work improves the theoretical foundations of neural likelihood approximation by alleviating limitations of restrictive model classes: we show that, by working with un-normalized potentials and folding normalization into the training objective, the resulting learning problem is strictly convex. We show that empirical minimizers of the resulting data-driven objective converge to the true likelihood as the sample size grows. Numerical experiments for the neural likelihood approximation are conducted for a deblurring and a non-linear PDE based imaging problem.
Sources
- Double-loop randomized quasi-Monte Carlo estimator for nested integration
- Upper and lower bounds for local Lipschitz stability of Bayesian posteriors
- The Statistical Accuracy of Neural Posterior and Likelihood Estimation
- Bayesian optimal experimental design with Wasserstein information criteria
- Multilevel neural simulation-based inference
- Neural Density Estimation and Likelihood-free Inference
- BayesFlow: Learning complex stochastic models with invertible neural networks
- Conditional Density Estimation with Neural Networks: Best Practices and Benchmarks
- Sequential Bayesian Design for Efficient Surrogate Construction in the Inversion of Darcy Flows
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches