Multi-modal Laplace-based sampling
math.ST, cs.NA, math.NA, math.PR, stat.CO, stat.TH
Submitted: 2026-11-02
Updated: 2026-11-02
Comments: 40 pages, 8 Figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study the multi-modal Laplace approximation, given by a weighted Gaussian mixture centred at the local minima of the negative log-density…
Abstract
We study the multi-modal Laplace approximation, given by a weighted Gaussian mixture centred at the local minima of the negative log-density, for sampling from probability measures that concentrate around finitely many modes as an inverse-temperature parameter n grows. Such distributions occur in Bayesian inference in the small noise or large data regime. We derive explicit, non-asymptotic bounds on the Hellinger distance between the multi-modal Laplace approximation and the concentrating posterior. The bound decays like n-1/2 with n and the constants depend only polynomially on the dimension d. Moreover, we analyse the Laplace mixture as a proposal distribution for self-normalised importance sampling and independent Metropolis--Hastings (IMH) targeting concentration multimodal distributions. We derive asymptotic results, as n to infinity, on the effective sample size in case of importance sampling as well as on the average acceptance probability, lag-one autocorrelation, and spectral gap in case of IMH. Numerical experiments illustrate and verify our theoretical findings.
Sources
- On importance sampling and independent Metropolis-Hastings with an unbounded weight function
- Solving Poisson's equation for Wasserstein contractive Markov chains
- Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors
- Exact Convergence Rate Analysis of the Independent Metropolis-Hastings Algorithms
Related papers
- Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators
- Nonparametric Regression on Fixed-Cardinality Subsets:Minimax Risk and Random-Design Effects
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models
- Semiparametric Efficient Inference under Non-Informative Complex Survey Designs
- Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel