Amortised Inference through One-Step Implicit Sampling
cs.LG, stat.ML
Submitted: 2026-10-08
Updated: 2026-10-08
License: http://creativecommons.org/licenses/by/4.0/
The gist: Efficient sampling from Bayesian posterior distributions in high dimensions remains a hard problem, despite advances in theory and algorithms.
Abstract
Efficient sampling from Bayesian posterior distributions in high dimensions remains a hard problem, despite advances in theory and algorithms. Multiple steps are usually required to draw unbiased samples, for example, iterations of an MCMC algorithm or integration steps in a diffusion-based amortised sampler. However, scalable one-step amortised inference remains a challenging open problem. In this work, we build upon recent progress in data-driven generative modelling to offer a possible solution for one-step amortised sampling. Our one-step implicit sampler (OSIS) uses short-time MCMC dynamics to define a local target for the evolution of an implicitly modelled distribution. In the asymptotic limit, the modelled distribution is a fixed point of the MCMC kernel and thus samples from the target. We analyse the convergence properties of OSIS and show connections with other generative modelling and sampling algorithms. We study various design choices and show that OSIS performs well compared to multi-step amortised samplers and non-amortised MCMC baselines while being far more efficient at sampling time.
Sources
- One-Step Sampler for Boltzmann Distributions via Drifting
- Generative Modeling via Drifting
- Minibatch optimal transport distances; analysis and applications
- Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures
- On the Wasserstein Gradient Flow Interpretation of Drifting Models
- One-Step Generative Modeling via Wasserstein Gradient Flows
- Sinkhorn-Drifting Generative Models
- ESS-Flow: Training-free guidance of flow-based models as inference in source space
- Approximate Inference with Amortised MCMC
- Learning in Implicit Generative Models
- Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization
- Sobolev Regularized MMD Gradient Flow
- Learning Model Reparametrizations: Implicit Variational Inference by Fitting MCMC distributions
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks