Improved off-policy training of diffusion samplers
cs.LG, stat.ML
Submitted: 2024-02-07
Updated: 2026-08-28
Comments: NeurIPS 2024; code: https://github.com/GFNOrg/gfn-diffusion
DOI: 10.52202/079017-2575
Code: https://github.com/GFNOrg/gfn-diffusion
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function.
Terminology
Abstract
We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.
Sources
- Posterior samples of source galaxies in strong gravitational lenses with score-based priors
- An optimal control perspective on diffusion-based generative modeling
- From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training
- Nested Sampling Methods
- NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport
- Learning to Scale Logits for Temperature-Conditional GFlowNets
- Improving Gradient-guided Nested Sampling for Posterior Inference
- Thompson sampling for improved exploration in GFlowNets
- Generative Flow Networks as Entropy-Regularized RL
- Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
- Unifying Generative Models with GFlowNets and Beyond
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks