LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs
Maria-Nicoleta Crăciun, C. -H. Luke Ong, Tom Schrijvers, Sam Staton
stat.ML, cs.LG, cs.PL, stat.CO
Submitted: 2026-08-09
Updated: 2026-08-11
Comments: 41 pages
Journal ref: Proc. ACM Program. Lang. 10, ICFP, Article 298 (2026)
DOI: 10.1145/3828696
Code: https://github.com/lazyppl-team/lazyhmc
License: http://creativecommons.org/licenses/by/4.0/
The gist: Hamiltonian Monte Carlo (HMC) is a successful generic inference method in probabilistic programming, but in its ordinary formulation it needs gradients and finite-dimensional parameter spaces.
Terminology
Abstract
Hamiltonian Monte Carlo (HMC) is a successful generic inference method in probabilistic programming, but in its ordinary formulation it needs gradients and finite-dimensional parameter spaces. In Haskell, lazy evaluation lets probabilistic programs express stochastic processes and other non-parametric Bayesian models over implicit infinite-dimensional spaces. This paper develops new formulations of gradient-based HMC for this infinite-dimensional setting, via lazy evaluation. For automatic differentiation, we provide an analysis based on a new notion of "piecewise analytic under cylindrical analytic partition" (PACAP), to show that even if a program is infinite-dimensional and defined lazily, the gradient of the likelihood function is finitely supported. For the Monte Carlo method itself, we develop several HMC variants and a No-U-Turn Sampler that operate over the infinite-dimensional parameter space but are still productive because of lazy evaluation. Experiments cover Gaussian mixture clustering, random walks, and piecewise-constant regression with Poisson-process changepoints.
Sources
- A general perspective on the Metropolis-Hastings kernel
- A Conceptual Introduction to Hamiltonian Monte Carlo
- GIST: Gibbs self-tuning for locally adaptive Hamiltonian Monte Carlo
- Automating Involutive MCMC using Probabilistic and Differentiable Programming
- An Introduction to Probabilistic Programming
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches