Hamiltonian Monte Carlo for (Physics) Dummies

arXiv:2601.01422 · stat.CO, stat.ME · Submitted 2026-01-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Hamiltonian Monte Carlo for (Physics) Dummies".

Jane: Here's the gist of the paper. Sampling-based inference has seen a surge of interest in recent years Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to efficiently explore complex target distributions.

Tom: The Problem with Standard MCMC Standard algorithms such as the Metropolis–Hastings (Hastings, 1970; Metropolis et al., 1953) or the Gibbs sampler (Geman and Geman, 1984) often struggle in high dimensions or when model parameters exhibit strong correlations, leading to slow mixing and poor scalability in modern applications.

Jane: Hamiltonian Monte Carlo (HMC), introduced by Duane et al.

Tom: (1987) and made popular by Neal (2011), has been a breakthrough algorithm providing a sampling solution that overcomes many of these hurdles.

Title and authors: Tom: So to recap, this paper "Hamiltonian Monte Carlo for (Physics) Dummies" is essentially a guide designed to explain the mechanics of HMC, which uses concepts from Hamiltonian dynamics, specifically to sample from complex distributions that are hard for standard Markov chain Monte Carlo methods.

Jane: It’s about taking something that feels very abstract—Hamiltonian dynamics—and showing how it can actually be used for practical sampling problems in Bayesian inference, which is what we do a lot in machine learning.

Lu: The authors are trying to make HMC more accessible, which suggests they are focusing on the foundational concepts and the challenges practitioners face when deploying these algorithms in real-world applications.

Meng: I wonder if this focus on accessibility helps us actually deploy it faster, or if it just gives us a better understanding of why our current implementations sometimes fail to scale up?

Lalam: It lays out that standard methods struggle when your model parameters are highly correlated or you're dealing with very high dimensions, and HMC is presented as the solution because it uses those dynamics to move efficiently through the space.

Tom: Exactly. We know HMC is powerful, but sometimes the implementation details get buried in hyperparameter tuning, so this paper seems to be unpacking that inner workings for the listener who just wants to know what’s going on.

The paper's summary: Jane: So, the summary of "Hamiltonian Monte Carlo for (Physics) Dummies" explains that a typical problem in Bayesian applications is getting samples from a distribution, say the posterior density pi(x), when the functional form of that density isn't something you know explicitly.

Tom: They point out that usually you only know the distribution up to some normalizing constant, which makes getting independent samples really tricky even for moderately sized problems.

Lu: The paper explains that MCMC methods, including HMC, provide a general framework for sampling from these intractable target densities by building a Markov chain whose stationary distribution is exactly what you want.

Meng: So the main point is that the algorithm constructs this chain so it eventually settles on the right distribution pi, even if we don't know the exact shape of pi beforehand. That’s important because most of our models are complex.

Lalam: It emphasizes that an appropriately constructed Markov chain needs to be ergodic, meaning it is guaranteed to converge to that target distribution regardless of where you start the chain.

Jane: And then they introduce the idea that HMC uses Hamiltonian dynamics—basically physics principles—to propose the next candidate sample in a way that respects these underlying physical laws.

The paper's improvements: Tom: Now, this is where it gets interesting because the paper doesn't just explain HMC; it also discusses ways to improve it, and they highlight several key enhancements that make deployment more practical.

Lu: They talk about adaptive integration time selection through something called the No-U-Turn Sampler, or NUTS, which lets the algorithm decide how long to simulate those dynamics automatically instead of you having to manually set a fixed simulation length.

Meng: If you can automate that part of the tuning—the simulation length—that cuts down a lot of the manual labor for setting up and running simulations on our systems, which is huge for production where time matters.

Jane: And they also discuss using stochastic gradient HMC, or SGHMC, when calculating the full gradient is too expensive because you have massive datasets.

Lalam: The paper suggests that SGHMC lets you sample from high-dimensional posteriors even when computing the full gradient is infeasible, and they claim it gives superior posterior estimation accuracy compared to some variational inference methods.

Tom: That sounds promising for big data scenarios, but I always want to check the caveat here—does this superior accuracy come with a trade-off in terms of computational cost per sample?

Lu: Well, the paper also mentions Riemannian manifold HMC, or RHMC, which incorporates a second order position dependent metric tensor. This lets the algorithm exploit the local geometric structure of your target distribution when it’s not a simple ellipse.

Conclusion: Jane: So to wrap up this discussion on "Hamiltonian Monte Carlo for (Physics) Dummies," we see that HMC is a powerful tool because it uses physical principles to explore complex spaces, and this paper makes the mechanics of how it works much clearer for non-physics experts.

Tom: It’s clear that the practical utility comes from understanding the limitations, especially those challenges around tuning and scaling, which this overview addresses by pointing out adaptive methods like NUTS.

Lu: The paper points toward advanced techniques like SGHMC and RHMC as ways to push HMC further into tackling harder problems, specifically when dealing with very large datasets or complex geometric structures in the target distribution.

Meng: From a deployment standpoint, the ability to use adaptive simulation lengths and handle high-dimensional posteriors via stochastic methods is what makes this framework relevant for us right now in production AI.

Lalam: Ultimately, "Hamiltonian Monte Carlo for (Physics) Dummies" shows that while HMC is elegant, its success in real-world applications depends on understanding the mechanics of the Hamiltonian dynamics and knowing when to use these advanced variants.

Arghya Mukherjee, Dootika Vats

Department of Mathematics and Statistics IIT Kanpur

stat.CO, stat.ME

Submitted: 2026-01-04

Updated: 2026-04-04

Comments: 40 pages, 12 figures, 1 table

Code: https://github.com/ArghyaStat/HMC_for_dummies

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 76/100

The gist: Sampling-based inference has seen a surge of interest in recent years Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to…

Key concepts

Hamiltonian Dynamics
This concept treats the target probability distribution as a potential energy landscape. The movement of samples through this space is governed by Hamilton's equations, which describe how a system moves based on its kinetic and potential energy, providing a structured way to explore complex areas of the distribution.
Leapfrog Integrator
To simulate the continuous physical motion described by Hamiltonian dynamics on a computer, the paper uses numerical integration. The leapfrog integrator is a specific type of method that is 'symplectic,' meaning it preserves key properties like volume in phase space, ensuring the simulated movement remains physically meaningful over many small steps.
No-U-Turn Sampler (NUTS)
NUTS is an improvement to HMC that automatically manages how long the simulation runs. Instead of needing a fixed time limit, NUTS adapts by checking for 'turns' in the trajectory. This allows the algorithm to efficiently explore complex distributions without requiring a user to manually set parameters for simulation length.

Terminology

Summary

Sampling-based inference has seen a surge of interest in recent years Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to efficiently explore complex target distributions. p. 0

Where the paper says it

in Abstract Sampling-based inference has seen a surge of interest in recent years. Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to efficiently explore complex target distributions.

The Problem with Standard MCMC

Standard algorithms such as the Metropolis–Hastings (Hastings, 1970; Metropolis et al., 1953) or the Gibbs sampler (Geman and Geman, 1984) often struggle in high dimensions or when model parameters exhibit strong correlations, leading to slow mixing and poor scalability in modern applications. p. 1

Where the paper says it

Yet, standard algorithms such as the Metropolis–Hastings (Hastings, 1970; Metropolis et al. , 1953) or the Gibbs sampler (Geman and Geman, 1984) often struggle in high dimensions or when model parameters exhibit strong correlations, leading to slow mixing and poor scalability in modern applications.

Hamiltonian Monte Carlo (HMC), introduced by Duane et al. p. 1
Where the paper says it

, 1953) or the Gibbs sampler (Geman and Geman, 1984) often struggle in high dimensions or when model parameters exhibit strong correlations, leading to slow mixing and poor scalability in modern applications. Hamiltonian Monte Carlo (HMC), introduced by Duane et al.

(1987) and made popular by Neal (2011), has been a breakthrough algorithm providing a sampling solution that overcomes many of these hurdles. p. 1
Where the paper says it

Hamiltonian Monte Carlo (HMC), introduced by Duane et al. (1987) and made popular by Neal (2011), has been a breakthrough algorithm providing a sampling solution that overcomes many of these hurdles.

Despite its elegance, the practical use of HMC entails many challenges. p. 1
Where the paper says it

A standard HMC algorithm introduces auxiliary variables and employs Hamiltonian dynamics to obtain a next candidate draw. Despite its elegance, the practical use of HMC entails many challenges.

Foundational Concepts in Sampling

The Metropolis-Hastings (MH) algorithm constructs a Markov chain using a user-chosen proposal distribution that generates a candidate sample based on the current state. The MH acceptance function is given by the formula (1) which keeps π invariant for this algorithm. p. 2

Where the paper says it

The specifics are provided in Algorithm 1, which employs a proposal distribution Q with conditional density q(· | x) to generate a candidate sample x ∗ . The acceptance function αMH(x (t) , x∗ ) = min 1, π(x ∗ ) π(x (t)) q(x (t) | x ∗ ) q(x ∗ | x (t)) , (1) is the MH acceptance function that keeps π invariant for this algorithm.

A good proposal should often propose values in high probability areas of π and should also sufficiently explore the tails. The Metropolis-Hastings-Green algorithm (MHGJ) allows for a deterministic proposal in an augmented state-space by augmenting the state with auxiliary variables. p. 4
Where the paper says it

Green (1995), however, identified that one may augment the state-space in such a way that a deterministic proposal in an augmented state-space is in fact random in the original state-space. This is done through the Metropolis–Hastings–Green algorithm with Jacobians (MHGJ)2 .

Hamiltonian Dynamics as the Core Mechanism

The core idea in HMC is to assume that the target density corresponds to a potential energy function, specifically setting U(x) = - log π(x). Points located in the tails of the distribution correspond to regions of high potential energy, and points in the center of the distribution correspond to having low potential energy. p. 8

Where the paper says it

Then a draw X = x ∼ π can be interpreted as a state possessing potential energy U(x). Points located in the tails of the distribution correspond to regions of high potential energy, and points in the center of the distribution correspond to having low potential energy; see Figure 1.

The motion is governed by Hamilton’s equations: dxt/dt = ∂H(xt, pt)/∂pt and dpt/dt = −∂H(xt, pt)/∂xt. p. 7
Where the paper says it

The conservation of the Hamiltonian is incredibly useful in determining how the object will move in a given system. Given U and K for a system, the temporal evolution of the object is governed by Hamilton’s equations: dxt dt = ∂H(xt, pt) ∂pt and dpt dt = − ∂H(xt, pt) ∂xt .

The ideal HMC algorithm uses the map Ps which is an involution that conserves the Hamiltonian. p. 6
Where the paper says it

Implementing the MHGJ algorithm thus entails specifying a suitable auxiliary distribution S and a deterministic map g. HMC does exactly this by choosing S and g by drawing connections with Hamiltonian dynamics.

Practical Implementation and Tuning

In practice, the map Ts is approximated through numerical integration using a leapfrog integrator, which yields an approximation to Ts that is used instead of Ts in Algorithm 3. The leapfrog integrator is a symplectic integrator, implying both volume-preservation and the fact that repeated ε steps do not yield poorer approximations. A successful implementation requires careful tuning of s, ε, L, and M. p. 1

Where the paper says it

In Section 2, we present the Metropolis-Hastings algorithm and its Jacobian-based extension, the Metropolis-Hastings-Green method, from which we lay the foundation of HMC in Section 3. An effective HMC implementation requires careful choice of tuning parameters, heuristics and theory for which are outlined in Section 4.

The choice of mass matrix M controls how the algorithm moves through the augmented parameter space. p. 4
Where the paper says it

Green (1995), however, identified that one may augment the state-space in such a way that a deterministic proposal in an augmented state-space is in fact random in the original state-space. This is done through the Metropolis–Hastings–Green algorithm with Jacobians (MHGJ)2 .

Advanced HMC Variants

Several extensions have been developed to address specific challenges in HMC, including the No-U-Turn Sampler (NUTS) which adaptively determines how long to simulate Hamiltonian dynamics. Stochastic gradient HMC methods are used when full gradient computation is infeasible due to massive datasets. p. 0

Where the paper says it

Keywords— Gradient-based sampling, Hamiltonian dynamics, No U-Turn sampler, Machine learning, Bayesian computation. 1 Introduction Markov chain Monte Carlo (MCMC) methods lie at the heart of Bayesian computation, providing a general mechanism to sample from complex posterior distributions that are analytically intractable.

Riemannian manifold HMC (RHMC) incorporates a second-order position-dependent metric tensor, typically the local Hessian of the potential function. Furthermore, Magnetic HMC (MHMC) and Repelling-Attracting Hamiltonian Monte Carlo (RAHMC) are proposed to address issues like multi-modal distributions. p. 0
Where the paper says it

in Abstract Sampling-based inference has seen a surge of interest in recent years. Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to efficiently explore complex target distributions.

Conclusion and Applications

The introduction of HMC has brought somewhat of a resurgence to Bayesian computation, and it has been successfully employed in a wide variety of applications. The use of HMC algorithms within the machine learning community has become prevalent, particularly for sampling from high-dimensional and potentially complicated distributions. p. 6

Where the paper says it

Since its introduction to statistics by Neal (1996) in neural network modeling, “Hybrid Monte Carlo” has become very popular and was later adopted as “Hamiltonian Monte Carlo” as a generic tool for sampling smooth distributions in Bayesian statistical inference. The Hamiltonian approach offers an alternative methodology for describing motion in particle systems and has served as the foundation for the development of HMC in the statistics community.

The QDHMC framework exemplifies an emerging intersection between quantum simulation and Bayesian computation. p. 0
Where the paper says it

This review article seeks to make HMC more accessible to applied researchers by highlighting its advantages, limitations, and role in enabling scalable and exact Bayesian inference for complex models. Keywords— Gradient-based sampling, Hamiltonian dynamics, No U-Turn sampler, Machine learning, Bayesian computation.

Acknowledgments

The authors are thankful to Andrew Holbrook for his wonder talk available at https://www.youtube.com/Byr9JVBI9cUv and some useful discussions that helped highlight the relationship between HMC and the MHGJ Algorithm. p. 6

Where the paper says it

Implementing the MHGJ algorithm thus entails specifying a suitable auxiliary distribution S and a deterministic map g. HMC does exactly this by choosing S and g by drawing connections with Hamiltonian dynamics.

The authors are also thankful to Sameer Deshpande and Amy Herring for their encouragement to write this article. p. 0
Where the paper says it

We provide a pedagogical overview of HMC that aims to bridge the gap between its theoretical foundations and practical applicability. This review article seeks to make HMC more accessible to applied researchers by highlighting its advantages, limitations, and role in enabling scalable and exact Bayesian inference for complex models.

DV thanks Arushi Agarwal and Sarthak Parikh for creating a safe space to ask all the questions about Hamiltonian dynamics a “physics dummy” would ask. p. 1
Where the paper says it

However, arguably, all works assume a certain amount of knowledge of physics. Despite these excellent works, the authors of this paper still found it challenging to intuitively understand the fine interplay between Hamiltonian dynamics and Monte Carlo sampling.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive integration time selection via No-U-Turn Sampler (NUTS) allows HMC to adaptively determine how long to simulate Hamiltonian dynamics based on local geometric features of the target distribution, substantially improving sampling efficiency without requiring manual tuning of L.

  2. Implementation of Stochastic Gradient HMC (SGHMC) using minibatches of data allows for sampling from high-dimensional posteriors where full gradient computation is infeasible, enabling inference in deep Gaussian processes with superior posterior estimation accuracy compared to the state-of-the-art doubly stochastic variational inference framework.

  3. Integration of Riemannian manifold HMC (RHMC) by incorporating a second order position dependent metric tensor allows the algorithm to exploit the local geometric structure of complex targets, which is beneficial when dealing with non elliptical structures that linear preconditioning cannot regularize everywhere.

  4. Use of diagonal preconditioning based on inverse marginal sample variances improves mixing for components with scale imbalance in high dimensional models, leading to substantially more reliable posterior density estimation.

  5. Utilization of higher order symplectic integrators, such as the implicit midpoint integrator or higher order methods, ensures better conservation of energy and phase space volume during numerical integration, which is crucial for maintaining the quality of the Markov chain when using inexact Hamiltonian trajectory approximations.

Abstract

Sampling-based inference has seen a surge of interest in recent years. Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to efficiently explore complex target distributions. Variants of HMC are available in popular software packages, enabling off-the-shelf implementations that have greatly benefited the statistics and machine learning communities. At the same time, the availability of such black-box implementations has made it challenging for users to understand the inner workings of HMC, especially when they are unfamiliar with the underlying physical principles. We provide a pedagogical overview of HMC that aims to bridge the gap between its theoretical foundations and practical applicability. This review article seeks to make HMC more accessible to applied researchers by highlighting its advantages, limitations, and role in enabling scalable and exact Bayesian inference for complex models.

Sources

Related papers