Tamed Stochastic Gradient Hamiltonian Monte Carlo

arXiv:2607.14862 · math.OC, math.NA, stat.ML · Submitted 2026-07-16 · Read on arXiv

Zhuoran Wang, Ying Zhang

math.OC, math.NA, stat.ML

Submitted: 2026-07-16

Code: https://github.com/ZhuoranWangCn/tSGHMC

Project page: https://dutangc.github.io/CASdatasets/reference/danish.html

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Langevin Dynamics — 11× in this paper · explained in 3 episodes
• Regularization — 11× in this paper · explained in 3 episodes
• Accuracy — 7× in this paper · explained in 1 episode
• Langevin Monte Carlo — 7× in this paper
• Adam — 4× in this paper · explained in 3 episodes
• Conditional Value-at-Risk — 4× in this paper
• Loss Function — 4× in this paper · explained in 2 episodes
• Theoretical Guarantees — 4× in this paper · explained in 2 episodes
• Lyapunov function — 2× in this paper · explained in 2 episodes
• Bayesian Inference — 1× in this paper · explained in 6 episodes
• Composition — 1× in this paper · explained in 2 episodes
• Conditional Value-at-Risk (CVaR) — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Convergence Guarantees — 1× in this paper · explained in 3 episodes
• Monotonicity — 1× in this paper · explained in 2 episodes
• Non-asymptotic Analysis — 1× in this paper · explained in 2 episodes
• Optimal Transport — 1× in this paper · explained in 5 episodes
• Wasserstein Distance — 1× in this paper · explained in 6 episodes

Sources

• Sharp convergence rates for Langevin dynamics in the nonconvex setting
• An explicit splitting SAV scheme for the kinetic Langevin dynamics
• Adam: A Method for Stochastic Optimization
• Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks
• Contractive kinetic Langevin samplers beyond global Lipschitz continuity
• Robust SGLD algorithm for solving non-convex distributionally robust optimisation problems
• A fully data-driven approach to minimizing CVaR for portfolio of assets via SGLD with discontinuous updating
• Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL Divergence

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
• Lower Bounds for Randomized Linear-Oracle Optimization over Strongly Convex Sets
• Second-Order Optimization with Asynchronous Curvature