Kinetic Langevin Splitting Schemes for Constrained Sampling
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Kinetic Langevin Splitting Schemes for Constrained Sampling".
Paper summary: Tom: So, to recap, the paper "Kinetic Langevin Splitting Schemes for Constrained Sampling" proposes new sampling algorithms based on kinetic Langevin dynamics and motivated by splitting order schemes like BU and BAO families.
Jane: The main thesis is that these methods offer favorable strong order bias rates and computational efficiency when dealing with constrained sampling problems.
Lu: They introduce the CUBU and CBAOAB algorithms, which are based on splitting order discretization schemes like UBU and BAOAB, to sample from a modified target density exp of negative U lambda.
Meng: The motivation for these is advanced numerical methods that split dynamics into components like the BU and BAO families, because they have good strong order bias rates and computational efficiency.
Lalam: They extend this by providing stochastic gradient extensions too, specifically SG-CUBU and SG-BAOAB.
Tom: They then do a step-size stability analysis using assumptions on the underlying potential function, like convexity and smoothness.
Jane: The convergence analysis translates that into complexity bounds in terms of Wasserstein convergence, which tells us the number of steps needed to reach a required accuracy epsilon greater than zero.
Lu: The results demonstrate that these new algorithms offer improved complexity bounds over existing non-splitting methodologies for constrained sampling.
Meng: And when we look at the numerical experiments, they tested this on several problems including a toy example and Bayesian linear regression, circular constraint, triangular constraint, and square constraint.
Tom: The experiments show the performance gains of these constrained splitting schemes compared to older algorithms like CKLMC and SG-CKLMC.
Conclusion: Jane: So looking at the whole paper "Kinetic Langevin Splitting Schemes for Constrained Sampling," what’s the big picture we are getting here?
Tom: Basically, this work successfully introduces new statistical algorithms for constrained sampling based on kinetic Langevin dynamics.
Lu: The authors demonstrated that these proposed schemes show complexity gains over the KLD with the Euler-Maruyama discretization schemes, as shown in Table one <ref:2603.23397#pg1>.
Meng: So for someone who just listens to this show, it means we have a new set of tools for constrained sampling that are theoretically more efficient than what we had before.
Lalam: The numerical evidence backs up the theoretical findings through experiments on a Bayesian linear regression problem.
Tom: It’s about putting these kinetic Langevin splitting schemes into practice, and they’re showing tangible results in terms of how they handle those constraints.
Jane: The implication is that we can expect better efficiency and accuracy when we apply these methods to problems where sampling under constraints is the main challenge in science or engineering.
City University of Hong Kong
stat.ME, cs.NA, math.NA, stat.ML
Submitted: 2026-03-24
Updated: 2026-11-18
Comments: 35 pages
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints <ref:2603.23397#pg2>.
Key concepts
- Constrained Sampling
- This is the challenging task of generating random samples from a probability distribution while ensuring those samples adhere to specific boundaries or conditions. It is crucial in many statistical problems where the data must fit within certain limits, making traditional sampling methods difficult.
- Kinetic Langevin Dynamics
- This refers to a specific type of numerical method used for sampling. It models the evolution of a system over time by incorporating both stochastic noise (like Brownian motion) and deterministic forces derived from the distribution's potential energy, allowing it to explore complex probability spaces.
- Splitting Order Schemes
- These are advanced numerical techniques that break down a complex dynamic process into simpler, manageable parts. By splitting the dynamics into components (e.g., BU and BAO families), researchers can design algorithms that handle different aspects of the sampling problem more effectively.
Terminology
Summary
Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints <ref:2603. p. 0 Kinetic Langevin Splitting Schemes for Constrained Sampling
Neil K. Chada∗ Lu Yu†
April 1, 2026
Abstract
Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints. In this article we propose a series of new sampling algorithms based on
the latter of these, specifically the kinetic Langevin dynamics. Our series of algorithms are motivated on advanced numerical methods which are splitting order schemes, which include the BU
and BAO families of splitting schemes.Where the paper says it
Where the paper says it
Motivation for Splitting Schemes
The motivation for the proposed algorithms stems from advanced numerical methods known as splitting order schemes, which split the dynamics into different components, such as the BU and BAO families <ref:2603.23397#pg5>. These schemes are advantageous because they possess favorable strong order (bias) rates and computational efficiency <ref:2603. p. 0 Our series of algorithms are motivated on advanced numerical methods which are splitting order schemes, which include the BU
and BAO families of splitting schemes. Their advantage lies in the fact that they have favorable
strong order (bias) rates and computationally efficiency. This also translates to SDEs which include the KLD. We
will introduce two family of splitting schemes, (i) the BU splitting family, and (ii) the BAO splitting
3Where the paper says it
Where the paper says it
Proposed Algorithms
The authors introduce new algorithms for constrained sampling based on sampling from a modified target density, denoted as νλ ∝ exp(−Uλ) <ref:2603.23397#pg2>. Specifically, they propose the CUBU and CBAOAB algorithms, which are based on splitting order discretization schemes like UBU and BAOAB <ref:2603. p. 4 Specifically we introduce ULA-type algorithms, based on splitting order discretization schemes,
namely UBU and BAOAB. As a result our new algorithms we propose for this work include the
CUBU and CBAOAB algorithms. • To complement the newly developed constrained sampling algorithms, we also provide extensions
to the case of stochastic gradients. This will include both SG-CUBU and SG-BAOAB, as well
as a non-splitting scheme for comparison, which was not previously derived.Where the paper says it
Where the paper says it
Theoretical Analysis of Convergence
The paper develops a step-size stability analysis for the new algorithms, utilizing assumptions on the underlying potential, such as convexity and smoothness <ref:2603.23397#pg2>. They provide complexity analysis in terms of Wasserstein convergence, which translates to the number of steps required to achieve an accuracy ε > 0 <ref:2603. p. 4 Such
an analysis utilizes assumptions on the underlying potential, such as convexity and smoothness
assumptions, which we will provide later in within the document. • We provide a complexity analysis in terms of Wasserstein convergence, which translates to the
number of steps to achieve an order of accuracy ε > 0. In particular we provide a number of
theoretical insights which include a Wasserstein contraction and convergence results. We are able
to demonstrate favorable results, such as improved complexity bounds over existing non-splitting
methodologies.Where the paper says it
Where the paper says it
Numerical Verification and Performance Gains
Numerical experiments are conducted on various problems, including a toy example, Bayesian linear regression, circular constraint, triangular constraint, and square constraint <ref:2603.23397#pg2>. The results demonstrate the performance gains of the constrained splitting schemes compared to existing algorithms like CKLMC and SG-CKLMC <ref:2603. p. 4 These are also
presented in Table 1, which also demonstrate favorable complexity over SG-CKLMC, which we
also consider and provide proofs, for this work. • Numerical experiments are provided on a range of problems to demonstrate the performance
gains of the constrained splitting schemes.Where the paper says it
Conclusion
The work successfully introduces new statistical algorithms for constrained sampling based on kinetic Langevin dynamics <ref:2603.23397#pg2>. The proposed schemes demonstrate complexity gains over the KLD with the Euler-Maruyama discretization schemes, as shown in Table 1 <ref:2603. p. 0 com
†Department of Data Science, City University of Hong Kong, Hong Kong SAR, lu. hk
1
arXiv:2603. Therefore one must resort to discretization schemes. The most obvious discretization method is the Euler-Maruyama (EM) scheme, however existing results suggest alternative
methods, based on the bias, complexity and stability. We are able
to demonstrate favorable results, such as improved complexity bounds over existing non-splitting
methodologies. Our results are verified through numerical experiments on a range of models with
constraints, which include a toy example and Bayesian linear regression. Thus far, there has been no connection in understanding splitting schemes for KLD in the context
of constrained sampling, which acts as our motivation, in the penalized setting. Before we discuss our
contributions of this paper, we provide a brief overview of related work on constrained sampling using
various Langevin-based algorithms.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Algorithm Complexity Comparison
The complexity analysis in terms of the number of iterations highlights the improvements offered by the splitting schemes over traditional discretization schemes <ref:2603.23397#pg2>. For instance, for CUBU, the number of iterations required to achieve W2(ν CUBU n, ν) ⩽ ε is of order O(ε-11p+2/4+2p) when p > 2 <ref:2603. p. 4 Such
an analysis utilizes assumptions on the underlying potential, such as convexity and smoothness
assumptions, which we will provide later in within the document. • We provide a complexity analysis in terms of Wasserstein convergence, which translates to the
number of steps to achieve an order of accuracy ε > 0.Where the paper says it
Stochastic Gradient Complexity
The complexity results for the stochastic gradient versions, such as SG-CUBU and SG-CBAOAB, are also presented in Table 1 <ref:2603.23397#pg2>. While the SG-CBAOAB has a slower convergence rate than SG-CUBU due to an additional B-step bringing in extra stochastic-gradient error <ref:2603.23397#pg2>. The complexity of SG-CBAOAB is O(ε-25p+6/2p+4) for W2(ν SG-CBAOAB n, ν) <ref:2603. p. 0 1 Extension to SG-CUBU . 2 Convergence Analysis of CBAOAB .Where the paper says it
Proof of Convergence
The proofs for the convergence of the algorithms are provided in Appendix A, covering CUBU, SG-CUBU, CBAOAB, SG-CBAOAB and SG-CKLMC <ref:2603.23397#pg2>. The proof for Theorem 3. p. 5 Algorithm Complexity Reference Metric Projection
CLMC (PULMC) O(ε
−7
) [GHZ24] W2 Euclidean
CKLMC O(ε
−4
) [YY25b] W2 Euclidean
CUBU O(ε
−3
) This work, Theorem 3. 1 W1 Bregman, Gauge
CUBU O(ε
−3
) This work, Theorem 3. However, this is not the only setup one could consider for constrained settings. A natural, but difficult
setting could be to consider manifolds directly, which require sophisticated mathematical methods to
effectively sample from the target. • Numerical experiments are provided on a range of problems to demonstrate the performance
gains of the constrained splitting schemes. This will include a toy problem, with constraints
defined through both a triangle and circle, and a more advanced numerical example of Bayesian
logistic regression. 1 Extension to SG-CBAOAB . 3 Results for SG-CKLMC . (ii) E
h
(G, ρ) − ∇f(θ)
2
θ
i
⩽ σ
2
1L
2
(∥θ∥
2
) (bound on the stochastic gradient). (iii) E
h
∇θ(G, ρ) − ∇2f(θ)
2
θ
i
⩽ σ
2
2
(bound on the second derivative).Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Numerical Results Summary
The numerical experiments confirm that the sampling capability of CKLMC performs the worst, with many samples outside the convex set K <ref:2603.23397#pg2>. In contrast, splitting schemes like CUBU and CBAOAB show similar performance in terms of the number of samples within the set <ref:2603.23397#pg2>. The comparison between full-gradient and stochastic gradient methods shows that splitting schemes perform better in both cases <ref:2603.23397#pg2>. The runtime comparisons in Table 2 show that CKLMC is cheaper but performs worse than the splitting schemes <ref:2603. p. 4 These are also
presented in Table 1, which also demonstrate favorable complexity over SG-CKLMC, which we
also consider and provide proofs, for this work. • Numerical experiments are provided on a range of problems to demonstrate the performance
gains of the constrained splitting schemes.Where the paper says it
Stochastic Gradient Error Analysis
The analysis of the SG-CBAOAB chain shows that its complexity results are worse than SG-CUBU because of the additional B-step bringing in extra stochastic-gradient error <ref:2603. p. 9 The key underlying idea is that if one shows
contraction between two measures, then this implies a unique invariant measure and convergence
towards it. The general setup of these results follow from
Wq(ν0P
n
h
, ν) ≤ Wq(ν0P
n
h
, νλ
)
| {z }
sampling error
+ Wq(ν
λ
, ν)
| {z }
approxiamtion error
,
where ν0 is some initial measure, P
n
h
is a Markov transition kernel based on some algorithm with
step-size h > 0 after n-steps, and ν is the target measure. However, unlike UBU, they
do not attain the same rate for the strong error. A contraction and convergence analysis was presented,
4 However, unlike UBU, they
do not attain the same rate for the strong error. A contraction and convergence analysis was presented,
4Where the paper says it
Where the paper says it
Where the paper says it
Convergence Rates in Wasserstein Distance
The theoretical results provide bounds on the Wasserstein distance, demonstrating that Wq(ν, νλ) → 0 as λ → 0 <ref:2603.23397#pg2>. The analysis of the SG-CBAOAB chain shows that its complexity results are worse than SG-CUBU because of the additional B-step bringing in extra stochastic-gradient error <ref:2603.23397#pg2>. The slower convergence rate is mainly due to this additional error term leading to a less favorable overall error bound <ref:2603.23397#pg2>.
Final Bound Derivation
The final bound for the Wasserstein-1 distance is derived by combining bounds from the kernel perturbation, bias to the target, and kernel-level Lyapunov drift in V <ref:2603.23397#pg2>. This yields a comprehensive result for W2(ν SG-CBAOAB n, ν) <ref:2603. p. 4 Specifically we introduce ULA-type algorithms, based on splitting order discretization schemes,
namely UBU and BAOAB. As a result our new algorithms we propose for this work include the
CUBU and CBAOAB algorithms. The general setup of these results follow from
Wq(ν0P
n
h
, ν) ≤ Wq(ν0P
n
h
, νλ
)
| {z }
sampling error
+ Wq(ν
λ
, ν)
| {z }
approxiamtion error
,
where ν0 is some initial measure, P
n
h
is a Markov transition kernel based on some algorithm with
step-size h > 0 after n-steps, and ν is the target measure. Providing bounds of the form, translates to
how many n-steps are required to achieve an accuracy of ε, for ε > 0.Where the paper says it
Where the paper says it
Algorithm Implementation Details
The algorithms are presented in algorithmic form in Appendix B <ref:2603.23397#pg2>. Algorithm 1 for SG-CUBU is detailed on page 35, and Algorithm 2 for SG-CBAOAB is detailed on page 35 <ref:2603. p. 8 2)
for λ > 0, arising from Eqn. Algorithm 2 in the appendix provides an algorithmic form of the
CBAOAB. Algorithm Complexity Reference Metric Projection
CLMC (PULMC) O(ε
−7
) [GHZ24] W2 Euclidean
CKLMC O(ε
−4
) [YY25b] W2 Euclidean
CUBU O(ε
−3
) This work, Theorem 3. 1 W1 Bregman, Gauge
CUBU O(ε
−3
) This work, Theorem 3. This will include both SG-CUBU and SG-BAOAB, as well
as a non-splitting scheme for comparison, which was not previously derived. These are also
presented in Table 1, which also demonstrate favorable complexity over SG-CKLMC, which we
also consider and provide proofs, for this work.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Splitting order schemes like CUBU and CBAOAB can be used to generate samples from distributions under constraints by splitting the kinetic Langevin dynamics into different components, which is motivated by
advanced numerical methods which are splitting order schemes
. -
The new algorithms allow for complexity analysis in terms of Wasserstein convergence, providing
improved complexity bounds over existing non-splitting methodologies
and showing better rates compared to methods like CKLMC. -
Extensions to stochastic gradients, such as SG-CUBU and SG-CBAOAB, enable the use of mini-batching without replacement for stochastic gradient approximations of the potential function, which allows for
unbiased estimation using underdamped Langevin dynamics
. -
The algorithms can be tuned with a tuning parameter lambda to control approximation error, and theoretical results provide bounds on Wasserstein distance that depend on this parameter, allowing for precise control over the accuracy achieved.
-
The step-size stability analysis developed for these new algorithms provides guarantees on performance under different conditions, such as
assumptions on the underlying potential, such as convexity and smoothness assumptions
.
Abstract
Constrained sampling, the task of generating samples from a distribution restricted to a convex set, is an important problem in computational statistics. In this paper, we propose new unadjusted sampling algorithms obtained by applying the symmetric BAOAB and ABOBA splittings, of the kinetic Langevin dynamics, to a penalized surrogate of the constrained target. This also includes their stochastic gradient versions. Since the Hessian of the penalized surrogate is discontinuous across the boundary of the constraint set, analyses relying on a Lipschitz Hessian do not apply. We therefore develop an analysis based only on strong convexity and smoothness, proving one-step Wasserstein contraction schemes, non-asymptotic Wasserstein bounds of both schemes and complexity bounds. The resulting complexities for the stochastic gradient versions improve substantially on traditional Euler-type discretizations. Numerical experiments verify our theory, which is tested on a toy problem and Bayesian constrained regression problem. Overall, the splitting schemes demonstrate stability at step sizes several times larger than Euler-type methods, require five to seventeen times fewer gradient evaluations for a given accuracy.
Sources
- Complexity of randomized algorithms for underdamped Langevin dynamics
- Non-Reversible Langevin Algorithms for Constrained Sampling
- Instance-dependent Convergence Theory for Diffusion Models
- Numerical integrators for confined Langevin dynamics
- Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Acceleration
Related papers
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes
- Covariance-Calibrated Prediction-Powered Inference