Kinetic Langevin Splitting Schemes for Constrained Sampling

arXiv:2603.23397 · stat.ME, cs.NA, math.NA, stat.ML · Submitted 2026-03-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Kinetic Langevin Splitting Schemes for Constrained Sampling".

Paper summary: Tom: So, to recap, the paper "Kinetic Langevin Splitting Schemes for Constrained Sampling" proposes new sampling algorithms based on kinetic Langevin dynamics and motivated by splitting order schemes like BU and BAO families.

Jane: The main thesis is that these methods offer favorable strong order bias rates and computational efficiency when dealing with constrained sampling problems.

Lu: They introduce the CUBU and CBAOAB algorithms, which are based on splitting order discretization schemes like UBU and BAOAB, to sample from a modified target density exp of negative U lambda.

Meng: The motivation for these is advanced numerical methods that split dynamics into components like the BU and BAO families, because they have good strong order bias rates and computational efficiency.

Lalam: They extend this by providing stochastic gradient extensions too, specifically SG-CUBU and SG-BAOAB.

Tom: They then do a step-size stability analysis using assumptions on the underlying potential function, like convexity and smoothness.

Jane: The convergence analysis translates that into complexity bounds in terms of Wasserstein convergence, which tells us the number of steps needed to reach a required accuracy epsilon greater than zero.

Lu: The results demonstrate that these new algorithms offer improved complexity bounds over existing non-splitting methodologies for constrained sampling.

Meng: And when we look at the numerical experiments, they tested this on several problems including a toy example and Bayesian linear regression, circular constraint, triangular constraint, and square constraint.

Tom: The experiments show the performance gains of these constrained splitting schemes compared to older algorithms like CKLMC and SG-CKLMC.

Conclusion: Jane: So looking at the whole paper "Kinetic Langevin Splitting Schemes for Constrained Sampling," what’s the big picture we are getting here?

Tom: Basically, this work successfully introduces new statistical algorithms for constrained sampling based on kinetic Langevin dynamics.

Lu: The authors demonstrated that these proposed schemes show complexity gains over the KLD with the Euler-Maruyama discretization schemes, as shown in Table one <ref:2603.23397#pg1>.

Meng: So for someone who just listens to this show, it means we have a new set of tools for constrained sampling that are theoretically more efficient than what we had before.

Lalam: The numerical evidence backs up the theoretical findings through experiments on a Bayesian linear regression problem.

Tom: It’s about putting these kinetic Langevin splitting schemes into practice, and they’re showing tangible results in terms of how they handle those constraints.

Jane: The implication is that we can expect better efficiency and accuracy when we apply these methods to problems where sampling under constraints is the main challenge in science or engineering.

City University of Hong Kong

stat.ME, cs.NA, math.NA, stat.ML

Submitted: 2026-03-24

Updated: 2026-11-18

Comments: 35 pages

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints <ref:2603.23397#pg2>.

Key concepts

Constrained Sampling
This is the challenging task of generating random samples from a probability distribution while ensuring those samples adhere to specific boundaries or conditions. It is crucial in many statistical problems where the data must fit within certain limits, making traditional sampling methods difficult.
Kinetic Langevin Dynamics
This refers to a specific type of numerical method used for sampling. It models the evolution of a system over time by incorporating both stochastic noise (like Brownian motion) and deterministic forces derived from the distribution's potential energy, allowing it to explore complex probability spaces.
Splitting Order Schemes
These are advanced numerical techniques that break down a complex dynamic process into simpler, manageable parts. By splitting the dynamics into components (e.g., BU and BAO families), researchers can design algorithms that handle different aspects of the sampling problem more effectively.

Terminology

Summary

Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints <ref:2603. p. 0

Where the paper says it

Kinetic Langevin Splitting Schemes for Constrained Sampling Neil K. Chada∗ Lu Yu† April 1, 2026 Abstract Constrained sampling is an important and challenging task in computational statistics, concerned with generating samples from a distribution under certain constraints.

23397#pg2>. This article proposes a series of new sampling algorithms based on the kinetic Langevin dynamics, motivated by advanced numerical methods like splitting order schemes, to achieve favorable strong order rates and computational efficiency <ref:2603. p. 0
Where the paper says it

In this article we propose a series of new sampling algorithms based on the latter of these, specifically the kinetic Langevin dynamics. Our series of algorithms are motivated on advanced numerical methods which are splitting order schemes, which include the BU and BAO families of splitting schemes.

23397#pg6>.

Motivation for Splitting Schemes

The motivation for the proposed algorithms stems from advanced numerical methods known as splitting order schemes, which split the dynamics into different components, such as the BU and BAO families <ref:2603.23397#pg5>. These schemes are advantageous because they possess favorable strong order (bias) rates and computational efficiency <ref:2603. p. 0

Where the paper says it

Our series of algorithms are motivated on advanced numerical methods which are splitting order schemes, which include the BU and BAO families of splitting schemes. Their advantage lies in the fact that they have favorable strong order (bias) rates and computationally efficiency.

23397#pg2>. The paper introduces two families of splitting schemes, the BU splitting family and the BAO splitting family <ref:2603. p. 2
Where the paper says it

This also translates to SDEs which include the KLD. We will introduce two family of splitting schemes, (i) the BU splitting family, and (ii) the BAO splitting 3

23397#pg5>.

Proposed Algorithms

The authors introduce new algorithms for constrained sampling based on sampling from a modified target density, denoted as νλ ∝ exp(−Uλ) <ref:2603.23397#pg2>. Specifically, they propose the CUBU and CBAOAB algorithms, which are based on splitting order discretization schemes like UBU and BAOAB <ref:2603. p. 4

Where the paper says it

Specifically we introduce ULA-type algorithms, based on splitting order discretization schemes, namely UBU and BAOAB. As a result our new algorithms we propose for this work include the CUBU and CBAOAB algorithms.

23397#pg5>. Extensions to stochastic gradients are also provided through SG-CUBU and SG-BAOAB <ref:2603. p. 4
Where the paper says it

• To complement the newly developed constrained sampling algorithms, we also provide extensions to the case of stochastic gradients. This will include both SG-CUBU and SG-BAOAB, as well as a non-splitting scheme for comparison, which was not previously derived.

23397#pg2>.

Theoretical Analysis of Convergence

The paper develops a step-size stability analysis for the new algorithms, utilizing assumptions on the underlying potential, such as convexity and smoothness <ref:2603.23397#pg2>. They provide complexity analysis in terms of Wasserstein convergence, which translates to the number of steps required to achieve an accuracy ε > 0 <ref:2603. p. 4

Where the paper says it

Such an analysis utilizes assumptions on the underlying potential, such as convexity and smoothness assumptions, which we will provide later in within the document. • We provide a complexity analysis in terms of Wasserstein convergence, which translates to the number of steps to achieve an order of accuracy ε > 0.

23397#pg2>. The results show that the new algorithms offer improved complexity bounds over existing non-splitting methodologies <ref:2603. p. 0
Where the paper says it

In particular we provide a number of theoretical insights which include a Wasserstein contraction and convergence results. We are able to demonstrate favorable results, such as improved complexity bounds over existing non-splitting methodologies.

23397#pg2>.

Numerical Verification and Performance Gains

Numerical experiments are conducted on various problems, including a toy example, Bayesian linear regression, circular constraint, triangular constraint, and square constraint <ref:2603.23397#pg2>. The results demonstrate the performance gains of the constrained splitting schemes compared to existing algorithms like CKLMC and SG-CKLMC <ref:2603. p. 4

Where the paper says it

These are also presented in Table 1, which also demonstrate favorable complexity over SG-CKLMC, which we also consider and provide proofs, for this work. • Numerical experiments are provided on a range of problems to demonstrate the performance gains of the constrained splitting schemes.

23397#pg2>.

Conclusion

The work successfully introduces new statistical algorithms for constrained sampling based on kinetic Langevin dynamics <ref:2603.23397#pg2>. The proposed schemes demonstrate complexity gains over the KLD with the Euler-Maruyama discretization schemes, as shown in Table 1 <ref:2603. p. 0

Where the paper says it

com †Department of Data Science, City University of Hong Kong, Hong Kong SAR, lu. hk 1 arXiv:2603.

p. 2
Where the paper says it

Therefore one must resort to discretization schemes. The most obvious discretization method is the Euler-Maruyama (EM) scheme, however existing results suggest alternative methods, based on the bias, complexity and stability.

23397#pg2>. Numerical evidence verifies these theoretical findings through experiments on a Bayesian linear regression problem <ref:2603. p. 0
Where the paper says it

We are able to demonstrate favorable results, such as improved complexity bounds over existing non-splitting methodologies. Our results are verified through numerical experiments on a range of models with constraints, which include a toy example and Bayesian linear regression.

23397#pg2>. The paper suggests directions for future work, such as using additional SG-splitting schemes or exploring confined sampling methods <ref:2603. p. 4
Where the paper says it

Thus far, there has been no connection in understanding splitting schemes for KLD in the context of constrained sampling, which acts as our motivation, in the penalized setting. Before we discuss our contributions of this paper, we provide a brief overview of related work on constrained sampling using various Langevin-based algorithms.

23397#pg2>.

Algorithm Complexity Comparison

The complexity analysis in terms of the number of iterations highlights the improvements offered by the splitting schemes over traditional discretization schemes <ref:2603.23397#pg2>. For instance, for CUBU, the number of iterations required to achieve W2(ν CUBU n, ν) ⩽ ε is of order O(ε-11p+2/4+2p) when p > 2 <ref:2603. p. 4

Where the paper says it

Such an analysis utilizes assumptions on the underlying potential, such as convexity and smoothness assumptions, which we will provide later in within the document. • We provide a complexity analysis in terms of Wasserstein convergence, which translates to the number of steps to achieve an order of accuracy ε > 0.

23397#pg2>.

Stochastic Gradient Complexity

The complexity results for the stochastic gradient versions, such as SG-CUBU and SG-CBAOAB, are also presented in Table 1 <ref:2603.23397#pg2>. While the SG-CBAOAB has a slower convergence rate than SG-CUBU due to an additional B-step bringing in extra stochastic-gradient error <ref:2603.23397#pg2>. The complexity of SG-CBAOAB is O(ε-25p+6/2p+4) for W2(ν SG-CBAOAB n, ν) <ref:2603. p. 0

Where the paper says it

1 Extension to SG-CUBU . 2 Convergence Analysis of CBAOAB .

23397#pg2>.

Proof of Convergence

The proofs for the convergence of the algorithms are provided in Appendix A, covering CUBU, SG-CUBU, CBAOAB, SG-CBAOAB and SG-CKLMC <ref:2603.23397#pg2>. The proof for Theorem 3. p. 5

Where the paper says it

Algorithm Complexity Reference Metric Projection CLMC (PULMC) O(ε −7 ) [GHZ24] W2 Euclidean CKLMC O(ε −4 ) [YY25b] W2 Euclidean CUBU O(ε −3 ) This work, Theorem 3. 1 W1 Bregman, Gauge CUBU O(ε −3 ) This work, Theorem 3.

4 follows directly from Proposition A. p. 4
Where the paper says it

However, this is not the only setup one could consider for constrained settings. A natural, but difficult setting could be to consider manifolds directly, which require sophisticated mathematical methods to effectively sample from the target.

1 and using the triangle inequality which obtains the same constants from Proposition 2. p. 4
Where the paper says it

• Numerical experiments are provided on a range of problems to demonstrate the performance gains of the constrained splitting schemes. This will include a toy problem, with constraints defined through both a triangle and circle, and a more advanced numerical example of Bayesian logistic regression.

1 <ref:2603.23397#pg2>. The proof for SG-CKLMC relies on Lemma A. p. 0
Where the paper says it

1 Extension to SG-CBAOAB . 3 Results for SG-CKLMC .

2, which provides a uniform second moment bound for the SG-CKLMC iterates <ref:2603. p. 8
Where the paper says it

(ii) E h (G, ρ) − ∇f(θ) 2 θ i ⩽ σ 2 1L 2 (∥θ∥ 2 ) (bound on the stochastic gradient). (iii) E h ∇θ(G, ρ) − ∇2f(θ) 2 θ i ⩽ σ 2 2 (bound on the second derivative).

23397#pg2>.

Numerical Results Summary

The numerical experiments confirm that the sampling capability of CKLMC performs the worst, with many samples outside the convex set K <ref:2603.23397#pg2>. In contrast, splitting schemes like CUBU and CBAOAB show similar performance in terms of the number of samples within the set <ref:2603.23397#pg2>. The comparison between full-gradient and stochastic gradient methods shows that splitting schemes perform better in both cases <ref:2603.23397#pg2>. The runtime comparisons in Table 2 show that CKLMC is cheaper but performs worse than the splitting schemes <ref:2603. p. 4

Where the paper says it

These are also presented in Table 1, which also demonstrate favorable complexity over SG-CKLMC, which we also consider and provide proofs, for this work. • Numerical experiments are provided on a range of problems to demonstrate the performance gains of the constrained splitting schemes.

23397#pg2>.

Stochastic Gradient Error Analysis

The analysis of the SG-CBAOAB chain shows that its complexity results are worse than SG-CUBU because of the additional B-step bringing in extra stochastic-gradient error <ref:2603. p. 9

Where the paper says it

The key underlying idea is that if one shows contraction between two measures, then this implies a unique invariant measure and convergence towards it. The general setup of these results follow from Wq(ν0P n h , ν) ≤ Wq(ν0P n h , νλ ) | {z } sampling error + Wq(ν λ , ν) | {z } approxiamtion error , where ν0 is some initial measure, P n h is a Markov transition kernel based on some algorithm with step-size h > 0 after n-steps, and ν is the target measure.

23397#pg2>. The slower convergence rate is mainly due to this additional error term leading to a less favorable overall error bound <ref:2603. p. 3
Where the paper says it

However, unlike UBU, they do not attain the same rate for the strong error. A contraction and convergence analysis was presented, 4

p. 3
Where the paper says it

However, unlike UBU, they do not attain the same rate for the strong error. A contraction and convergence analysis was presented, 4

23397#pg2>.

Convergence Rates in Wasserstein Distance

The theoretical results provide bounds on the Wasserstein distance, demonstrating that Wq(ν, νλ) → 0 as λ → 0 <ref:2603.23397#pg2>. The analysis of the SG-CBAOAB chain shows that its complexity results are worse than SG-CUBU because of the additional B-step bringing in extra stochastic-gradient error <ref:2603.23397#pg2>. The slower convergence rate is mainly due to this additional error term leading to a less favorable overall error bound <ref:2603.23397#pg2>.

Final Bound Derivation

The final bound for the Wasserstein-1 distance is derived by combining bounds from the kernel perturbation, bias to the target, and kernel-level Lyapunov drift in V <ref:2603.23397#pg2>. This yields a comprehensive result for W2(ν SG-CBAOAB n, ν) <ref:2603. p. 4

Where the paper says it

Specifically we introduce ULA-type algorithms, based on splitting order discretization schemes, namely UBU and BAOAB. As a result our new algorithms we propose for this work include the CUBU and CBAOAB algorithms.

23397#pg2>. The bound is derived by combining bounds from the kernel perturbation, bias to the target, and kernel-level Lyapunov drift in V <ref:2603. p. 9
Where the paper says it

The general setup of these results follow from Wq(ν0P n h , ν) ≤ Wq(ν0P n h , νλ ) | {z } sampling error + Wq(ν λ , ν) | {z } approxiamtion error , where ν0 is some initial measure, P n h is a Markov transition kernel based on some algorithm with step-size h > 0 after n-steps, and ν is the target measure. Providing bounds of the form, translates to how many n-steps are required to achieve an accuracy of ε, for ε > 0.

23397#pg2>.

Algorithm Implementation Details

The algorithms are presented in algorithmic form in Appendix B <ref:2603.23397#pg2>. Algorithm 1 for SG-CUBU is detailed on page 35, and Algorithm 2 for SG-CBAOAB is detailed on page 35 <ref:2603. p. 8

Where the paper says it

2) for λ > 0, arising from Eqn. Algorithm 2 in the appendix provides an algorithmic form of the CBAOAB.

23397#pg2>. Algorithm 3 for SG-CKLMC is detailed on page 35 <ref:2603. p. 5
Where the paper says it

Algorithm Complexity Reference Metric Projection CLMC (PULMC) O(ε −7 ) [GHZ24] W2 Euclidean CKLMC O(ε −4 ) [YY25b] W2 Euclidean CUBU O(ε −3 ) This work, Theorem 3. 1 W1 Bregman, Gauge CUBU O(ε −3 ) This work, Theorem 3.

23397#pg2>. The paper provides a comparison of runtime between all the algorithms in Table 2 <ref:2603. p. 4
Where the paper says it

This will include both SG-CUBU and SG-BAOAB, as well as a non-splitting scheme for comparison, which was not previously derived. These are also presented in Table 1, which also demonstrate favorable complexity over SG-CKLMC, which we also consider and provide proofs, for this work.

23397#pg2>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Splitting order schemes like CUBU and CBAOAB can be used to generate samples from distributions under constraints by splitting the kinetic Langevin dynamics into different components, which is motivated by advanced numerical methods which are splitting order schemes.

  2. The new algorithms allow for complexity analysis in terms of Wasserstein convergence, providing improved complexity bounds over existing non-splitting methodologies and showing better rates compared to methods like CKLMC.

  3. Extensions to stochastic gradients, such as SG-CUBU and SG-CBAOAB, enable the use of mini-batching without replacement for stochastic gradient approximations of the potential function, which allows for unbiased estimation using underdamped Langevin dynamics.

  4. The algorithms can be tuned with a tuning parameter lambda to control approximation error, and theoretical results provide bounds on Wasserstein distance that depend on this parameter, allowing for precise control over the accuracy achieved.

  5. The step-size stability analysis developed for these new algorithms provides guarantees on performance under different conditions, such as assumptions on the underlying potential, such as convexity and smoothness assumptions.

Abstract

Constrained sampling, the task of generating samples from a distribution restricted to a convex set, is an important problem in computational statistics. In this paper, we propose new unadjusted sampling algorithms obtained by applying the symmetric BAOAB and ABOBA splittings, of the kinetic Langevin dynamics, to a penalized surrogate of the constrained target. This also includes their stochastic gradient versions. Since the Hessian of the penalized surrogate is discontinuous across the boundary of the constraint set, analyses relying on a Lipschitz Hessian do not apply. We therefore develop an analysis based only on strong convexity and smoothness, proving one-step Wasserstein contraction schemes, non-asymptotic Wasserstein bounds of both schemes and complexity bounds. The resulting complexities for the stochastic gradient versions improve substantially on traditional Euler-type discretizations. Numerical experiments verify our theory, which is tested on a toy problem and Bayesian constrained regression problem. Overall, the splitting schemes demonstrate stability at step sizes several times larger than Euler-type methods, require five to seventeen times fewer gradient evaluations for a given accuracy.

Sources

Related papers