Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging".
Jane: Here's the gist of the paper. Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent, leading to posterior distributions that combine continuous parameter uncertainty with discrete model uncertainty.
Tom: The authors propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels.
Jane: This integration is rejection-free and requires no additional likelihood evaluations while retaining the non-reversible and ballistic nature of the component PDMP dynamics.
Tom: They prove that this resulting process leaves the intended mixed-dimensional target invariant.
Title and authors: Tom: So we're looking at this paper called "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging" and I'm thinking about the title itself, it sounds pretty heavy. It suggests combining these different types of uncertainty into one process.
Jane: Exactly, and what it means for us is that we can assign a positive prior probability to lower dimensional submodels, which enriches the range of uncertainty that Bayesian models can represent.
Lu: That enrichment means the resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty in a way that was previously difficult to handle efficiently.
Meng: But the main challenge is figuring out how to move between these different submodels without having to do all those extra likelihood evaluations we usually have to run.
Lalam: The paper proposes the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes across different submodels in a rejection-free way.
The paper's summary: Tom: Let's unpack that framework a bit more. Basically, they are taking these piecewise deterministic Markov processes, or PDMPs, and extending them to handle traversing nested submanifolds and models embedded in a common model.
Jane: They use this construction to define a target measure which has two parts: one part governed by the standard d-dimensional Lebesgue measure in the ambient space, and another part governed by a density on an embedded Riemannian manifold.
Lu: The core idea is that when the process hits that manifold, it deterministically sticks to it, and then runs a separate Markov process on that lower dimensional space.
Meng: So what's the payoff for deployment? They claim this integration retains the non-reversible and ballistic nature of the component PDMP dynamics, which is important because we don't want to lose those properties.
Lalam: They prove that this resulting process leaves the intended mixed-dimensional target invariant, which means it stays on track with what we want to sample from.
The paper's improvements: Tom: I noticed they developed a specific algorithm, Algorithm two, which involves running the standard process until intersection, then sampling a new velocity using a sticking kernel and starting a manifold PDMP on that lower dimensional space.
Jane: They also deal with the unsticking part with an unsticking kernel, but this is where things get interesting for efficiency. They derive an unsticking rate formula, lambda(x) = mu v perp sigma omega one minus omega times the product of densities <ref:2611.17008#pg1>.
Lu: That unsticking rate formula is key because it shows the unsticking rate doesn't depend on the negative log-likelihood, which means we can compute it only through priors.
Meng: That's huge for scaling because it means we don't need to re-evaluate the likelihood every time a move occurs, which makes sense from a production standpoint.
Lalam: They showed that by replacing that exponential unsticking rate with a velocity-averaged one, lambda RB = c p to the two over two, the position marginal distribution is preserved.
Conclusion: Tom: So to wrap up, this paper gives us a rejection-free way to move between different models while keeping the underlying dynamics intact and not needing extra likelihood evaluations.
Jane: It allows us to sample from mixed dimensional distributions by combining continuous parameter uncertainty with discrete model uncertainty in a principled way using the sticky manifold construction.
Lu: The blow-up construction provides a geometric solution for high codimensions by resolving singular sets into codimension-one sticky boundaries, which makes the design choices for component dynamics and unsticking kernels explicit subject to velocity compatibility conditions.
Meng: For practical use, this method is agnostic to the choice of within-model dynamics and works for both linear and non-linear submanifolds, which simplifies things immensely when we're trying to deploy this on real traffic.
Lalam: This sticky manifold framework offers trans-dimensional moves without requiring additional likelihood evaluations, unlike many reversible jump MCMC algorithms, and they demonstrated it in several Bayesian model averaging examples.
Hirofumi Shiba, Luke Hardcastle, Kengo Kamatani
Institute of Statistical Mathematics, Tokyo, Japan · Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan · MRC Biostatistics Unit, University of Cambridge
stat.CO, stat.ME
Submitted: 2026-11-12
Updated: 2026-11-12
Comments: 55 pages, 18 figures, 17 tables
Code: https://github.com/162348/sticky-manifold-pdmp
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 100/100
The gist: Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent…
Terminology
Summary
Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent, leading to posterior distributions that combine continuous parameter uncertainty with discrete model uncertainty. p. 0 Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian
Model Averaging
Hirofumi Shiba1,2, Luke Hardcastle3 and Kengo Kamatani1,2
1
Institute of Statistical Mathematics, Tokyo, Japan
2Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan
3MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
November 13, 2026
Abstract
Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty
that Bayesian models can represent. The resulting posterior distributions combine continuous parameter
uncertainty with discrete model uncertainty. Efficient movement between these submodels, however, can
be difficult to design. We propose the sticky manifold construction as a principled, modular framework
for combining piecewise deterministic Markov processes (PDMPs) across different submodels. We propose the sticky manifold construction as a principled, modular framework
for combining piecewise deterministic Markov processes (PDMPs) across different submodels. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible
and ballistic nature of the component PDMP dynamics. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible
and ballistic nature of the component PDMP dynamics. We prove that the resulting process leaves
the intended mixed-dimensional target invariant.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The core concept involves a target measure pi(dx) proportional to (-U(x)) pi Rd(x) dx + (1 - omega) pi M(x) H d-1 M(dx), where pi Rd is a density with respect to the d-dimensional Lebesgue measure (dx) and pi M is a density on an embedded Riemannian manifold M with respect to the normalized Hausdorff measure (H d-1 M). p. 1 1 Our contributions
In its simplest form, we consider target measures of the form
π(dx) ∝ exp(−U(x))
ωπRd (x) dx + (1 − ω)πM(x)H
d−1
M (dx)
, (1. 1)
where πRd is a density with respect to the d-dimensional Lebesgue measure (denoted dx) and πM is a density
on the embedded Riemannian manifold M with respect to the normalised Hausdorff measure (denoted H
d−1
M ). Our primary contribution is to show that the following process, that we term the sticky manifold piecewise
deterministic Markov process (SM-PDMP) and illustrate in Figure 1, is π-invariant:
(1) In the ambient space, the process is given by a standard Euclidean PDMP with exp(−U(x))πRd (x) dx
as the marginal invariant measure for x. (2) When the ambient process intersects M, it deterministically sticks to the manifold, M.Where the paper says it
Where the paper says it
The SM-PDMP algorithm (Algorithm 2) involves a sequence: running a standard PDMP until intersection with M, sampling a new velocity via a sticking kernel Q stick, starting a manifold PDMP on M from the intersection point, parallel-transporting the normal component of the velocity, and finally recovering the ambient velocity via an unsticking kernel Q unstick. p. 7 Given a PDMP sampler with
position marginal e
−U(x)πRd (x) dx and a manifold PDMP sampler with e
−U(x)πM(x)H
d−1
M (dx), the sticky
manifold PDMP algorithm can be summarised as follows:
Algorithm 2 Sticky manifold PDMP. 1: Initialise z0 = (x0, v0) ∈ R
d × V
2: while t < T do
3: Run a standard PDMP until σ = inf{s > t | xs ∈ M}
4: Sample a new velocity vσ ∼ Qstick((xσ−, vσ−), −)
5: Start a manifold PDMP in M from (xσ, v ∥
σ
) until τ , the first Poisson arrival with rate Λ
6: Parallel-transport the normal component by setting v
⊥
τ− ← ⟨vσ, n(xσ)⟩n(xτ−)
7: Recover the ambient velocity by sampling vτ ∼ Qunstick((xτ−, vτ−), −)
8: end while
In the remainder of this subsection, we list examples of kernels Qstick, Qunstick in a few important situations. 3) from the remaining algorithmic choices of specific dynamics and
unsticking kernels (Sections 2. Our invariance results centre on Theorem 3. Our invariance results centre on Theorem 3. 5, whose proof rests
on a flow-wise Gauss–Green formula that provides a common basis for further generalisations. Our full algorithm (Algorithm 2)
further allows stochastic transformation upon sticking and unsticking and applies beyond the target (1. 1) to
the cases with a nested sequence of manifolds, in which the sampler can iteratively stick to the adjacent lower
dimensional submanifolds, and cases with multiple manifolds. 2), who consider coordinate-aligned linear subspaces. When combined with the Rao–Blackwellised
unsticking rate (Corollary 3. 2), who consider coordinate-aligned linear subspaces. When combined with the Rao–Blackwellised
unsticking rate (Corollary 3. 2), who consider coordinate-aligned linear subspaces. When combined with the Rao–Blackwellised
unsticking rate (Corollary 3. , 2020) in both the ambient
space and the submanifold, our method produces a single continuous non-reversible trajectory traversing the
two components, as shown in Figure 2 (a). For comparison, panel (c) shows the standard reversible-jump
MCMC with a random walk proposal, and (b) a simple PDMP analogue where the same trans-dimensional
jumps are triggered at a constant Poisson rate λ > 0 within the PDMP dynamics; see also Hardcastle et al. These
targets are common in Bayesian variable selection with spike and slab priors where the prior is constructed
coordinate-wise as a mixture of a Dirac mass at zero and a continuous density (Mitchell and Beauchamp,
1988). PDMPs with tractable deterministic dynamics can target these measures by deterministically sticking
to the spike when the process intersects it, and then unsticking after a random time (Chevallier et al.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
The sticky manifold framework offers trans-dimensional moves without requiring additional likelihood evaluations, unlike many reversible jump MCMC algorithms. p. 1 We refer to these dynamics throughout as sticky PDMPs following the terminology
of Bierkens et al. Under this construction trans-dimensional moves involve no computation of the
likelihood and require no additional tuning parameters, unlike most reversible jump MCMC algorithms. We also develop a manifold PDMP algorithm based
on coordinate charts to provide the within-submodel dynamics and a blow-up procedure that resolves
singular sets into codimension-one sticky boundaries. This geometric construction makes the remaining
design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily
verifiable velocity compatibility conditions. For comparison, panel (c) shows the standard reversible-jump
MCMC with a random walk proposal, and (b) a simple PDMP analogue where the same trans-dimensional
jumps are triggered at a constant Poisson rate λ > 0 within the PDMP dynamics; see also Hardcastle et al. As we demonstrate with a variety of examples in Section 4, the main strength of our construction
is that the same principle extends beyond this linear example, regardless of the geometry of the submanifold
or the particular choice of within-model PDMP dynamics.Where the paper says it
Where the paper says it
Where the paper says it
The authors demonstrate the algorithm in several Bayesian model averaging examples, including quantum state tomography, rank in Bayesian principal component analysis (Bayesian PCA), and parametric families in survival extrapolation. p. 0 This geometric construction makes the remaining
design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily
verifiable velocity compatibility conditions. Finally, we demonstrate the sticky manifold algorithms in
Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum
state tomography, rank in Bayesian principal component analysis, and parametric families in survival
extrapolation. Finally, we demonstrate the sticky manifold algorithms in
Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum
state tomography, rank in Bayesian principal component analysis, and parametric families in survival
extrapolation. Keywords: Trans-dimensional MCMC, mixed-dimensional distributions, Rejection-free sampling, geometric
MCMC, Rao–Blackwellization, Manifold sampling
1 Introduction
Statistical models are commonly defined such that the number of model parameters is unknown. Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian
Model Averaging
Hirofumi Shiba1,2, Luke Hardcastle3 and Kengo Kamatani1,2
1
Institute of Statistical Mathematics, Tokyo, Japan
2Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan
3MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
November 13, 2026
Abstract
Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty
that Bayesian models can represent. The resulting posterior distributions combine continuous parameter
uncertainty with discrete model uncertainty.Where the paper says it
Where the paper says it
Where the paper says it
In comparison to Reversible Jump Markov Chain (RJ-FECMC) and Reversible Jump Hamiltonian Monte Carlo (RJ-HMC), SM-FECMC demonstrates a computational advantage due to its rejection-free nature, achieving smaller Root Mean Square Errors (RMSEs) under both process time and evaluation budget measures. p. 0 In the
Bayesian paradigm, with a prior on all unknown quantities, Markov Chain Monte Carlo (MCMC) samplers
are then required to sample across several submodels. This is most commonly achieved using the celebrated
reversible jump MCMC algorithm (Green, 1995, 2003). These
targets are common in Bayesian variable selection with spike and slab priors where the prior is constructed
coordinate-wise as a mixture of a Dirac mass at zero and a continuous density (Mitchell and Beauchamp,
1988). PDMPs with tractable deterministic dynamics can target these measures by deterministically sticking
to the spike when the process intersects it, and then unsticking after a random time (Chevallier et al.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
1
No additional likelihood evaluations are required for trans-dimensional moves, which allows for rejection-free sampling across different submodels. The sticky manifold construction retains the non-reversible and ballistic nature of the component PDMP dynamics.
2
The system can sample from mixed dimensional distributions by combining continuous parameter uncertainty with discrete model uncertainty. This is achieved by assigning a positive prior probability to lower dimensional submodels.
3
The algorithm can handle nested sequences of manifolds and multiple manifolds through iterative sticking to adjacent lower dimensional submanifolds, or by using the blow-up construction to resolve singular sets into codimension-one sticky boundaries.
4
The sampler can be adapted for various target measures, including those defined on Riemannian manifolds based on their coordinate charts, providing a largely plug-and-play route to manifold sampling.
Abstract
Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent. The resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty. Efficient movement between these submodels, however, can be difficult to design. We propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible and ballistic nature of the component PDMP dynamics. We prove that the resulting process leaves the intended mixed-dimensional target invariant. We also develop a manifold PDMP algorithm based on coordinate charts to provide the within-submodel dynamics and a blow-up procedure that resolves singular sets into codimension-one sticky boundaries. This geometric construction makes the remaining design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily verifiable velocity compatibility conditions. Finally, we demonstrate the sticky manifold algorithms in Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum state tomography, rank in Bayesian principal component analysis, and parametric families in survival extrapolation.
Sources
- Large sample scaling analysis of the Zig-Zag algorithm for Bayesian inference
- Methods and applications of PDMP samplers with boundary conditions
- Smoothing Out Sticking Points: Sampling from Discrete-Continuous Mixtures with Dynamical Monte Carlo by Mapping Discrete Mass into a Latent Universe
- Diffusion piecewise exponential models for survival extrapolation using Piecewise Deterministic Monte Carlo
- Some aspects of robustness in modern Markov Chain Monte Carlo
- Diffusive Scaling Limits of Forward Event-Chain Monte Carlo: Provably Efficient Exploration with Partial Refreshment
Related papers
- A perturbation theory for Feynman-Kac models and their particle interpretations
- Fast Functional Clustering via Closed-Form Covariance Updates
- A fast non-reversible sampler for Bayesian mixture models
- Inventory and Assortment Optimization under General Choice: A Unified Approximation Framework
- Least squares variational inference
- A Computable Measure of Suboptimality for Entropy-Regularised Variational Objectives