Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging

arXiv:2611.17008 · stat.CO, stat.ME · Submitted 2026-11-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging".

Jane: Here's the gist of the paper. Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent, leading to posterior distributions that combine continuous parameter uncertainty with discrete model uncertainty.

Tom: The authors propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels.

Jane: This integration is rejection-free and requires no additional likelihood evaluations while retaining the non-reversible and ballistic nature of the component PDMP dynamics.

Tom: They prove that this resulting process leaves the intended mixed-dimensional target invariant.

Title and authors: Tom: So we're looking at this paper called "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging" and I'm thinking about the title itself, it sounds pretty heavy. It suggests combining these different types of uncertainty into one process.

Jane: Exactly, and what it means for us is that we can assign a positive prior probability to lower dimensional submodels, which enriches the range of uncertainty that Bayesian models can represent.

Lu: That enrichment means the resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty in a way that was previously difficult to handle efficiently.

Meng: But the main challenge is figuring out how to move between these different submodels without having to do all those extra likelihood evaluations we usually have to run.

Lalam: The paper proposes the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes across different submodels in a rejection-free way.

The paper's summary: Tom: Let's unpack that framework a bit more. Basically, they are taking these piecewise deterministic Markov processes, or PDMPs, and extending them to handle traversing nested submanifolds and models embedded in a common model.

Jane: They use this construction to define a target measure which has two parts: one part governed by the standard d-dimensional Lebesgue measure in the ambient space, and another part governed by a density on an embedded Riemannian manifold.

Lu: The core idea is that when the process hits that manifold, it deterministically sticks to it, and then runs a separate Markov process on that lower dimensional space.

Meng: So what's the payoff for deployment? They claim this integration retains the non-reversible and ballistic nature of the component PDMP dynamics, which is important because we don't want to lose those properties.

Lalam: They prove that this resulting process leaves the intended mixed-dimensional target invariant, which means it stays on track with what we want to sample from.

The paper's improvements: Tom: I noticed they developed a specific algorithm, Algorithm two, which involves running the standard process until intersection, then sampling a new velocity using a sticking kernel and starting a manifold PDMP on that lower dimensional space.

Jane: They also deal with the unsticking part with an unsticking kernel, but this is where things get interesting for efficiency. They derive an unsticking rate formula, lambda(x) = mu v perp sigma omega one minus omega times the product of densities <ref:2611.17008#pg1>.

Lu: That unsticking rate formula is key because it shows the unsticking rate doesn't depend on the negative log-likelihood, which means we can compute it only through priors.

Meng: That's huge for scaling because it means we don't need to re-evaluate the likelihood every time a move occurs, which makes sense from a production standpoint.

Lalam: They showed that by replacing that exponential unsticking rate with a velocity-averaged one, lambda RB = c p to the two over two, the position marginal distribution is preserved.

Conclusion: Tom: So to wrap up, this paper gives us a rejection-free way to move between different models while keeping the underlying dynamics intact and not needing extra likelihood evaluations.

Jane: It allows us to sample from mixed dimensional distributions by combining continuous parameter uncertainty with discrete model uncertainty in a principled way using the sticky manifold construction.

Lu: The blow-up construction provides a geometric solution for high codimensions by resolving singular sets into codimension-one sticky boundaries, which makes the design choices for component dynamics and unsticking kernels explicit subject to velocity compatibility conditions.

Meng: For practical use, this method is agnostic to the choice of within-model dynamics and works for both linear and non-linear submanifolds, which simplifies things immensely when we're trying to deploy this on real traffic.

Lalam: This sticky manifold framework offers trans-dimensional moves without requiring additional likelihood evaluations, unlike many reversible jump MCMC algorithms, and they demonstrated it in several Bayesian model averaging examples.

Hirofumi Shiba, Luke Hardcastle, Kengo Kamatani

Institute of Statistical Mathematics, Tokyo, Japan · Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan · MRC Biostatistics Unit, University of Cambridge

stat.CO, stat.ME

Submitted: 2026-11-12

Updated: 2026-11-12

Comments: 55 pages, 18 figures, 17 tables

Code: https://github.com/162348/sticky-manifold-pdmp

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 100/100

The gist: Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent…

Terminology

Summary

Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent, leading to posterior distributions that combine continuous parameter uncertainty with discrete model uncertainty. p. 0

Where the paper says it

Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging Hirofumi Shiba1,2, Luke Hardcastle3 and Kengo Kamatani1,2 1 Institute of Statistical Mathematics, Tokyo, Japan 2Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan 3MRC Biostatistics Unit, University of Cambridge, Cambridge, UK November 13, 2026 Abstract Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent. The resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty.

The authors propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels. p. 0
Where the paper says it

Efficient movement between these submodels, however, can be difficult to design. We propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels.

This integration is rejection-free and requires no additional likelihood evaluations while retaining the non-reversible and ballistic nature of the component PDMP dynamics. p. 0
Where the paper says it

We propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible and ballistic nature of the component PDMP dynamics.

They prove that this resulting process leaves the intended mixed-dimensional target invariant. p. 0
Where the paper says it

The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible and ballistic nature of the component PDMP dynamics. We prove that the resulting process leaves the intended mixed-dimensional target invariant.

The core concept involves a target measure pi(dx) proportional to (-U(x)) pi Rd(x) dx + (1 - omega) pi M(x) H d-1 M(dx), where pi Rd is a density with respect to the d-dimensional Lebesgue measure (dx) and pi M is a density on an embedded Riemannian manifold M with respect to the normalized Hausdorff measure (H d-1 M). p. 1

Where the paper says it

1 Our contributions In its simplest form, we consider target measures of the form π(dx) ∝ exp(−U(x)) ωπRd (x) dx + (1 − ω)πM(x)H d−1 M (dx) , (1. 1) where πRd is a density with respect to the d-dimensional Lebesgue measure (denoted dx) and πM is a density on the embedded Riemannian manifold M with respect to the normalised Hausdorff measure (denoted H d−1 M ).

The sticky manifold piecewise deterministic Markov process (SM-PDMP) is shown to be pi-invariant through three steps: (1) running a standard Euclidean PDMP in the ambient space governed by (-U(x)) pi Rd(x) dx; (2) deterministically sticking to the manifold M upon intersection; and (3) running a manifold PDMP on M governed by (-U(x)) pi M(x) H d-1 M(dx). p. 1
Where the paper says it

Our primary contribution is to show that the following process, that we term the sticky manifold piecewise deterministic Markov process (SM-PDMP) and illustrate in Figure 1, is π-invariant: (1) In the ambient space, the process is given by a standard Euclidean PDMP with exp(−U(x))πRd (x) dx as the marginal invariant measure for x. (2) When the ambient process intersects M, it deterministically sticks to the manifold, M.

The SM-PDMP algorithm (Algorithm 2) involves a sequence: running a standard PDMP until intersection with M, sampling a new velocity via a sticking kernel Q stick, starting a manifold PDMP on M from the intersection point, parallel-transporting the normal component of the velocity, and finally recovering the ambient velocity via an unsticking kernel Q unstick. p. 7

Where the paper says it

Given a PDMP sampler with position marginal e −U(x)πRd (x) dx and a manifold PDMP sampler with e −U(x)πM(x)H d−1 M (dx), the sticky manifold PDMP algorithm can be summarised as follows: Algorithm 2 Sticky manifold PDMP. 1: Initialise z0 = (x0, v0) ∈ R d × V 2: while t < T do 3: Run a standard PDMP until σ = inf{s > t | xs ∈ M} 4: Sample a new velocity vσ ∼ Qstick((xσ−, vσ−), −) 5: Start a manifold PDMP in M from (xσ, v ∥ σ ) until τ , the first Poisson arrival with rate Λ 6: Parallel-transport the normal component by setting v ⊥ τ− ← ⟨vσ, n(xσ)⟩n(xτ−) 7: Recover the ambient velocity by sampling vτ ∼ Qunstick((xτ−, vτ−), −) 8: end while In the remainder of this subsection, we list examples of kernels Qstick, Qunstick in a few important situations.

The invariance is established in Theorem 3. p. 2
Where the paper says it

3) from the remaining algorithmic choices of specific dynamics and unsticking kernels (Sections 2. Our invariance results centre on Theorem 3.

5, relying on flow-wise Gauss–Green formula and a flux balance condition (3. p. 2
Where the paper says it

Our invariance results centre on Theorem 3. 5, whose proof rests on a flow-wise Gauss–Green formula that provides a common basis for further generalisations.

4), which dictates the unsticking rate lambda(zeta) = omega 1 - omega d(J inQ stick) over d pi F(zeta) = eta, d r(xi) omega (1 - omega pi Rd(xi) pi M(xi)). p. 2
Where the paper says it

Our full algorithm (Algorithm 2) further allows stochastic transformation upon sticking and unsticking and applies beyond the target (1. 1) to the cases with a nested sequence of manifolds, in which the sampler can iteratively stick to the adjacent lower dimensional submanifolds, and cases with multiple manifolds.

Corollary 3. p. 8
Where the paper says it

2), who consider coordinate-aligned linear subspaces. When combined with the Rao–Blackwellised unsticking rate (Corollary 3.

p. 8
Where the paper says it

2), who consider coordinate-aligned linear subspaces. When combined with the Rao–Blackwellised unsticking rate (Corollary 3.

p. 8
Where the paper says it

2), who consider coordinate-aligned linear subspaces. When combined with the Rao–Blackwellised unsticking rate (Corollary 3.

8 shows that replacing the exponential unsticking rate (1. p. 3
Where the paper says it

, 2020) in both the ambient space and the submanifold, our method produces a single continuous non-reversible trajectory traversing the two components, as shown in Figure 2 (a). For comparison, panel (c) shows the standard reversible-jump MCMC with a random walk proposal, and (b) a simple PDMP analogue where the same trans-dimensional jumps are triggered at a constant Poisson rate λ > 0 within the PDMP dynamics; see also Hardcastle et al.

2) with a velocity-averaged one lambda RB = c p 2/2 leaves the position marginal invariant, yielding a reweighted velocity distribution nu RB F,x(dv) = p over pi 2|v| nu F x(dv). Corollary 3.9 further shows that if c is constant, replacing the exponential unsticking time with the deterministic residence time T = c-1p/2 preserves the position marginal distribution. p. 1
Where the paper says it

These targets are common in Bayesian variable selection with spike and slab priors where the prior is constructed coordinate-wise as a mixture of a Dirac mass at zero and a continuous density (Mitchell and Beauchamp, 1988). PDMPs with tractable deterministic dynamics can target these measures by deterministically sticking to the spike when the process intersects it, and then unsticking after a random time (Chevallier et al.

Corollary 3.10 handles one-sided cases by ensuring half of the rate lambda(x) = |v| squared omega (1 - omega pi Rd(x) pi M(x)) is used to maintain the correct position marginal under specific assumptions on ambient dynamics and velocity distributions.

The sticky manifold framework offers trans-dimensional moves without requiring additional likelihood evaluations, unlike many reversible jump MCMC algorithms. p. 1

Where the paper says it

We refer to these dynamics throughout as sticky PDMPs following the terminology of Bierkens et al. Under this construction trans-dimensional moves involve no computation of the likelihood and require no additional tuning parameters, unlike most reversible jump MCMC algorithms.

The blow-up construction provides a geometric solution for high codimensions by resolving singular sets into codimension-one sticky boundaries, making design choices for component dynamics and unsticking kernels explicit subject to velocity compatibility conditions. p. 0
Where the paper says it

We also develop a manifold PDMP algorithm based on coordinate charts to provide the within-submodel dynamics and a blow-up procedure that resolves singular sets into codimension-one sticky boundaries. This geometric construction makes the remaining design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily verifiable velocity compatibility conditions.

The method is agnostic to the choice of within-model dynamics and applicable to both linear and non-linear submanifolds. p. 3
Where the paper says it

For comparison, panel (c) shows the standard reversible-jump MCMC with a random walk proposal, and (b) a simple PDMP analogue where the same trans-dimensional jumps are triggered at a constant Poisson rate λ > 0 within the PDMP dynamics; see also Hardcastle et al. As we demonstrate with a variety of examples in Section 4, the main strength of our construction is that the same principle extends beyond this linear example, regardless of the geometry of the submanifold or the particular choice of within-model PDMP dynamics.

The authors demonstrate the algorithm in several Bayesian model averaging examples, including quantum state tomography, rank in Bayesian principal component analysis (Bayesian PCA), and parametric families in survival extrapolation. p. 0

Where the paper says it

This geometric construction makes the remaining design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily verifiable velocity compatibility conditions. Finally, we demonstrate the sticky manifold algorithms in Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum state tomography, rank in Bayesian principal component analysis, and parametric families in survival extrapolation.

They propose a novel computational approach using the SM-PDMP sampler with the generalized gamma family of distributions for model averaging in survival extrapolation, which avoids designing multiple reversible jump steps. p. 0
Where the paper says it

Finally, we demonstrate the sticky manifold algorithms in Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum state tomography, rank in Bayesian principal component analysis, and parametric families in survival extrapolation. Keywords: Trans-dimensional MCMC, mixed-dimensional distributions, Rejection-free sampling, geometric MCMC, Rao–Blackwellization, Manifold sampling 1 Introduction Statistical models are commonly defined such that the number of model parameters is unknown.

This approach favors the ammag model under uniform prior weights, showing posterior probabilities of 55%, 54%, and 46% for three models. p. 0
Where the paper says it

Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging Hirofumi Shiba1,2, Luke Hardcastle3 and Kengo Kamatani1,2 1 Institute of Statistical Mathematics, Tokyo, Japan 2Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan 3MRC Biostatistics Unit, University of Cambridge, Cambridge, UK November 13, 2026 Abstract Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent. The resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty.

In comparison to Reversible Jump Markov Chain (RJ-FECMC) and Reversible Jump Hamiltonian Monte Carlo (RJ-HMC), SM-FECMC demonstrates a computational advantage due to its rejection-free nature, achieving smaller Root Mean Square Errors (RMSEs) under both process time and evaluation budget measures. p. 0

Where the paper says it

In the Bayesian paradigm, with a prior on all unknown quantities, Markov Chain Monte Carlo (MCMC) samplers are then required to sample across several submodels. This is most commonly achieved using the celebrated reversible jump MCMC algorithm (Green, 1995, 2003).

The robustness of PDMP dynamics to multiscale structure is highlighted as a key advantage for trans-dimensional sampling where posterior geometry may vary across strata, contrasting with the need for stratumwise tuning in RJ-HMC. Experiments implemented in Julia on an Apple M5 processor confirm that SM-FECMC achieves smaller RMSEs than RJ-FECMC under both process time and evaluation budget measures. p. 1
Where the paper says it

These targets are common in Bayesian variable selection with spike and slab priors where the prior is constructed coordinate-wise as a mixture of a Dirac mass at zero and a continuous density (Mitchell and Beauchamp, 1988). PDMPs with tractable deterministic dynamics can target these measures by deterministically sticking to the spike when the process intersects it, and then unsticking after a random time (Chevallier et al.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

1

No additional likelihood evaluations are required for trans-dimensional moves, which allows for rejection-free sampling across different submodels. The sticky manifold construction retains the non-reversible and ballistic nature of the component PDMP dynamics.

2

The system can sample from mixed dimensional distributions by combining continuous parameter uncertainty with discrete model uncertainty. This is achieved by assigning a positive prior probability to lower dimensional submodels.

3

The algorithm can handle nested sequences of manifolds and multiple manifolds through iterative sticking to adjacent lower dimensional submanifolds, or by using the blow-up construction to resolve singular sets into codimension-one sticky boundaries.

4

The sampler can be adapted for various target measures, including those defined on Riemannian manifolds based on their coordinate charts, providing a largely plug-and-play route to manifold sampling.

Abstract

Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent. The resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty. Efficient movement between these submodels, however, can be difficult to design. We propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible and ballistic nature of the component PDMP dynamics. We prove that the resulting process leaves the intended mixed-dimensional target invariant. We also develop a manifold PDMP algorithm based on coordinate charts to provide the within-submodel dynamics and a blow-up procedure that resolves singular sets into codimension-one sticky boundaries. This geometric construction makes the remaining design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily verifiable velocity compatibility conditions. Finally, we demonstrate the sticky manifold algorithms in Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum state tomography, rank in Bayesian principal component analysis, and parametric families in survival extrapolation.

Sources

Related papers