Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging
summary
The gist
Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent…
This episode discusses
- Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging · Paper Radio
- Large sample scaling analysis of the Zig-Zag algorithm for Bayesian inference · Paper Radio
- Methods and applications of PDMP samplers with boundary conditions
- Smoothing Out Sticking Points: Sampling from Discrete-Continuous Mixtures with Dynamical Monte Carlo by Mapping Discrete Mass into a Latent Universe
- Diffusion piecewise exponential models for survival extrapolation using Piecewise Deterministic Monte Carlo
- Some aspects of robustness in modern Markov Chain Monte Carlo
- Diffusive Scaling Limits of Forward Event-Chain Monte Carlo: Provably Efficient Exploration with Partial Refreshment
The paper
Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging · Read on arXiv
Hirofumi Shiba, Luke Hardcastle, Kengo Kamatani
Institute of Statistical Mathematics, Tokyo, Japan · Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan · MRC Biostatistics Unit, University of Cambridge
Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent. The resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty. Efficient movement between these submodels, however, can be difficult to design. We propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible and ballistic nature of the component PDMP dynamics. We prove that the resulting process leaves the intended mixed-dimensional target invariant. We also develop a manifold PDMP algorithm based on coordinate charts to provide the within-submodel dynamics and a blow-up procedure that resolves singular sets into codimension-one sticky boundaries. This geometric construction makes the remaining design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily verifiable velocity compatibility conditions. Finally, we demonstrate the sticky manifold algorithms in Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum state tomography, rank in Bayesian principal component analysis, and parametric families in survival extrapolation.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging".
Jane: Here's the gist of the paper. Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent, leading to posterior distributions that combine continuous parameter uncertainty with discrete model uncertainty.
Tom: The authors propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels.
Jane: This integration is rejection-free and requires no additional likelihood evaluations while retaining the non-reversible and ballistic nature of the component PDMP dynamics.
Tom: They prove that this resulting process leaves the intended mixed-dimensional target invariant.
Title and authors: Tom: So we're looking at this paper called "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging" and I'm thinking about the title itself, it sounds pretty heavy. It suggests combining these different types of uncertainty into one process.
Jane: Exactly, and what it means for us is that we can assign a positive prior probability to lower dimensional submodels, which enriches the range of uncertainty that Bayesian models can represent.
Lu: That enrichment means the resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty in a way that was previously difficult to handle efficiently.
Meng: But the main challenge is figuring out how to move between these different submodels without having to do all those extra likelihood evaluations we usually have to run.
Lalam: The paper proposes the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes across different submodels in a rejection-free way.
The paper's summary: Tom: Let's unpack that framework a bit more. Basically, they are taking these piecewise deterministic Markov processes, or PDMPs, and extending them to handle traversing nested submanifolds and models embedded in a common model.
Jane: They use this construction to define a target measure which has two parts: one part governed by the standard d-dimensional Lebesgue measure in the ambient space, and another part governed by a density on an embedded Riemannian manifold.
Lu: The core idea is that when the process hits that manifold, it deterministically sticks to it, and then runs a separate Markov process on that lower dimensional space.
Meng: So what's the payoff for deployment? They claim this integration retains the non-reversible and ballistic nature of the component PDMP dynamics, which is important because we don't want to lose those properties.
Lalam: They prove that this resulting process leaves the intended mixed-dimensional target invariant, which means it stays on track with what we want to sample from.
The paper's improvements: Tom: I noticed they developed a specific algorithm, Algorithm two, which involves running the standard process until intersection, then sampling a new velocity using a sticking kernel and starting a manifold PDMP on that lower dimensional space.
Jane: They also deal with the unsticking part with an unsticking kernel, but this is where things get interesting for efficiency. They derive an unsticking rate formula, lambda(x) = mu v perp sigma omega one minus omega times the product of densities <ref:2611.17008#pg1>.
Lu: That unsticking rate formula is key because it shows the unsticking rate doesn't depend on the negative log-likelihood, which means we can compute it only through priors.
Meng: That's huge for scaling because it means we don't need to re-evaluate the likelihood every time a move occurs, which makes sense from a production standpoint.
Lalam: They showed that by replacing that exponential unsticking rate with a velocity-averaged one, lambda RB = c p to the two over two, the position marginal distribution is preserved.
Conclusion: Tom: So to wrap up, this paper gives us a rejection-free way to move between different models while keeping the underlying dynamics intact and not needing extra likelihood evaluations.
Jane: It allows us to sample from mixed dimensional distributions by combining continuous parameter uncertainty with discrete model uncertainty in a principled way using the sticky manifold construction.
Lu: The blow-up construction provides a geometric solution for high codimensions by resolving singular sets into codimension-one sticky boundaries, which makes the design choices for component dynamics and unsticking kernels explicit subject to velocity compatibility conditions.
Meng: For practical use, this method is agnostic to the choice of within-model dynamics and works for both linear and non-linear submanifolds, which simplifies things immensely when we're trying to deploy this on real traffic.
Lalam: This sticky manifold framework offers trans-dimensional moves without requiring additional likelihood evaluations, unlike many reversible jump MCMC algorithms, and they demonstrated it in several Bayesian model averaging examples.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates