Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging

summary

The gist

Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent…

This episode discusses

The paper

Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging · Read on arXiv

Hirofumi Shiba, Luke Hardcastle, Kengo Kamatani

Institute of Statistical Mathematics, Tokyo, Japan · Graduate University for Advanced Studies (SOKENDAI), Tokyo, Japan · MRC Biostatistics Unit, University of Cambridge

Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent. The resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty. Efficient movement between these submodels, however, can be difficult to design. We propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels. The integration is rejection-free, requires no additional likelihood evaluations, and retains the non-reversible and ballistic nature of the component PDMP dynamics. We prove that the resulting process leaves the intended mixed-dimensional target invariant. We also develop a manifold PDMP algorithm based on coordinate charts to provide the within-submodel dynamics and a blow-up procedure that resolves singular sets into codimension-one sticky boundaries. This geometric construction makes the remaining design choices explicit; component dynamics and unsticking kernels can be chosen freely, subject to easily verifiable velocity compatibility conditions. Finally, we demonstrate the sticky manifold algorithms in Bayesian model averaging examples, propagating uncertainty over pure versus mixed states in quantum state tomography, rank in Bayesian principal component analysis, and parametric families in survival extrapolation.

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging".

Jane: Here's the gist of the paper. Assigning positive prior probability to lower-dimensional submodels enriches the range of uncertainty that Bayesian models can represent, leading to posterior distributions that combine continuous parameter uncertainty with discrete model uncertainty.

Tom: The authors propose the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes (PDMPs) across different submodels.

Jane: This integration is rejection-free and requires no additional likelihood evaluations while retaining the non-reversible and ballistic nature of the component PDMP dynamics.

Tom: They prove that this resulting process leaves the intended mixed-dimensional target invariant.

Title and authors: Tom: So we're looking at this paper called "Sticky Manifold Piecewise Deterministic Monte Carlo for Bayesian Model Averaging" and I'm thinking about the title itself, it sounds pretty heavy. It suggests combining these different types of uncertainty into one process.

Jane: Exactly, and what it means for us is that we can assign a positive prior probability to lower dimensional submodels, which enriches the range of uncertainty that Bayesian models can represent.

Lu: That enrichment means the resulting posterior distributions combine continuous parameter uncertainty with discrete model uncertainty in a way that was previously difficult to handle efficiently.

Meng: But the main challenge is figuring out how to move between these different submodels without having to do all those extra likelihood evaluations we usually have to run.

Lalam: The paper proposes the sticky manifold construction as a principled, modular framework for combining piecewise deterministic Markov processes across different submodels in a rejection-free way.

The paper's summary: Tom: Let's unpack that framework a bit more. Basically, they are taking these piecewise deterministic Markov processes, or PDMPs, and extending them to handle traversing nested submanifolds and models embedded in a common model.

Jane: They use this construction to define a target measure which has two parts: one part governed by the standard d-dimensional Lebesgue measure in the ambient space, and another part governed by a density on an embedded Riemannian manifold.

Lu: The core idea is that when the process hits that manifold, it deterministically sticks to it, and then runs a separate Markov process on that lower dimensional space.

Meng: So what's the payoff for deployment? They claim this integration retains the non-reversible and ballistic nature of the component PDMP dynamics, which is important because we don't want to lose those properties.

Lalam: They prove that this resulting process leaves the intended mixed-dimensional target invariant, which means it stays on track with what we want to sample from.

The paper's improvements: Tom: I noticed they developed a specific algorithm, Algorithm two, which involves running the standard process until intersection, then sampling a new velocity using a sticking kernel and starting a manifold PDMP on that lower dimensional space.

Jane: They also deal with the unsticking part with an unsticking kernel, but this is where things get interesting for efficiency. They derive an unsticking rate formula, lambda(x) = mu v perp sigma omega one minus omega times the product of densities <ref:2611.17008#pg1>.

Lu: That unsticking rate formula is key because it shows the unsticking rate doesn't depend on the negative log-likelihood, which means we can compute it only through priors.

Meng: That's huge for scaling because it means we don't need to re-evaluate the likelihood every time a move occurs, which makes sense from a production standpoint.

Lalam: They showed that by replacing that exponential unsticking rate with a velocity-averaged one, lambda RB = c p to the two over two, the position marginal distribution is preserved.

Conclusion: Tom: So to wrap up, this paper gives us a rejection-free way to move between different models while keeping the underlying dynamics intact and not needing extra likelihood evaluations.

Jane: It allows us to sample from mixed dimensional distributions by combining continuous parameter uncertainty with discrete model uncertainty in a principled way using the sticky manifold construction.

Lu: The blow-up construction provides a geometric solution for high codimensions by resolving singular sets into codimension-one sticky boundaries, which makes the design choices for component dynamics and unsticking kernels explicit subject to velocity compatibility conditions.

Meng: For practical use, this method is agnostic to the choice of within-model dynamics and works for both linear and non-linear submanifolds, which simplifies things immensely when we're trying to deploy this on real traffic.

Lalam: This sticky manifold framework offers trans-dimensional moves without requiring additional likelihood evaluations, unlike many reversible jump MCMC algorithms, and they demonstrated it in several Bayesian model averaging examples.

More episodes

← Home