Transformed Samplers with Variance Reduction

Detailed Research Summary: Transformed Samplers with Variance Reduction (Combining MCMC Theory and Empirical Results)

This research addresses a critical challenge in Markov Chain Monte Carlo (MCMC) by extending exact solutions for control variates from simple reference densities to general, complex target distributions through the use of learned change of variables, such as normalizing flows. The core theoretical contribution establishes that Markov kernels and their Poisson solutions are invariant under any bijection (like a flow), allowing for the derivation of explicit control variates in the latent space.

Theoretical Framework: Exact Control Variates via Change of Variables

The paper fundamentally connects MCMC methods with change of variables by demonstrating that if a Markov kernel is transformed by a bijection T, its Poisson solutions are also transformed by T. This property allows researchers to take exact Poisson solutions, derived from a simple reference distribution rho and kernel Q (which possesses an exact solution), and transform them into control variates for any target distribution reached via the map T.

The construction hinges on three necessary properties of the latent sampler: (C1) for the reference kernel, (C2) for computing control variates, and (C3) ensuring the resulting solution accurately reflects the actual kernel run. Mathematically, Theorem 4.2 proves that under condition (A1), for every bijection T, the transformed kernel Q T is positive semidefinite if and only if its eigenvalues lambda j are non-negative for all j. Furthermore, it establishes that the functions G j = phi j T-1 form an orthonormal basis of the relevant function space, where Q T G j = lambda j G j. Crucially, this implies that the spectrum of the transformed kernel (Q T) does not depend on the transformation T, and its absolute spectral gap is determined solely by the spectral gap of the original reference kernel Q (specifically, 1 -).

This theoretical foundation guarantees that running samplers in this transformed latent space yields explicit, exact control variates. Moreover, Theorem 4.3 provides consistency results (mu to pi(F) almost surely) for various samplers—including SNIS, TGI-RWM (for Gaussian reference), TGI-MALA, and TESS—provided specific tail conditions (like E rho[w 2(1 + (F T) 2)] < infinity) are met.

Empirical Application and Diagnostic Tools

The paper moves beyond theory to demonstrate the practical utility of this construction by applying it to various samplers in the latent space. The dictionary of derived control variates is tailored to the reference distribution; for instance, Hermite polynomials are used when a Gaussian reference is employed. A key advantage highlighted is that these control variate coefficients can be computed from cheap reference draws rather than being fitted to the chain itself.

To guide practical implementation, the authors provide diagnostics based on draws from the flow. These include:

1. Importance Sampling Effective Sample Size (ESSIS): To assess estimator quality.

2. Largest Normalized Weight: A metric indicating which component or member of a multimodal target is dominating the sampling process, suggesting which specific chain member should be prioritized for control variate application (e.g., The largest importance weight of the flow tells which member to run).

Comparative Performance Analysis

The empirical results compare these novel estimators against state-of-the-art samplers and traditional control variates on both synthetic targets and real posterior distributions. The performance gains are highly dependent on the quality of the transport (the mapping T). Training dominates the computational cost when budgets are small, but where the flow creates a neck or seam between modes, the control variates yield minimal benefit. Conversely, for posteriors where mismatch is spread across many coordinates, standard HMC on the same flow remains superior. A significant finding is that control variate methods also prefer bounded weights in the tails—a condition often not enforced by standard training objectives—and a penalty in training can encourage this desired behavior.

Summary of Specific Sampler Insights (From Section B)

The empirical data, detailed across Tables S11, S12, and S13, provides granular insights into the practical trade-offs between different samplers:

• SNIS+CV: Often the best overall method but suffers from uneven performance gaps. It falls behind significantly on complex structures like the t mixture or sparse regression coefficients (where weights are heaviest).

• TGI-RWM+CV: Shows superior stability, exhibiting the smallest mean gap and spread among methods, making it a reliable choice. It is rarely the absolute best but consistently performs well across many test functions.

• TESS+CV: Nearly as stable as TGI-RWM, though it can fall significantly behind on highly multimodal targets where TGI-MALA struggles.

• Step Selection (TGI-RWM/TGI-MALA): The mechanism J for step selection (gamma J) is highly dependent on the target structure. For TGI-RWM, J selects steps near gamma=1 on accurate members but below 1 on coarse members (like the banana or funnel). For TGI-MALA, J selects steps well below 1 except for the accurate mixture. The ratio of ESS times VRF (R(gamma, gamma')) quantifies these differences across various target types and step selections.

**In conclusion, this work successfully bridges rigorous MCMC theory with practical variance reduction techniques by showing how learned change of variables enable the construction of exact control variates for general targets. The empirical evidence suggests that while the theoretical framework is powerful, its practical efficacy is maximized when combined with intelligent diagnostics (like importance weights) and a deep understanding of the transport quality.

1. Bold header: Control variates for MCMC on transformed targets. This allows for explicit control variates by transforming Markov kernels through a learned change of variables (a normalizing flow), yielding an estimator that is consistent under mild tail conditions on the map and target.

2. Bold header: Dictionary pruning based on spectral properties. The paper suggests keeping indices with largest positive scores vj (c 2j - s 2j/N) to minimize asymptotic variance, which can be implemented via structural pruning and magnitude pruning based on reference draws.

3. Bold header: Adaptive step size selection for TGI samplers. The system can maximize the objective function J(γ) = log(2γ−γ 2)+log a(γ)−log 1+V0 (1−a(γ)) to choose a step size that balances ESS × VRF, optimizing efficiency over mere acceptance rate or speed measures.

4. Bold header: Tail penalty integration for flow training. By adding the loss term E max[0, log w(ζi) − ¯l − c]2, the system can be trained to better handle heavy tails, as this acts only on weights that are too large, which make chains stick.

5. Bold header: Mode detection and sampler selection. The system can monitor diagnostics like ESSIS = (P i w(ζ i)) squared / P i w(ζ i) 2 to switch dynamically between estimators (SNIS, TGI-RWM, TESS) based on the largest normalized importance weight of the flow, avoiding performance degradation when the map leaves a neck or a seam between modes.

Markov chain Monte Carlo (MCMC) methods are the standard tool for computing expectations under complex probability distributions. Control variates reduce the variance of the resulting estimates, but a good control variate requires solving the Poisson equation of the sampler, which rarely admits a closed-form solution. Exact solutions are available when the sampler's kernel has a known spectral decomposition on a simple reference density. In our work, we extend these solutions to general targets through a learned change of variables. A bijection, such as a normalizing flow, is trained so that the target becomes close to the reference in a latent space, and we show that Markov kernels and their Poisson solutions are transformed by any bijection. Running such samplers in the latent space then yields explicit control variates, and the estimator is consistent under mild tail conditions on the map and target. Importance sampling (IS) from the flow is the limiting case of the same construction and the control variates apply to it as well. Experiments on synthetic targets and real posteriors compare the procedure against state-of-the-art samplers and control variates.