Leveraging tails for adaptation
Sergios Agapiou, Ismaël Castillo, Paul Egels
math.ST, stat.ML
Submitted: 2026-06-18
Comments: 59 pages, 3 figures, added missing reference
License: http://creativecommons.org/licenses/by/4.0/
The gist: We consider contraction of Bayesian posterior distributions in nonparametric settings where coefficients of a function over a basis or dictionary are given priors with p --exponential tails,
Terminology
Abstract
We consider contraction of Bayesian posterior distributions in nonparametric settings where coefficients of a function over a basis or dictionary are given priors with p --exponential tails, including Laplace tails (p=1) and heavier tails (p<1). It is shown that contraction rates improve as p decreases and that full adaptation to smoothness, up to logarithmic factors, is obtained in an appropriate p to 0 regime. As applications, we consider both series priors in white noise regression and shallow ReLU neural networks in random design regression. In particular, we show that overparametrised shallow ReLU networks can adapt to any regularity 0 beta 2. Through a simulation study, we show strong empirical agreement with the behavior predicted by our theory.
Sources
- Deep Gaussian Process Priors for Bayesian Inference in Nonlinear Inverse Problems
- Heavy-tailed and Horseshoe priors for regression and sparse Besov rates
- Dimension-Robust MCMC in Bayesian Inverse Problems
- On strong posterior contraction rates for Besov-Laplace priors in the white noise model
- Vecchia Gaussian Processes: on probabilistic and statistical properties
Related papers
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel
- Bentkus-type asymptotic e-values
- High-Dimensional Asymptotics of Differentially Private PCA
- A proximal gradient algorithm for composite log-concave sampling
- KL Convergence Guarantees for Score diffusion models under minimal data assumptions