Stein's unbiased risk estimate and Hyv"arinen's score matching

arXiv:2502.20123 · math.ST, stat.ME, stat.ML, stat.TH · Submitted 2025-02-27 · Read on arXiv

math.ST, stat.ME, stat.ML, stat.TH

Submitted: 2025-02-27

Updated: 2026-10-13

Code: https://github.com/sulagna-ghosh/score-matching-empirical-bayes

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Generative Modeling — 2× in this paper · explained in 3 episodes
• Heteroscedasticity — 1× in this paper · explained in 2 episodes
• Regret — 1× in this paper · explained in 3 episodes

Sources

• Nonparametric Empirical Bayes Estimation on Heterogeneous Data
• A nonparametric regression alternative to empirical Bayes approaches to simultaneous estimation
• Generative Modeling with Denoising Auto-Encoders and Langevin Sampling
• A comment and erratum on "Excess Optimism: How Biased is the Apparent Error of an Estimator Tuned by SURE?"
• Empirical Bayes When Estimation Precision Predicts Parameters
• DDPM Score Matching and Distribution Learning
• From optimal score matching to optimal sampling
• Efficiently learning and sampling multimodal distributions with data-based initialization
• Optimal Shrinkage Estimation of Fixed Effects in Linear Panel Data Models
• Sharp regret bounds for empirical Bayes and compound decision problems
• Deep Energy Estimator Networks
• Optimistic Rates for Learning with a Smooth Loss
• Empirical Bayes, SURE and Sparse Normal Mean Models

Related papers

• Conformal Prediction for Dyadic Regression Under Complex Missingness
• Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel
• Bentkus-type asymptotic e-values
• High-Dimensional Asymptotics of Differentially Private PCA
• A proximal gradient algorithm for composite log-concave sampling
• KL Convergence Guarantees for Score diffusion models under minimal data assumptions