PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation
stat.ML, cs.AI, cs.LG, stat.ME
Submitted: 2024-02-06
Updated: 2026-08-31
Comments: ICLR 2025; code: https://github.com/Ciela-Institute/PQM
Code: https://github.com/Ciela-Institute/PQM
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: We propose a likelihood-free method for comparing two distributions given samples from each, with the goal of assessing the quality of generative models.
Terminology
Abstract
We propose a likelihood-free method for comparing two distributions given samples from each, with the goal of assessing the quality of generative models. The proposed approach, PQMass, provides a statistically rigorous method for assessing the performance of a single generative model or the comparison of multiple competing models. PQMass divides the sample space into non-overlapping regions and applies chi-squared tests to the number of data samples that fall within each region, giving a p-value that measures the probability that the bin counts derived from two sets of samples are drawn from the same multinomial distribution. PQMass does not depend on assumptions regarding the density of the true distribution, nor does it rely on training or fitting any auxiliary models. We evaluate PQMass on data of various modalities and dimensions, demonstrating its effectiveness in assessing the quality, novelty, and diversity of generated samples. We further show that PQMass scales well to moderately high-dimensional data and thus obviates the need for feature extraction in practical applications.
Sources
- Event-horizon-scale Imaging of M87* under Different Assumptions via Deep Generative Image Priors
- Denoising Diffusion Delensing Delight: Reconstructing the Non-Gaussian CMB Lensing Potential with Diffusion Models
- Tackling the Problem of Distributional Shifts: Correcting Misspecified, High-Dimensional Data-Driven Priors for Inverse Problems
- Diffusion Models for Probabilistic Deconvolution of Galaxy Images
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing
- From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier