Sufficiently Reduced Distributional Regression
stat.ME, cs.LG, stat.ML
Submitted: 2026-09-24
Updated: 2026-09-24
Code: https://github.com/liutiang/SRDR
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: We propose Sufficiently Reduced Distributional Regression (SRDR), a generative method that combines conditional distribution estimation with nonlinear sufficient dimension reduction (SDR).
Terminology
Abstract
We propose Sufficiently Reduced Distributional Regression (SRDR), a generative method that combines conditional distribution estimation with nonlinear sufficient dimension reduction (SDR). It builds on a characterization of sufficiency through strictly proper scoring rules: a dimension reduction is sufficient if and only if predicting the response from the reduced covariates incurs no loss in expected score relative to the full covariates. Sufficient dimension reduction thus becomes a risk minimization problem. SRDR jointly trains a dimension reduction map and a generative prediction model by minimizing the energy score, which can be estimated by sampling without density evaluation or adversarial training. The framework extends to multi-environment data and to classification. We prove that the estimated conditional distributions converge in energy distance to the true ones, which implies that the learned representation is asymptotically sufficient. In simulations and applications to CT slice localization, superconductivity, and digit classification, SRDR recovers low-dimensional sufficient structure and matches or outperforms state-of-the-art nonlinear SDR methods in representation quality and predictive performance.
Sources
- Error Analysis of Neural-Network-Based Engression
- Transfer Learning through Enhanced Sufficient Representation: Enriching Source Domain Knowledge with Target Data
- Conditional Stochastic Interpolation for Generative Learning
- Theoretical Analysis of Engression and Reverse Markov Engression
- Deep Transfer Learning: Model Framework and Error Analysis
- Auto-Encoding Variational Bayes
- GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution
- Conditional Generative Adversarial Nets
- Distributional Principal Autoencoders
- Belted Engression: Sufficient Dimension Reduction for Generative Distributional Regression
- On Conditional Stochastic Interpolation for Generative Nonlinear Sufficient Dimension Reduction
Related papers
- Doubly robust inference via calibration
- Bayesian Empirical Bayes: Simultaneous Inference from Probabilistic Symmetries
- Optimal Watermark Localization in Mixed-Source Large Language Model Texts
- Small area learning
- Flexible Nonparametric Inference for Causal Effects under the Front-Door Model
- A Greedy PDE Router for Blending Neural Operators and Classical Methods