Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation
math.ST, stat.ML, stat.TH
Submitted: 2026-10-30
Updated: 2026-10-30
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study finite sample guarantees for transport based mutual information estimation.
Terminology
Abstract
We study finite sample guarantees for transport based mutual information estimation. Mutual information can be estimated by first learning the conditional and marginal densities and then evaluating their log-density ratio by Monte Carlo. Given training samples, we use triangular transport maps to learn these densities and then compute the mutual information estimate on an independent batch of evaluation samples. For conditional density estimation using triangular transport maps, we establish non-asymptotic guarantees over realizable Hölder classes by viewing the conditional density as an ordinary density with respect to the product of the law of the conditioning variable and Lebesgue measure. This allows standard density estimation techniques to be applied directly to the conditional density estimation. These results yield finite sample guarantees for the resulting mutual information estimator and characterize how a fixed sample budget should be allocated between training and evaluation in order to minimize the mean squared error. Numerical experiments support the predicted training and evaluation error rates and illustrate the resulting sample allocation tradeoff.
Sources
- Subspace accelerated measure transport methods for fast and scalable sequential experimental design, with application to photoacoustic imaging
- Variational Bayesian Optimal Experimental Design with Normalizing Flows
- Global rates of convergence in log-concave density estimation
- Tractable Optimal Experimental Design using Transport Maps
- On minimax density estimation via measure transport
Related papers
- Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators
- Nonparametric Regression on Fixed-Cardinality Subsets:Minimax Risk and Random-Design Effects
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models
- Semiparametric Efficient Inference under Non-Informative Complex Survey Designs
- Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel