Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data".
Jane: Here's the gist of the paper. Convolutional Maximum Mean Discrepancy for Inference in Noisy Data introduces a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution using convolutional MMD (convMMD) to compare distributions after noise convolution The authors establish finite-sample deviation bounds unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing Leveraging these insights, they introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations, establishing its consistency and asymptotic normality using stochastic gradient descent.
Tom: Theoretical Foundation of the Metric The paper begins by reviewing the Maximum Mean Discrepancy (MMD) statistic, defining it as a measure of distance between two distributions through their embeddings in a reproducing kernel Hilbert space (RKHS).
Jane: A well-known direct computational form for MMD2 is given by MMD2(p, q) = EX,X′
k (X, X′): + EY,Y ′
k (Y, Y′): − 2EX,Y
k (X, Y): .
Tom: The authors then move to the challenge of noisy data by defining the convolution MMD as convMMD(p, q, m) ≡ MMD(p ∗ m, q ∗ m) = sup f∈H,∥f∥H≤1 Ep∗m(f(X)) − Eq∗m(f(Y)).
Paper summary: Tom: To recap what we've covered, this paper, "Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data," introduces a new way to do inference when your data has measurement error that varies depending on the location. The authors claim their core contribution is the convolutional MMD, or convMMD, which compares distributions after convolving them with the noise distribution.
Jane: They set out to show that this framework allows for distribution-free inference even when samples are contaminated by heteroscedastic noise from a known distribution. They establish that convMMD is a valid metric and they prove it's equivalent to a noiseless comparison using a smoothed kernel, which is essentially the ek kernel on page two of THIS PAPER <ref:2604.12022#pg1>.
Lu: The paper then develops an estimator called theta b N, defined as the minimizer of this semi-empirical convMMD objective function LN on page three of THIS PAPER <ref:2604.12022#pg1>. They rigorously prove its consistency and asymptotic normality under the conditions they set out.
Meng: What's important for us in a production setting is how scalable this estimation is. They show that even when fitting it to noisy data, the estimator converges almost surely to the true parameter star if you use samples of size N from the modified distribution q theta star m on page three of THIS PAPER <ref:2604.12022#pg1>.
Lalam: And they provide a concrete measure for deployment by showing that the asymptotic normality is given by sqrt N times the difference between your estimator and true parameter star converging to a normal distribution with Cescore as its covariance matrix, which is on page three of THIS PAPER <ref:2604.12022#pg1>.
Tom: So they’ve built an estimator that’s consistent and has a predictable asymptotic behavior, even with noise present, by using this convMMD approach. It moves beyond standard likelihood approaches that usually assume perfect data on page three of THIS PAPER <ref:2604.12022#pg1>.
Jane: And crucially, they don't just claim it works for Gaussian noise; they validate it across various scenarios, including Gaussian Mixture Models and Error-in-Variables Regression models, showing superior robustness when the noise is non-Gaussian like Laplace or Student’s t distributions on page four of THIS PAPER.
Lu: So the paper moves from theory to practice by demonstrating that this method is competitive with likelihood benchmarks while maintaining stability even when outliers are included in the data in regression analysis.
Meng: It sounds promising because it suggests we can use more flexible tools for inference without needing to know the noise law perfectly upfront, which simplifies our setup significantly.
Lalam: The overall message of this paper is that you can get a principled way to do inference with noisy data using convMMD, providing a flexible tool that's theoretically grounded on page three of THIS PAPER <ref:2604.12022#pg1>.
Conclusion: Tom: Looking at the title, "Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data," it sounds like they’ve put together a robust simulation framework to handle this specific type of data corruption. It’s about using convMMD to navigate noise.
Jane: And what does that mean in plain terms? It means we have a method, the convMMD estimator, that can take messy observations—where the noise level isn't constant—and give us reliable parameter estimates without needing perfect knowledge of the noise structure.
Lu: The authors establish strong theoretical foundations by proving consistency and asymptotic normality for this estimator on page three of THIS PAPER, which means we have a mathematically sound tool to rely on <ref:2604.12022#pg1>.
Meng: For someone who runs production models, the implication is that instead of spending months trying to figure out complex correction techniques, you can use this estimator and get a result that’s stable and well-characterized.
Lalam: And they give you the explicit characterization of the uncertainty through Cescore, so you know exactly how much your confidence intervals are inflated by the noise level on page three of THIS PAPER <ref:2604.12022#pg1>.
Tom: So, the main point is this framework offers a flexible alternative to classical likelihood methods that don't account for measurement error and provides a path toward more robust inference on noisy data.
Jane: It’s about moving away from models that demand perfect data and towards tools that can handle the reality of real-world measurements where noise is messy, which is what this work achieves.
Ritwik Vashistha, Jeff M. Phillips, Abhra Sarkar, Arya Farahi
Department of Statistics and Data Sciences, The University of Texas at Austin · NSF-Simons AI Institute for Cosmic Origins, USA · Kahlert School of Computing, University of Utah
stat.ME, stat.ML
Submitted: 2026-04-13
Updated: 2026-10-27
Code: https://github.com/afarahi/American-Housing-Survey-Study-
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 86/100
The gist: Convolutional Maximum Mean Discrepancy for Inference in Noisy Data introduces a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution…
Key concepts
- Maximum Mean Discrepancy (MMD)
- MMD measures the distance between two probability distributions by embedding them into a high-dimensional space using a kernel. It quantifies how different two data sets are, allowing researchers to compare distributions without assuming a specific underlying data model.
- Convolutional MMD (convMMD)
- convMMD extends MMD by considering the convolution of the true distribution with a noise distribution. This allows the metric to effectively compare distributions even when samples are corrupted by heteroscedastic noise, providing a more accurate comparison than standard MMD.
- Equivalence to Noisy Comparison
- A key finding is that comparing noisy distributions using convMMD is mathematically equivalent to comparing noiseless distributions using a modified kernel. This means the noise contamination can be handled by simply adjusting the kernel used in the comparison, simplifying theoretical analysis.
- Asymptotic Normality
- This property describes how an estimator behaves as the sample size grows large. Proving asymptotic normality shows that the convMMD estimator is statistically well-behaved, allowing researchers to construct confidence intervals and test hypotheses reliably.
Terminology
Summary
Convolutional Maximum Mean Discrepancy for Inference in Noisy Data introduces a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution using convolutional MMD (convMMD) to compare distributions after noise convolution The authors establish finite-sample deviation bounds unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing Leveraging these insights, they introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations, establishing its consistency and asymptotic normality using stochastic gradient descent. p. 0 We establish finite-sample deviation bounds that are unaffected by measurement error
and prove an equivalence between testing under noise and kernel smoothing. Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy,
heteroscedastic observations.Where the paper says it
Theoretical Foundation of the Metric
The paper begins by reviewing the Maximum Mean Discrepancy (MMD) statistic, defining it as a measure of distance between two distributions through their embeddings in a reproducing kernel Hilbert space (RKHS). A well-known direct computational form for MMD2 is given by MMD2(p, q) = EX,X′ [k (X, X′)] + EY,Y ′ [k (Y, Y′)] − 2EX,Y [k (X, Y)]. p. 6 Let k be the
kernel associated with RKHS H (Berlinet and Thomas-Agnan, 2011; Muandet et al. A well-known direct computational form for MMD2
is
MMD2
(p, q) = EX,X′ [k (X, X′
)] + EY,Y ′ [k (Y, Y ′
)] − 2EX,Y [k (X, Y )] ,
where X, X′
iid∼ p and Y, Y ′
iid∼ q. In this work, we introduce a novel framework for inference with samples
corrupted by potentially heteroscedastic noise from a known distribution. Central to
our approach is the convolutional MMD (convMMD), which compares distributions
after noise convolution and retains metric validity under standard kernel conditions. Central to
our approach is the convolutional MMD (convMMD), which compares distributions
after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error
and prove an equivalence between testing under noise and kernel smoothing.Where the paper says it
Where the paper says it
Where the paper says it
Equivalence to Noisy Comparison
A key theoretical result establishes an equivalence between the noisy comparison and a noiseless comparison using a noise-smoothed kernel. Theorem 3.10 shows that MMD between the noisy distributions p ∗ m and q ∗ m with respect to a translation-invariant kernel k is equal to the MMD between the true, noiseless distributions p and q with respect to a modified kernel ek. p. 3 vMMD) that compares distributions once noise has been added, and show that it remains a
proper metric. For translation-invariant kernels, we establish an equivalence: convMMD between noisy samples corresponds to MMD between clean distributions with a noise-smoothed
kernel. Let C(X ) be a class of real-valued functions. We define F as the unit
ball in a Reproducing Kernel Hilbert Space (RKHS) of real-valued functions defined on X ,
where ∥f∥H ≤ 1 for f ∈ H, then MMD is defined as
MMD(p, q) = sup
f∈H,∥f∥H≤1
|Ep(f(X)) − Eq(f(Y ))|. We focus on scenarios with independent
heteroscedastic noise with known error distributions and a parametric form for the latent
distribution. We show that, under mild regularity conditions, minimizing MMD on noisy
data is mathematically equivalent to minimizing MMD on clean data using noise-adjusted
kernels.Where the paper says it
Where the paper says it
Where the paper says it
Estimation and Asymptotic Properties
The paper develops a parametric estimation method by utilizing convMMD as a loss function, proving its consistency and asymptotic normality. The estimator θbN is the minimizer of the semi-empirical convMMD objective LN (θ) ≡ MMD2(p∗m)N,qθ∗m. p. 0 We establish finite-sample deviation bounds that are unaffected by measurement error
and prove an equivalence between testing under noise and kernel smoothing. Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy,
heteroscedastic observations. We derive finite-sample deviation bounds that demonstrate that estimation error is
governed by sample size rather than noise magnitude. Then, we propose a parametric estimation method that utilizes convMMD as a loss function, prove consistency and asymptotic
normality of the resulting estimator, and develop a computationally efficient stochastic gradient algorithm. How do the
concentration and deviation properties of kernel-based estimators behave in the presence of
noise, and under what conditions do they preserve their finite-sample guarantees? Furthermore, can MMD serve as a valid loss function for parametric estimation from noisy data,
facilitating likelihood-free yet statistically rigorous inference? Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy,
heteroscedastic observations. We establish its consistency and asymptotic normality,
and provide an efficient implementation using stochastic gradient descent.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Empirical Validation and Application
The proposed method is validated through simulations across various scenarios, including Gaussian Mixture Models (GMMs) and Error-in-Variables Regression (EIVR) models. In the GMM study, convMMD performs comparably to likelihood-based benchmarks under Gaussian noise but shows superior robustness when the noise is non-Gaussian, such as Laplace or Student’s t distributions. p. 4 This makes direct likelihood-based
inference intractable for all but the most trivial cases (e. , when all distributions are Gaussian). Our approach avoids discretization,
numerical challenges associated with Fourier inversion, and high computational demands of
posterior sampling, utilizing instead a stochastic gradient descent algorithm for high efficiency. Finally, while we leverage the known noise structure for efficiency, the well-studied
robustness properties of the MMD metric (Ch´erief-Abdellatif and Alquier, 2022) ensure our
estimator remains robust to the presence of outliers in the data, achieving a fine balance
between statistical efficiency and high-level robustness. Then, we propose a parametric estimation method that utilizes convMMD as a loss function, prove consistency and asymptotic
normality of the resulting estimator, and develop a computationally efficient stochastic gradient algorithm. Through simulations and applications in astronomy, anthropometry, and
housing survey data, we demonstrate the validity and efficacy of the proposed method and
tangible improvements over classical techniques, specifically for non-Gaussian noises.Where the paper says it
Where the paper says it
Where the paper says it
Statistical Cost of Measurement Error
The analysis quantifies the statistical cost of measurement error through Theorem 3.12, which bounds the expected squared error of the unbiased convMMD U-statistic. p. 0 Central to
our approach is the convolutional MMD (convMMD), which compares distributions
after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error
and prove an equivalence between testing under noise and kernel smoothing. , 2010), environment (Gryparis et al. , 2009),
economics (Bound and Krueger, 1991; Jerzak and Jessee, 2025), epidemiology (Keogh et al. For example, in
rounding to the nearest integer, the error may be modeled as U ∼ Uniform(−0. 6 allows us to uniquely identify p and q in the presence of noise if and only if
Ep∗m[f(Xe)] = Eq∗m[f(Ye)] for all f ∈ C(R
d
), where C(R
d
) is the space of bounded continuous
functions in R
d
.Where the paper says it
Where the paper says it
Where the paper says it
Algorithmic Implementation
The method is implemented using an efficient stochastic gradient descent algorithm (Algorithm 1) which computes an unbiased gradient estimate Jbθ using Monte Carlo sampling of latent samples and simulated noise. The simulation design details the generation of noisy data from a true latent distribution p belonging to a parametric family qθ, allowing for flexible testing of different noise models like Gaussian, Uniform, Laplace, and Student’s t distributions. p. 5 Specifically, we propose a simulation-based approach that uses MMD to perform implicit deconvolution, achieved by minimizing the discrepancy between the convolved
model distribution and the observed noisy data. We focus on scenarios with independent
heteroscedastic noise with known error distributions and a parametric form for the latent
distribution. Then, we propose a parametric estimation method that utilizes convMMD as a loss function, prove consistency and asymptotic
normality of the resulting estimator, and develop a computationally efficient stochastic gradient algorithm. Through simulations and applications in astronomy, anthropometry, and
housing survey data, we demonstrate the validity and efficacy of the proposed method and
tangible improvements over classical techniques, specifically for non-Gaussian noises.Where the paper says it
Where the paper says it
Conclusion and Future Directions
The authors propose a flexible alternative to classical likelihood and Fourier inversion-based approaches, establishing theoretical foundations for the convMMD estimator, proving its consistency and asymptotic normality The key finding is that measurement error does not degrade the rate of convergence but instead results in an explicitly characterized inflation of the asymptotic variance. Future research aims to relax assumptions on a known noise model or develop nonparametric frameworks with guarantees under model misspecification. p. 2 These challenges motivate the need for
inferential frameworks that are both theoretically principled and computationally efficient. Our broader goal is to develop a methodology that remains robust to realistic unit-specific
error structures while retaining the flexibility and power of modern nonparametric tools.Where the paper says it
Acknowledgment
The authors thank Dr. Antonio Linero, Dr. Connor Jerzak and Ikjun Choi for their valuable suggestions. p. 1 , 2010), environment (Gryparis et al. , 2009),
economics (Bound and Krueger, 1991; Jerzak and Jessee, 2025), epidemiology (Keogh et al. farahi@austin. edu
1Department of Statistics and Data Sciences,
The University of Texas at Austin, USA
2The NSF-Simons AI Institute for Cosmic Origins, USA
3Kahlert School of Computing, University of Utah, USA
Abstract
Modern data analyses frequently encounter settings where samples of variables are
contaminated by measurement error.Where the paper says it
Where the paper says it
Supplementary Materials
The Supplementary Materials provide detailed proofs of the theoretical results, additional numerical illustrations of the main theorems, and additional information on the algorithm, simulation experiments, and real datasets analyzed in the main paper The data sources are detailed in the supplementary materials. The authors also provide an open research direction regarding kernel selection to minimize asymptotic variance. p. 0 Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy,
heteroscedastic observations. We establish its consistency and asymptotic normality,
and provide an efficient implementation using stochastic gradient descent.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Develop a flexible parameter estimation framework for models corrupted by heteroscedastic measurement error using convolutional maximum mean discrepancy (convMMD). This system can estimate parameters in astronomy and social sciences while remaining robust to non-Gaussian noise distributions that traditional likelihood methods fail on.
-
Implement a simulation-based, likelihood-free objective function for deconvolution, allowing the AI to implicitly perform deconvolution without needing intractable integrals. This enables parameter estimation from noisy data across a wide range of latent models and noise processes.
-
Design an efficient stochastic gradient descent algorithm specifically tailored for minimizing the convMMD objective function during parameter estimation. This allows the system to find optimal parameters in high-dimensional settings with computational efficiency, as detailed by Algorithm 1 in the supplementary materials.
-
Establish a formal frequentist framework for inference by proving consistency and asymptotic normality of the estimator under measurement error, providing a Central Limit Theorem for large sample sizes. This guarantees that parameter estimates converge at a rate of N to zero, even with noise present in the data.
-
Integrate noise-aware kernel bandwidth selection heuristics into the optimization process using median rules on pairwise distances between samples. This allows for an adaptive kernel choice that minimizes the asymptotic variance of the estimator, maximizing statistical efficiency.
Abstract
Ignoring measurement noise degrades statistical inference. Existing correction techniques are often computationally costly and inefficient. Kernel methods based on maximum mean discrepancy have enabled efficient, likelihood-free inference, but they typically assume clean data. We introduce a framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution. Central to our method is the convolutional maximum mean discrepancy, which compares distributions after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error and an equivalence between testing under noise and kernel smoothing. Based on this discrepancy, we propose an estimator, establish its consistency and asymptotic normality, and provide an efficient implementation using stochastic gradient descent. We demonstrate the practical effectiveness of our method through simulations and applications in astronomy, anthropometry, and the social sciences. Our method provides a unified, likelihood-free, simulation-based framework for efficient and robust inference for broad classes of problems in noisy data.
Sources
- The DECADE cosmic shear project I: A new weak lensing shape catalog of 107 million galaxies
- Statistical Inference for Generative Models with Maximum Mean Discrepancy
- Total robustness in Bayesian Nonlinear Regression
- Correcting Bias When Using Latent Regressors
- A Practical Introduction to Kernel Discrepancies: MMD, HSIC & KSD
- Characterizing Measurement Error in the German Socio-Economic Panel Using Linked Survey and Administrative Data
- Denoising Data with Measurement Error Using a Reproducing Kernel-Based Diffusion Model
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes