Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data

arXiv:2604.12022 · stat.ME, stat.ML · Submitted 2026-04-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data".

Jane: Here's the gist of the paper. Convolutional Maximum Mean Discrepancy for Inference in Noisy Data introduces a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution using convolutional MMD (convMMD) to compare distributions after noise convolution The authors establish finite-sample deviation bounds unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing Leveraging these insights, they introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations, establishing its consistency and asymptotic normality using stochastic gradient descent.

Tom: Theoretical Foundation of the Metric The paper begins by reviewing the Maximum Mean Discrepancy (MMD) statistic, defining it as a measure of distance between two distributions through their embeddings in a reproducing kernel Hilbert space (RKHS).

Jane: A well-known direct computational form for MMD2 is given by MMD2(p, q) = EX,X′

k (X, X′): + EY,Y ′

k (Y, Y′): − 2EX,Y

k (X, Y): .

Tom: The authors then move to the challenge of noisy data by defining the convolution MMD as convMMD(p, q, m) ≡ MMD(p ∗ m, q ∗ m) = sup f∈H,∥f∥H≤1 Ep∗m(f(X)) − Eq∗m(f(Y)).

Paper summary: Tom: To recap what we've covered, this paper, "Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data," introduces a new way to do inference when your data has measurement error that varies depending on the location. The authors claim their core contribution is the convolutional MMD, or convMMD, which compares distributions after convolving them with the noise distribution.

Jane: They set out to show that this framework allows for distribution-free inference even when samples are contaminated by heteroscedastic noise from a known distribution. They establish that convMMD is a valid metric and they prove it's equivalent to a noiseless comparison using a smoothed kernel, which is essentially the ek kernel on page two of THIS PAPER <ref:2604.12022#pg1>.

Lu: The paper then develops an estimator called theta b N, defined as the minimizer of this semi-empirical convMMD objective function LN on page three of THIS PAPER <ref:2604.12022#pg1>. They rigorously prove its consistency and asymptotic normality under the conditions they set out.

Meng: What's important for us in a production setting is how scalable this estimation is. They show that even when fitting it to noisy data, the estimator converges almost surely to the true parameter star if you use samples of size N from the modified distribution q theta star m on page three of THIS PAPER <ref:2604.12022#pg1>.

Lalam: And they provide a concrete measure for deployment by showing that the asymptotic normality is given by sqrt N times the difference between your estimator and true parameter star converging to a normal distribution with Cescore as its covariance matrix, which is on page three of THIS PAPER <ref:2604.12022#pg1>.

Tom: So they’ve built an estimator that’s consistent and has a predictable asymptotic behavior, even with noise present, by using this convMMD approach. It moves beyond standard likelihood approaches that usually assume perfect data on page three of THIS PAPER <ref:2604.12022#pg1>.

Jane: And crucially, they don't just claim it works for Gaussian noise; they validate it across various scenarios, including Gaussian Mixture Models and Error-in-Variables Regression models, showing superior robustness when the noise is non-Gaussian like Laplace or Student’s t distributions on page four of THIS PAPER.

Lu: So the paper moves from theory to practice by demonstrating that this method is competitive with likelihood benchmarks while maintaining stability even when outliers are included in the data in regression analysis.

Meng: It sounds promising because it suggests we can use more flexible tools for inference without needing to know the noise law perfectly upfront, which simplifies our setup significantly.

Lalam: The overall message of this paper is that you can get a principled way to do inference with noisy data using convMMD, providing a flexible tool that's theoretically grounded on page three of THIS PAPER <ref:2604.12022#pg1>.

Conclusion: Tom: Looking at the title, "Simulation-Based Inference with Convolutional Maximum Mean Discrepancy for Noisy Data," it sounds like they’ve put together a robust simulation framework to handle this specific type of data corruption. It’s about using convMMD to navigate noise.

Jane: And what does that mean in plain terms? It means we have a method, the convMMD estimator, that can take messy observations—where the noise level isn't constant—and give us reliable parameter estimates without needing perfect knowledge of the noise structure.

Lu: The authors establish strong theoretical foundations by proving consistency and asymptotic normality for this estimator on page three of THIS PAPER, which means we have a mathematically sound tool to rely on <ref:2604.12022#pg1>.

Meng: For someone who runs production models, the implication is that instead of spending months trying to figure out complex correction techniques, you can use this estimator and get a result that’s stable and well-characterized.

Lalam: And they give you the explicit characterization of the uncertainty through Cescore, so you know exactly how much your confidence intervals are inflated by the noise level on page three of THIS PAPER <ref:2604.12022#pg1>.

Tom: So, the main point is this framework offers a flexible alternative to classical likelihood methods that don't account for measurement error and provides a path toward more robust inference on noisy data.

Jane: It’s about moving away from models that demand perfect data and towards tools that can handle the reality of real-world measurements where noise is messy, which is what this work achieves.

Ritwik Vashistha, Jeff M. Phillips, Abhra Sarkar, Arya Farahi

Department of Statistics and Data Sciences, The University of Texas at Austin · NSF-Simons AI Institute for Cosmic Origins, USA · Kahlert School of Computing, University of Utah

stat.ME, stat.ML

Submitted: 2026-04-13

Updated: 2026-10-27

Code: https://github.com/afarahi/American-Housing-Survey-Study-

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 86/100

The gist: Convolutional Maximum Mean Discrepancy for Inference in Noisy Data introduces a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution…

Key concepts

Maximum Mean Discrepancy (MMD)
MMD measures the distance between two probability distributions by embedding them into a high-dimensional space using a kernel. It quantifies how different two data sets are, allowing researchers to compare distributions without assuming a specific underlying data model.
Convolutional MMD (convMMD)
convMMD extends MMD by considering the convolution of the true distribution with a noise distribution. This allows the metric to effectively compare distributions even when samples are corrupted by heteroscedastic noise, providing a more accurate comparison than standard MMD.
Equivalence to Noisy Comparison
A key finding is that comparing noisy distributions using convMMD is mathematically equivalent to comparing noiseless distributions using a modified kernel. This means the noise contamination can be handled by simply adjusting the kernel used in the comparison, simplifying theoretical analysis.
Asymptotic Normality
This property describes how an estimator behaves as the sample size grows large. Proving asymptotic normality shows that the convMMD estimator is statistically well-behaved, allowing researchers to construct confidence intervals and test hypotheses reliably.

Terminology

Summary

Convolutional Maximum Mean Discrepancy for Inference in Noisy Data introduces a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution using convolutional MMD (convMMD) to compare distributions after noise convolution The authors establish finite-sample deviation bounds unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing Leveraging these insights, they introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations, establishing its consistency and asymptotic normality using stochastic gradient descent. p. 0

Where the paper says it

We establish finite-sample deviation bounds that are unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing. Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations.

Theoretical Foundation of the Metric

The paper begins by reviewing the Maximum Mean Discrepancy (MMD) statistic, defining it as a measure of distance between two distributions through their embeddings in a reproducing kernel Hilbert space (RKHS). A well-known direct computational form for MMD2 is given by MMD2(p, q) = EX,X′ [k (X, X′)] + EY,Y ′ [k (Y, Y′)] − 2EX,Y [k (X, Y)]. p. 6

Where the paper says it

Let k be the kernel associated with RKHS H (Berlinet and Thomas-Agnan, 2011; Muandet et al. A well-known direct computational form for MMD2 is MMD2 (p, q) = EX,X′ [k (X, X′ )] + EY,Y ′ [k (Y, Y ′ )] − 2EX,Y [k (X, Y )] , where X, X′ iid∼ p and Y, Y ′ iid∼ q.

The authors then move to the challenge of noisy data by defining the convolution MMD as convMMD(p, q, m) ≡ MMD(p ∗ m, q ∗ m) = sup f∈H,∥f∥H≤1 Ep∗m(f(X)) − Eq∗m(f(Y)). p. 0
Where the paper says it

In this work, we introduce a novel framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution. Central to our approach is the convolutional MMD (convMMD), which compares distributions after noise convolution and retains metric validity under standard kernel conditions.

They prove that convMMD is a valid metric under certain conditions, showing convMMD(p, q, m) = 0 if and only if p = q. p. 0
Where the paper says it

Central to our approach is the convolutional MMD (convMMD), which compares distributions after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing.

Equivalence to Noisy Comparison

A key theoretical result establishes an equivalence between the noisy comparison and a noiseless comparison using a noise-smoothed kernel. Theorem 3.10 shows that MMD between the noisy distributions p ∗ m and q ∗ m with respect to a translation-invariant kernel k is equal to the MMD between the true, noiseless distributions p and q with respect to a modified kernel ek. p. 3

Where the paper says it

vMMD) that compares distributions once noise has been added, and show that it remains a proper metric. For translation-invariant kernels, we establish an equivalence: convMMD between noisy samples corresponds to MMD between clean distributions with a noise-smoothed kernel.

This modified kernel ek is defined as ek(x, y) = EU,U′∼m [k (x + U, y + U′)]. p. 6
Where the paper says it

Let C(X ) be a class of real-valued functions. We define F as the unit ball in a Reproducing Kernel Hilbert Space (RKHS) of real-valued functions defined on X , where ∥f∥H ≤ 1 for f ∈ H, then MMD is defined as MMD(p, q) = sup f∈H,∥f∥H≤1 |Ep(f(X)) − Eq(f(Y ))|.

This demonstrates that noise contamination is mathematically equivalent to a noiseless MMD comparison using a smoother, convolved kernel. p. 5
Where the paper says it

We focus on scenarios with independent heteroscedastic noise with known error distributions and a parametric form for the latent distribution. We show that, under mild regularity conditions, minimizing MMD on noisy data is mathematically equivalent to minimizing MMD on clean data using noise-adjusted kernels.

Estimation and Asymptotic Properties

The paper develops a parametric estimation method by utilizing convMMD as a loss function, proving its consistency and asymptotic normality. The estimator θbN is the minimizer of the semi-empirical convMMD objective LN (θ) ≡ MMD2(p∗m)N,qθ∗m. p. 0

Where the paper says it

We establish finite-sample deviation bounds that are unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing. Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations.

Theorem 3.15 provides a generalization bound and consistency result, showing that the empirical convMMD estimator θbN converges almost surely to θ⋆ if it is fitted on noisy data and samples of size N from qθ ∗ m. p. 3
Where the paper says it

We derive finite-sample deviation bounds that demonstrate that estimation error is governed by sample size rather than noise magnitude. Then, we propose a parametric estimation method that utilizes convMMD as a loss function, prove consistency and asymptotic normality of the resulting estimator, and develop a computationally efficient stochastic gradient algorithm.

Furthermore, Theorem 3. p. 2
Where the paper says it

How do the concentration and deviation properties of kernel-based estimators behave in the presence of noise, and under what conditions do they preserve their finite-sample guarantees? Furthermore, can MMD serve as a valid loss function for parametric estimation from noisy data, facilitating likelihood-free yet statistically rigorous inference?

16 establishes the asymptotic normality of the estimator, showing that √N(θbN − θ⋆) → N (0, Cescore), where Cescore is the Godambe information matrix. p. 0
Where the paper says it

Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations. We establish its consistency and asymptotic normality, and provide an efficient implementation using stochastic gradient descent.

Empirical Validation and Application

The proposed method is validated through simulations across various scenarios, including Gaussian Mixture Models (GMMs) and Error-in-Variables Regression (EIVR) models. In the GMM study, convMMD performs comparably to likelihood-based benchmarks under Gaussian noise but shows superior robustness when the noise is non-Gaussian, such as Laplace or Student’s t distributions. p. 4

Where the paper says it

This makes direct likelihood-based inference intractable for all but the most trivial cases (e. , when all distributions are Gaussian).

In EIVR regression, convMMD consistently achieves the lowest Absolute Error (AE) and remains stable even when outliers are included in the data, unlike naive OLS. p. 5
Where the paper says it

Our approach avoids discretization, numerical challenges associated with Fourier inversion, and high computational demands of posterior sampling, utilizing instead a stochastic gradient descent algorithm for high efficiency. Finally, while we leverage the known noise structure for efficiency, the well-studied robustness properties of the MMD metric (Ch´erief-Abdellatif and Alquier, 2022) ensure our estimator remains robust to the presence of outliers in the data, achieving a fine balance between statistical efficiency and high-level robustness.

Real-world applications include galaxy cluster data in astronomy and anthropometry studies, where convMMD provides lower RMSE than existing methods. p. 3
Where the paper says it

Then, we propose a parametric estimation method that utilizes convMMD as a loss function, prove consistency and asymptotic normality of the resulting estimator, and develop a computationally efficient stochastic gradient algorithm. Through simulations and applications in astronomy, anthropometry, and housing survey data, we demonstrate the validity and efficacy of the proposed method and tangible improvements over classical techniques, specifically for non-Gaussian noises.

Statistical Cost of Measurement Error

The analysis quantifies the statistical cost of measurement error through Theorem 3.12, which bounds the expected squared error of the unbiased convMMD U-statistic. p. 0

Where the paper says it

Central to our approach is the convolutional MMD (convMMD), which compares distributions after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error and prove an equivalence between testing under noise and kernel smoothing.

This bound shows that E[nconvMMD 2(p, q, m)]o2 ≤ E[nMMD 2(p, q)]o2 + 2N(N − 1) 32L 2kEϕ∼g(ϕψ) α(ϕ) + 8Kq 32L 2kEϕ∼g(ϕψ) α(phi). p. 1
Where the paper says it

, 2010), environment (Gryparis et al. , 2009), economics (Bound and Krueger, 1991; Jerzak and Jessee, 2025), epidemiology (Keogh et al.

This result formally establishes that the expected squared error is bounded by a term related to the noise variance E[α(phi)]. p. 9
Where the paper says it

For example, in rounding to the nearest integer, the error may be modeled as U ∼ Uniform(−0. 6 allows us to uniquely identify p and q in the presence of noise if and only if Ep∗m[f(Xe)] = Eq∗m[f(Ye)] for all f ∈ C(R d ), where C(R d ) is the space of bounded continuous functions in R d .

The asymptotic covariance matrix Cescore is shown to be directly influenced by the noise distribution m, confirming that higher noise levels lead to a larger Cescore and thus wider confidence intervals.

Algorithmic Implementation

The method is implemented using an efficient stochastic gradient descent algorithm (Algorithm 1) which computes an unbiased gradient estimate Jbθ using Monte Carlo sampling of latent samples and simulated noise. The simulation design details the generation of noisy data from a true latent distribution p belonging to a parametric family qθ, allowing for flexible testing of different noise models like Gaussian, Uniform, Laplace, and Student’s t distributions. p. 5

Where the paper says it

Specifically, we propose a simulation-based approach that uses MMD to perform implicit deconvolution, achieved by minimizing the discrepancy between the convolved model distribution and the observed noisy data. We focus on scenarios with independent heteroscedastic noise with known error distributions and a parametric form for the latent distribution.

The algorithm utilizes a mixture of Gaussian kernels with bandwidths selected using the median rule for robustness. p. 3
Where the paper says it

Then, we propose a parametric estimation method that utilizes convMMD as a loss function, prove consistency and asymptotic normality of the resulting estimator, and develop a computationally efficient stochastic gradient algorithm. Through simulations and applications in astronomy, anthropometry, and housing survey data, we demonstrate the validity and efficacy of the proposed method and tangible improvements over classical techniques, specifically for non-Gaussian noises.

Conclusion and Future Directions

The authors propose a flexible alternative to classical likelihood and Fourier inversion-based approaches, establishing theoretical foundations for the convMMD estimator, proving its consistency and asymptotic normality The key finding is that measurement error does not degrade the rate of convergence but instead results in an explicitly characterized inflation of the asymptotic variance. Future research aims to relax assumptions on a known noise model or develop nonparametric frameworks with guarantees under model misspecification. p. 2

Where the paper says it

These challenges motivate the need for inferential frameworks that are both theoretically principled and computationally efficient. Our broader goal is to develop a methodology that remains robust to realistic unit-specific error structures while retaining the flexibility and power of modern nonparametric tools.

The framework provides a robust, tractable, theoretically sound, and potentially extensible tool for inference in the presence of measurement error The authors also suggest an open research direction in establishing a formal procedure to select kernels that minimize the asymptotic variance of the convMMD estimator.

Acknowledgment

The authors thank Dr. Antonio Linero, Dr. Connor Jerzak and Ikjun Choi for their valuable suggestions. p. 1

Where the paper says it

, 2010), environment (Gryparis et al. , 2009), economics (Bound and Krueger, 1991; Jerzak and Jessee, 2025), epidemiology (Keogh et al.

The work is supported by NSF under Cooperative Agreement 2421782, and the Simons Foundation grant MPS-AI-00010515 awarded to NSF-Simons AI Institute for Cosmic Origins (CosmicAI). p. 0
Where the paper says it

farahi@austin. edu 1Department of Statistics and Data Sciences, The University of Texas at Austin, USA 2The NSF-Simons AI Institute for Cosmic Origins, USA 3Kahlert School of Computing, University of Utah, USA Abstract Modern data analyses frequently encounter settings where samples of variables are contaminated by measurement error.

Supplementary Materials

The Supplementary Materials provide detailed proofs of the theoretical results, additional numerical illustrations of the main theorems, and additional information on the algorithm, simulation experiments, and real datasets analyzed in the main paper The data sources are detailed in the supplementary materials. The authors also provide an open research direction regarding kernel selection to minimize asymptotic variance. p. 0

Where the paper says it

Leveraging these insights, we introduce a convMMD-based estimator for inference with noisy, heteroscedastic observations. We establish its consistency and asymptotic normality, and provide an efficient implementation using stochastic gradient descent.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Develop a flexible parameter estimation framework for models corrupted by heteroscedastic measurement error using convolutional maximum mean discrepancy (convMMD). This system can estimate parameters in astronomy and social sciences while remaining robust to non-Gaussian noise distributions that traditional likelihood methods fail on.

  2. Implement a simulation-based, likelihood-free objective function for deconvolution, allowing the AI to implicitly perform deconvolution without needing intractable integrals. This enables parameter estimation from noisy data across a wide range of latent models and noise processes.

  3. Design an efficient stochastic gradient descent algorithm specifically tailored for minimizing the convMMD objective function during parameter estimation. This allows the system to find optimal parameters in high-dimensional settings with computational efficiency, as detailed by Algorithm 1 in the supplementary materials.

  4. Establish a formal frequentist framework for inference by proving consistency and asymptotic normality of the estimator under measurement error, providing a Central Limit Theorem for large sample sizes. This guarantees that parameter estimates converge at a rate of N to zero, even with noise present in the data.

  5. Integrate noise-aware kernel bandwidth selection heuristics into the optimization process using median rules on pairwise distances between samples. This allows for an adaptive kernel choice that minimizes the asymptotic variance of the estimator, maximizing statistical efficiency.

Abstract

Ignoring measurement noise degrades statistical inference. Existing correction techniques are often computationally costly and inefficient. Kernel methods based on maximum mean discrepancy have enabled efficient, likelihood-free inference, but they typically assume clean data. We introduce a framework for inference with samples corrupted by potentially heteroscedastic noise from a known distribution. Central to our method is the convolutional maximum mean discrepancy, which compares distributions after noise convolution and retains metric validity under standard kernel conditions. We establish finite-sample deviation bounds that are unaffected by measurement error and an equivalence between testing under noise and kernel smoothing. Based on this discrepancy, we propose an estimator, establish its consistency and asymptotic normality, and provide an efficient implementation using stochastic gradient descent. We demonstrate the practical effectiveness of our method through simulations and applications in astronomy, anthropometry, and the social sciences. Our method provides a unified, likelihood-free, simulation-based framework for efficient and robust inference for broad classes of problems in noisy data.

Sources

Related papers