Asymptotics of Nonparametric Estimation under General Non-monotone MAR Missingness: A Nonparametric Maximum Likelihood Approach

summary

The gist

Missing data constitute a pervasive challenge in empirical research Consequently, there is an ever-growing number of methods designed to address this challenge…

In short

The paper develops a nonparametric maximum likelihood approach for estimating density when missingness follows a general non-monotone Missing At Random (MAR) mechanism. It proves that this estimator achieves minimax optimal rates up to logarithmic factors in a Hölder class, regardless of the specific missing pattern configuration. The missingness only affects the convergence rate through an additive constant.

Key concepts

Missing At Random (MAR)
This describes a situation where the probability of data being missing depends on other observed data, but not on the actual value of the missing data itself. It is a common pattern in empirical research where this assumption simplifies analysis without requiring a complex model for how data might be missing.
Sieve MLE Estimation Rate
This refers to deriving the convergence rate for Maximum Likelihood Estimation (MLE) when dealing with missing data under MAR. The paper establishes that this rate is essentially the same as in the complete-data setting, meaning the presence of missing data doesn't fundamentally slow down how fast an estimate converges.
Hölder Class
This is a mathematical space used to describe functions (like probability density functions) based on their smoothness. Functions in this class are continuous and have a certain degree of regularity, which allows the authors to prove optimal estimation rates for density estimation.

Terminology used across episodes

This episode discusses

The paper

Asymptotics of Nonparametric Estimation under General Non-monotone MAR Missingness: A Nonparametric Maximum Likelihood Approach · Read on arXiv

University of North Carolina at Chapel Hill · Research Institute for Statistics and Information Science, University of Geneva

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Asymptotics of Nonparametric Estimation under General Non-monotone MAR Missingness".

Jane: Here's the gist of the paper. Missing data constitute a pervasive challenge in empirical research The paper derives a general rate of convergence under MAR that requires no modeling of the missingness mechanism and no restriction on the configuration of missing patterns, beyond MAR itself and a natural positivity condition Applying this result to density estimation, it shows that the complete-data density can be estimated at the minimax rate over a Hölder class, up to a logarithmic factor, for any prescribed smoothness level The missingness does not affect the rate and enters only through a constant Theoretical Foundation of Sieve MLE The paper establishes a general sieve MLE estimation rate result under MAR, showing that essentially the same rate can be attained as in the complete data setting, up to the constant 1/√c0 related to the positivity assumption The authors use recent results in Kaji (2026) to remove the stringent assumption of bounded density ratios They derive a new ignoring density estimator that achieves minimax optimal rates up to logarithmic factors in a Hölder class, inspired by the Bayesian estimator discussed in Ghosal and van der Vaart (2017a) The complete-data rate is attained for any smoothness level with only an increased constant related to a natural positivity assumption Density Estimation and Algorithm Implementation The derived result is applied to density estimation, developing a new (ignoring) estimator that achieves minimax optimal rates up to logarithmic factors in a Hölder class This estimator allows for drawing new data points from the estimated complete-data distribution The estimator is implemented using an Expectation Maximization (EM) algorithm that optimizes the parameters of the mixture distribution and operates on the missing data directly Key Contributions to MAR Estimation The paper presents several key contributions, including deriving a general sieve MLE estimation rate result under non-monotone MAR Applying this result to density estimation, it develops a new (ignoring) estimator that achieves minimax optimal rates up to logarithmic factors in a Hölder class The implementation uses an EM algorithm that operates on the missing data directly and is not related to the usual parametric EM algorithms that use implicit imputations of the missing values Simulation Results and Robustness The simulation study assesses the estimator's ability to address challenges illustrated in a motivating example where non-monotone MAR is present The method predicts the 0.1 quantile of X1 with good overall accuracy even when the lower bound of P(M = 0 X = x) varies from 0.01 to 0.1 The estimator performs well in recovering the complete-data density in high-dimension settings, showing robustness against varying levels of missingness Rate Analysis and Complexity The rate analysis demonstrates that MAR enters only as a constant The first inequality in the main result shows that KL(f P0∥Pn) ≤ (3 + 4√MMAR)δ 2n, which is obtained by bounding the exposed tail The random terms are controlled using Chebyshev's inequality, leading to a bound of OP(δ 2n ∨ n-1) for the empirical-process terms The paper concludes that the proposed estimator reaches the minimax rate up to logarithmic factors for any smoothness level, with a rate that degrades only by a constant pertaining to the positivity assumption The work opens the door to develop efficient and well-performing new estimators through closed-form iterations in practice The approach is shown to be much more widely applicable, particularly assuming P to be the set of all Gaussian mixtures Pruning redundant reasoning tokens Missing data constitute a pervasive challenge in empirical research The paper derives a general rate of convergence under MAR that requires no modeling of the missingness mechanism and no restriction on the configuration of missing patterns, beyond MAR itself and a natural positivity condition Applying this result to density estimation, it shows that the complete-data density can be estimated at the minimax rate over a Hölder class, up to a logarithmic factor, for any prescribed smoothness level The missingness does not affect the rate and enters only through a constant How the training signal is built The authors derive a general sieve MLE estimation rate result under MAR, showing that essentially the same rate can be attained as in the complete data setting, up to the constant 1/√c0 related to the positivity assumption They use recent results in Kaji (2026) to remove the stringent assumption of bounded density ratios Where the gains come from The rate analysis demonstrates that MAR enters only as a constant The first inequality in the main result shows that KL(f P0∥Pn) ≤ (3 + 4√MMAR)δ 2n, which is obtained by bounding the exposed tail The random terms are controlled using Chebyshev's inequality, leading to a bound of OP(δ 2n ∨ n-1) for the empirical-process terms Pruning redundant reasoning tokens The authors derive a general sieve MLE estimation rate result under MAR, showing that essentially the same rate can be attained as in the complete data setting, up to the constant 1/√c0 related to the positivity assumption They use recent results in Kaji (2026) to remove the stringent assumption of bounded density ratios Pruning redundant reasoning tokens

Paper summary: Tom: So we're looking at this paper today: "Asymptotics of Nonparametric Estimation under General Non-monotone MAR Missingness: A Nonparametric Maximum Likelihood Approach". Basically, it tackles the problem of missing data in empirical research, which is a big headache.

Jane: Right. The main idea they are pushing here is that you don't need to model *how* the data is missing at random, as long as it follows the general condition of MAR and there's this natural positivity condition.

Lu: So they establish a general rate of convergence using sieve maximum likelihood estimation under MAR, which means it works regardless of whether the missingness pattern is monotone or not, and they don't need to restrict the configuration beyond that basic MAR rule.

Meng: Wait, so no complex modeling of the mechanism itself? That sounds a lot simpler than what we usually have to do when missing data gets messy.

Lalam: Exactly. They use this sieve maximum likelihood approach, which is essentially an ignoring MLE version where you only model the observed marginal distributions, and they show that the rate of convergence is essentially the same as in the complete data setting up to a constant factor.

Tom: That constant factor is important because it means missingness doesn't actually slow down your estimation rate much, it just adds a bit of noise through that constant.

Jane: And when they apply this to density estimation, they get a result where you can estimate the complete data density at the minimax rate over a Hölder class up to a logarithmic factor, for any smoothness level you want.

Lu: That's significant because it means we can actually use this method for estimating the true underlying distribution even when some pieces of data are missing.

Meng: But what about implementation? How does this actually translate into something I can deploy in production, not just a theoretical result?

Tom: They develop a new ignoring estimator, and they implement it using an expectation maximization algorithm that operates directly on the missing data, which is different from those standard parametric EM algorithms that rely on filling in the missing values first.

Jane: That direct operation is key because it lets you draw new data points from the estimated complete-data distribution, which gives you more flexibility in building your model.

Lalam: The paper shows that this estimator achieves minimax optimal rates up to logarithmic factors in a Hölder class, and they use recent results from Kaji two thousand twenty-six to get around some of those older requirements about bounded density ratios <ref:2608.10113#pg2>.

Tom: So, essentially, they've found a way to get the best possible estimation rate without needing super restrictive assumptions on the missingness pattern itself.

Jane: The simulation results they show are pretty robust; even when the lower bound of the probability of missing data changes between zero point one and zero point one their estimator still predicts things like a quantile with good overall accuracy.

Lu: That robustness against varying levels of missingness in high-dimension settings is something I find very compelling for representation learning tasks.

Meng: From a deployment standpoint, the complexity is important; the rate analysis shows that the missingness term only enters as a constant, which means it doesn't blow up our computational time as n gets large.

Tom: Right, and they show that in their main result inequality, KL of the true density to their estimate is bounded by something like three plus four times the square root of the missingness rate times delta squared n.

Jane: So even with missing data, you still get a convergence bound that scales with n at a rate related to delta squared n, which is what we expect from standard estimation methods.

Lu: The random terms in that analysis are controlled using Chebyshev's inequality, leading to a bound of big O of delta squared n or one over n to the power of one for the empirical process terms.

Meng: So, if I'm worried about running this on real traffic, I need to know what the practical cost is beyond just the asymptotic rate.

Tom: The paper concludes that the proposed estimator reaches this minimax rate up to logarithmic factors for any smoothness level, and that it only degrades by a constant related to that natural positivity assumption.

Jane: It opens the door for developing more efficient estimators through closed-form iterations in practice, especially if we assume P is the set of all Gaussian mixtures.

Lu: I think what's most interesting is how this result applies broadly, not just to a specific scenario, but to any setup where MAR holds and you have that natural positivity condition.

Meng: So for production use, it suggests that as long as we stick to the MAR framework, we can aim for these near-optimal rates without getting bogged down in needing an explicit model of why the data is missing.

Tom: It's a solid result because it removes a big modeling hurdle and gives us concrete convergence benchmarks for density estimation under this kind of complex missingness.

Jane: So, to wrap up, this paper tackles the general problem of non-monotone MAR missingness by using sieve MLE to show that we can estimate densities at optimal rates without needing strong assumptions about the missingness pattern itself.

Conclusion: Tom: So, we're wrapping up this paper on asymptotics of nonparametric estimation under general non-monotone MAR missingness, published by those authors.

Jane: It boils down to a general rate of convergence under missing data that doesn't care how the missingness pattern looks, as long as it follows the basic MAR rule.

Lu: Essentially, they use a sieve maximum likelihood estimation approach to show that you can estimate the complete data density at a minimax rate over any smoothness level.

Meng: So, what does this actually mean for us when we're trying to ship models? Does it just mean our training takes less time?

Tom: Not exactly. It means the missingness doesn't slow down your estimation rate much; it only enters as a constant factor in the error bound.

Jane: That constant factor is what they relate to a natural positivity condition, so that’s where the main caveat lies for practical deployment.

Lu: They are essentially saying that you don't need to model the missingness mechanism itself, which removes a huge modeling hurdle for many of our representation learning problems.

Meng: But you still need that natural positivity assumption to get those optimal rates, right? That sounds like another layer of complexity we have to deal with.

Tom: Exactly. And they show this works even when the missingness isn't monotone, which is a big win because most real-world data isn't perfectly ordered like that.

Jane: The simulation results confirm this robustness; the estimator still recovers the complete data density well even when the missingness probability varies a lot in high-dimension settings.

Lu: From what I see, it opens up new avenues for developing estimators because they aren't tied to filling in missing values first.

Meng: So, if we want to deploy this idea, we need to focus on building an EM algorithm that works directly on the missing data without needing those pre-imputations?

Tom: That’s the implementation aspect you have to worry about. The paper shows this approach is more widely applicable than some of the older methods.

Jane: It’s about getting near-optimal performance for density estimation even when you have messy, non-monotone missing data, provided that basic MAR condition holds true.

Lu: The next thing we should look at is how this applies specifically to Gaussian mixture models, as the authors mention that's where it gets really flexible.

More episodes

← Home