Semiparametric Efficient Inference under Non-Informative Complex Survey Designs

arXiv:2609.17288 · math.ST, stat.ME, stat.TH · Submitted 2026-09-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.

Ada: We're looking at the paper: "Semiparametric Efficient Inference under Non-Informative Complex Survey Designs".

Paper summary: Ada: So to wrap up, the paper "Semiparametric Efficient Inference under Non-Informative Complex Survey Designs" shows that for dependent survey designs, we can still achieve locally asymptotically normal behavior with a tangent space structure from a Poisson experiment.

Emmy: The authors show that first order efficiency only needs the design's limiting inclusion probability function to describe the dependence on the design itself. They handle finite population targets by using first order asymptotic expansions, which is different from exact random sums.

Kolya: What this means for us is that they provide a framework where superpopulation and finite population centerings are equivalent in terms of local regularity, linked by a Pythagorean decomposition that gives us an explicit correction term. They also give conditions under which cross fitted estimators can reach both bounds consistently estimate their variances.

Ada: The paper characterizes observed data influence functions using tangent space projection for general pathwise differentiable targets while keeping full data model restrictions without needing a known moment representation.

Emmy: This is significant because it establishes local regularity and efficiency results for corresponding random finite population targets under the joint limit theory, which is a big structural finding here.

Kolya: The paper also notes that specific results match prior work from Armstrong and Bai et al., showing consistency under further specializations like known propensity and moment conditions. But they also point out a limitation: sample splitting alone might not eliminate the first order remainder for mean and conditional mean regression targets.

Ada: That's a fair caveat; it tells us where this specific method stops working without more stringent assumptions on the estimator itself.

Emmy: Overall, this work provides a way to understand how complex survey designs affect efficiency by linking different theoretical frameworks together, moving from the full data model to the finite population context.

Conclusion: Ada: So we're wrapping up this piece on "Semiparametric Efficient Inference under Non-Informative Complex Survey Designs."

Emmy: That paper tackles how to do efficient inference when you have these really messy survey designs, where everything is dependent on the sampling indicators.

Kolya: It shows that even with all that complexity, you can get locally asymptotically normal behavior, which means your results behave nicely near a specific point.

Ada: And they're using a tangent-space structure inherited from a simpler Poisson experiment to describe this observed data experiment.

Emmy: They break down the problem by showing that the first order efficiency actually only depends on the design through its limiting inclusion probability function. That’s a big simplification for how we model things.

Kolya: But they also handle those finite population targets, which are usually tricky because they involve exact sums, by using first order asymptotic expansions instead.

Ada: They establish a link between superpopulation and finite population centerings through this Pythagorean decomposition, giving us a generalized finite population correction term.

Emmy: It means the local regularity under the survey experiment is equivalent to the superpopulation regularity under assumption A three. The two convolution theorems share the same residual component, which simplifies things.

Kolya: So what this really means for someone doing statistical inference is that if you're efficient under one centering, you’ll be efficient under the other because of that decomposition.

Ada: And they provide high level sufficient conditions for cross fitted estimators to reach both those bounds and consistently estimate their variances.

Emmy: They find specific results for mean and conditional mean regression targets where just splitting the sample isn't enough to eliminate that first order remainder.

Kolya: It’s important that we know exactly where the method stops working without more stringent assumptions on the estimator itself.

Ada: The paper characterizes observed data influence functions using tangent space projection for general pathwise differentiable targets, keeping full data model restrictions intact even without needing a known moment representation.

Emmy: This is crucial because it sets up local regularity and efficiency results for random finite population targets under the joint limit theory.

Kolya: It’s a solid piece of structural work, but we still need to see how robust these bounds are when the sampling indicators are really non informative.

Hiroki Chiba, Kosuke Morikawa

Department of Statistics, Iowa State University

math.ST, stat.ME, stat.TH

Submitted: 2026-09-15

Updated: 2026-11-17

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The paper develops semiparametric efficiency theory for observed-data experiments under a broad class of dependent survey sampling designs, focusing on two key complications…

Key concepts

Design-induced dependence
This refers to situations where the way data is sampled is not independent across different indicators. The theory shows how this dependence affects the local asymptotic normality of the experiment, requiring specific handling in efficiency calculations.
Local Asymptotic Normality (LAN)
LAN describes how an observed-data experiment behaves locally around a true parameter value. For dependent designs, it inherits a structure from simpler Poisson experiments, which helps characterize the efficient influence function for these complex surveys.
Superpopulation and Finite-Population Regularity
These are two ways to describe the local regularity of an estimator. The paper shows they are linked by a Pythagorean decomposition, meaning the total efficiency bound can be broken down into a full-data component and a finite-population correction term.

Terminology

Summary

The paper develops semiparametric efficiency theory for observed-data experiments under a broad class of dependent survey sampling designs, focusing on two key complications: design-induced dependence among sampling indicators and the randomness of finite-population targets under the superpopulation law. p. 0

For general semiparametric full-data models, the observed-data experiment generated by a dependent design is locally asymptotically normal (LAN) with a tangent-space structure inherited from a reference Poisson experiment. p. 0 First-order efficiency in this setting depends on the design only through its limiting inclusion-probability function. p. 0 Standard missing-at-random projection in this reference model characterizes the observed-data efficient influence function (EIF), while finite-population targets are handled via first-order asymptotic expansions, moving beyond exact random sums to incorporate nonlinear census characteristics. p. 0 Superpopulation and finite-population centerings yield equivalent notions of local regularity, linked by a Pythagorean decomposition that yields a generalized finite-population correction term. p. 0 The central result provides an experiment-level representation of first-order semiparametric efficiency under dependent survey sampling. p. 1

The analysis further addresses the random-target theory for finite-population counterparts. p. 1 This includes developing theory for targets without requiring an exact random-sum representation and allowing dependent sampling indicators in survey experiments, where nonlinear census roots enter through first-order expansions rather than exact sums. p. 2 The equivalence of local regularity under the joint survey experiment and superpopulation local regularity is established under Assumption A3. p. 2 Furthermore, the two convolution theorems share the same residual component, implying that an estimator is efficient under one centering if and only if it is efficient under the other, with bounds satisfying a Pythagorean decomposition: Superpopulation bound = Full-data bound + Finite-population bound, resulting in a generalized finite-population correction. p. 2

The paper also provides high-level sufficient conditions for cross-fitted estimators to attain both bounds and consistently estimate their variances. p. 2 Specific results are derived for mean and conditional-mean regression targets under dependent sampling, showing that sample splitting alone may not eliminate the first-order remainder, necessitating specific sufficient conditions. p. 2 The relationship with prior work is precisely characterized: fixed-submodel LAN expansions agree with those of Armstrong (2022), and under further specialization (unrestricted full-data law, known propensity, known nuisance-free moment conditions), the observed score and efficiency bound match those of Armstrong (2022) and Bai et al. p. 3 (2026). The paper concludes by characterizing observed-data EIFs via tangent-space projection for general finite-dimensional pathwise-differentiable targets, retaining full-data model restrictions without requiring a known moment representation, and establishing local regularity/efficiency results for corresponding random finite-population targets under the joint limit theory. p. 3

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Precise Efficiency Bound Determination: The improved system can determine if an estimator attains efficiency for either superpopulation or finite-population targets by checking if a residual component W in the convolution decomposition is degenerate at zero, as shown in Theorem 3.9.

  2. Design-Specific Optimal Sampling: The system can identify the optimal limiting inclusion probability pi star by solving a convex optimization problem involving the expected variance of d pi (L) conditional on X, ensuring that the selected design satisfies both budget constraints and asymptotic efficiency requirements.

  3. Cross-Fitted Estimator Validation: The system can verify sufficient conditions for cross-fitted estimators to attain both bounds and consistently estimate their asymptotic variances by checking conditions C1 through C4, including mean-square consistency of estimated influence functions.

  4. Heteroskedasticity Robustness Analysis: For regression targets, the system can compare the performance of complete-case Ordinary Least Squares (OLS) against feasible Generalized Least Squares (GLS) using cross-fitted conditional variance estimators to assess how heteroskedasticity location affects efficiency gains across high and low leverage covariate values.

  5. Design-Agnostic Sampling Comparison: The system can compare different complex survey designs, such as Poisson sampling, SRSWOR, cluster sampling, and stratified two-stage sampling, by testing whether they share the same limiting inclusion probabilities and how this shared limit dictates the first-order efficiency bound.

Sources

Related papers