Finite Sample Inference in Incomplete Models
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: We're looking at the paper: "Finite Sample Inference in Incomplete Models".
Warren: Here's the gist of the paper. Finite-sample inference in incomplete models proposes confidence regions for parameters of incomplete models with exact coverage in finite samples Our proposed procedure is based on a new test statistic that generalizes Monte Carlo tests to incomplete models and relies on simulation draws from the distribution of latent variables.
Ruth: Theoretical Framework Incomplete structural models are called incomplete because the model structure predicts a set of possible values for the outcome variables, and incompleteness arises because of multiple equilibria in games, unobserved heterogeneity in choice sets in limited attention models, interval predictions in auctions, and unknown sample selection mechanisms.
Warren: The procedure proposed applies to parametric incomplete structural models where both the support restriction and the distribution of unobserved heterogeneity are known up to a finite-dimensional parameter vector.
Ruth: Model incompleteness generally leads to partial identification, where more than one value of the model parameter could have given rise to the true data-generating process for the observed variables.
Paper summary: Ruth: To wrap up, "Finite Sample Inference in Incomplete Models" by Lixiong Li and Marc Henry proposes a confidence region method that gives exact coverage for incomplete models even with finite samples.
Warren: They achieve this by using a test statistic derived from an optimal transport formulation of the model structure, which they compute using solutions to discrete linear programming problems.
Iris: The key is that both the test statistic and its critical values rely on simulation draws from the distribution of latent variables, which allows them to achieve finite-sample valid coverage because one statistic first order stochastically dominates the other.
Ruth: They also included a fast preliminary search method using a parameter-free critical value, which avoids those common asymptotic approximations.
Warren: The paper applies this to airline entry and price competition data, showing that their procedure is more efficient than some existing methods in terms of size and power.
Iris: Their main limitation is that this approach is only for parametric incomplete structural models where both the support restriction and the distribution of unobserved heterogeneity are known up to a finite-dimensional parameter vector.
Ruth: So, in simple terms, it’s a method for getting reliable results from complex economic models when you don't have an infinite amount of data or perfect information about everything.
Conclusion: Ruth: So we've been talking about how this paper tackles getting reliable answers from models that don't have all the data or all the information they need, and now we need to talk about what this specific work is actually called and who wrote it.
Warren: It’s called "Finite Sample Inference in Incomplete Models," written by Lixiong Li and Marc Henry.
Iris: What that title tells you right away is the focus here—they're looking at how you can make inferences when the model itself is incomplete, meaning some parts of what’s happening are just hidden or we don't see them directly.
Ruth: Exactly. And what this paper does, in plain terms, is it proposes a new way to build confidence regions for parameters in these complex models that actually works even when you only have a finite sample size.
Warren: It’s not about getting perfect results with infinite data anymore; it’s about making sure the results you get with what you *do* have are actually correct.
Iris: The big idea is they use a specific test statistic, which they build from optimal transport theory—that's a fancy way of saying they’re looking at the best possible way to match the data to their incomplete model structure.
Ruth: And the crucial part for us is that this method has exact coverage in finite samples. That means if you use it, you can trust the confidence region because it actually covers your true parameter value with a certain probability, and that probability doesn't depend on how much data you have in the long run.
Warren: So what does this mean for us running models? It cuts out a lot of the messy stuff they usually have to deal with, like needing huge amounts of data just to get an idea of where your estimate is going.
Iris: And because it avoids those asymptotic approximations, it's much more practical for real-world application. It’s about getting real answers without having to guess how big the sample needs to be before you can trust your results.
Ruth: That leads us right into the next part of our conversation—how this approach actually compares to the other methods out there that people are currently using for these kinds of problems.
econ.EM
Submitted: 2022-04-01
Updated: 2026-11-10
Comments: JEL codes: C15, C57, C61
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 92/100
The gist: Finite-sample inference in incomplete models proposes confidence regions for parameters of incomplete models with exact coverage in finite samples Our proposed procedure is based on a new test…
Key concepts
- Incomplete Structural Models
- These models are incomplete because the model structure allows for multiple possible outcomes due to factors like multiple equilibria in games or unobserved heterogeneity. Incompleteness means that more than one model parameter value could explain the observed data, leading to partial identification.
- Optimal-Transport Characterization
- This is a method used to construct a test statistic by solving a discrete optimal-transport problem, which is a specific type of linear programming problem. It characterizes how well the observed data aligns with the model structure.
- Finite-Sample Validity and Exactness
- The proposed confidence region has exact coverage in finite samples because the test statistic and its simulated counterpart first-order stochastically dominate each other. This means that for sharp identified parameters, a true data-generating process can be constructed where the coverage probability is guaranteed to meet the nominal level.
Terminology
Summary
Finite-sample inference in incomplete models proposes confidence regions for parameters of incomplete models with exact coverage in finite samples Our proposed procedure is based on a new test statistic that generalizes Monte Carlo tests to incomplete models and relies on simulation draws from the distribution of latent variables. p. 2
Theoretical Framework
Incomplete structural models are called incomplete because the model structure predicts a set of possible values for the outcome variables, and incompleteness arises because of multiple equilibria in games, unobserved heterogeneity in choice sets in limited attention models, interval predictions in auctions, and unknown sample selection mechanisms. The procedure proposed applies to parametric incomplete structural models where both the support restriction and the distribution of unobserved heterogeneity are known up to a finite-dimensional parameter vector. p. 1 Model incompleteness generally leads to partial identification, where more than one value of the model parameter could have given rise to the true data-generating process for the observed variables. p. 1
Inference Methodology
The authors propose an alternative method that circumvents many moments and conditioning issues by constructing a confidence region by inverting a test. The test statistic is based on an optimal-transport characterization of data/model compatibility, which is the solution of a discrete optimal-transport problem, a special kind of linear programming problem. p. 2 The test statistic and critical values rely on simulation draws from the known conditional distribution of latent variables. p. 2
Finite-Sample Validity and Exactness
The proposed confidence region has finite-sample valid coverage because both the test statistic and the simulated test statistic first order stochastically dominate each other. Exact finite-sample inference is achieved because a data-generating process can be constructed under which both statistics have the same distribution. p. 4 Theorem 1 states that for all parameters in the sharp identified region, inf P(n)∈P(n)theta P(n)(Tn(theta) ≤ cn,1−alpha(theta)) ≥ 1 −alpha. p. 1
Computational Advantages and Consistency
The procedure offers several advantages over existing methods, including avoiding reliance on asymptotic approximations and user-chosen tuning parameters. The parameter-free critical value allows for a fast preliminary search in the parameter space, and the outer confidence region can be computed with a test statistic independent of the parameter value. p. 0 Consistency results show that as sample size increases, the test eventually rejects all parameter values that violate implications of correct structural model specification. p. 3
Empirical Illustration and Performance
The methodology is applied to the model of airline entry and price competition in Ciliberto et al. [2021]. Simulation evidence shows that the proposed procedure dominates competing inference strategies like CCK and AS in terms of size, power, and computing time. p. 4 The simulation results indicate that coverage probabilities are close to nominal levels in all cases and all sample sizes. Table 3 shows that the finite-sample-valid confidence region leads to projections that differ from those reported in Ciliberto et al. p. 4 [2021]. The discussion concludes that the procedure has many advantages, including avoiding asymptotic approximations and removing the need for transforming conditional into unconditional moment inequalities. p. 3
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Robust inference for incomplete models can be achieved by using an optimal transport formulation of a test statistic based on a discrete analogue of a new optimal-transport formulation of the structural model, which generalizes Monte Carlo tests to incomplete models.
-
The improved system can derive confidence regions for the parameters of incomplete models with exact coverage in finite samples, which is the
first procedure to derive confidence regions for the true value of the structural parameters of incomplete models that are valid and exact in finite samples.
-
The system can perform a fast preliminary search in parameter space using an alternative, more conservative yet consistent test based on a parameter-free critical value, allowing for an
exact finite-sample inference
by combining this with the main proposal. -
The system can handle models where the structural support constraint depends on the sample size or where endogenous variables are discrete, as it uses a test statistic based on a
discrete analogue of a new optimal-transport formulation of the structural model.
-
The system can provide inference in models with finite outcome spaces by using a refinement that bypasses the need for an approximation of the distribution Q U of latent variables, utilizing probability mass functions with n K support points to define the test statistic.
-
The system can conduct size control tests on observed data processes where stationarity and ergodicity assumptions are not required, as it relies on
generalized Artstein inequalities
which do not require stationarity or ergodicity of the true data-generating process for the observed outcomes. -
The system can provide an inference procedure that is robust to changes in the choice of discrepancy by comparing its performance against three competing strategies: OTE (unweighted discrepancy), OTW (weighted discrepancy in latent variable space), and OTY (weighted discrepancy on the outcome space).
Sources
- Information Based Inference in Models with Set-Valued Predictions and Misspecification
- Universal Inference for Incomplete Discrete Choice Models
Related papers
- Stationarity with Occasionally Binding Constraints
- Backtesting High-Dimensional Expected Shortfall
- Higher-Order Neyman Orthogonality in Moment-Condition Models
- Online Bayesian Inference for a Dirichlet Process Mixture of Binary Logit Models
- Debiased Conditional Independence Tests for Causal Effects
- Sharp Testable Implications of Encouragement Designs