Inference After Model Refutation: Uniform Post-Relaxation Inference for Partially Identified Models

summary

The gist

When data reject a model, researchers must decide which identifying assumptions to discard.

In short

The Post-Relaxation Projection (PR) addresses uncertainty when data reject a model by simultaneously considering every possible minimal repair. PR captures targets compatible with a band defined by repair-specific restrictions, ensuring that even if the best repair is unknown, the resulting inference covers all structurally supported values across all plausible repairs.

Key concepts

Post-Relaxation Projection (PR)
A method that handles model rejection by carrying one simultaneous confidence event through every minimal repair. It captures exactly those post-relaxation targets compatible with a band defined by the restrictions of each individual repair, ensuring robustness when the best repair is unknown.
$R^ ext{*} (P)$
A discrete population object learned from the same data used for inference. Selecting a specific repair and then inferring can fail if that selection step deletes a component of the target set. PR avoids this selection by retaining every repair that could still be a population-minimum on an event, ensuring all supported values are kept.
Bandwise Completeness ($C_{PR}^n$)
This establishes that the canonical PR method covers all necessary targets within a specific band defined by the repair menu. If any set $D$ contains the targets for every primitive function in the band, it must contain $C_{PR}^n$, providing a guarantee of comprehensive coverage under certain conditions.

Terminology used across episodes

This episode discusses

The paper

Inference After Model Refutation: Uniform Post-Relaxation Inference for Partially Identified Models · Read on arXiv

Ulrich Hounyo

University at Albany, State University of New York

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Inference After Model Refutation".

Warren: Here's the gist of the paper. When data reject a model, researchers must decide which identifying assumptions to discard.

Ruth: If multiple repairs restore compatibility, and the repair itself is unknown, inference under any single repair might exclude structural values supported by others.

Warren: The proposed Post-Relaxation Projection (PR) addresses this by carrying one simultaneous confidence event through every minimal repair.

Ruth: For a rectangular band defined on the repair-specific restrictions, PR captures exactly those post-relaxation targets compatible with that band.

Title and authors: Ruth: So we’re back to the title, "Inference After Model Refutation: Uniform Post-Relaxation Inference for Partially Identified Models." It sets up the whole problem we just discussed about model rejection and multiple repairs.

Warren: It means the paper is focused on figuring out how to do inference when you have several equally good ways to fix your model, but you don't know which one is actually right.

Iris: The focus seems to be on creating a uniform way to make inferences that works across all those possible repairs instead of just picking one.

Ruth: Right. It’s not just about finding *a* valid conclusion; it’s about making sure the conclusion holds regardless of which repair you happened to select from the menu.

Warren: The authors are proposing this Post-Relaxation Projection method as a way to handle that uncertainty systematically, carrying one simultaneous confidence event through every minimal repair.

Iris: And they build a rectangular band on the restrictions specific to each repair, and this captures precisely what targets are compatible with that band.

Ruth: The difficulty they highlight is learning the set of population minimum repairs from the data itself, because that’s a discrete object you have to learn from the same sample.

Warren: That selection step can delete crucial components of your target even if the next step is exact, so they are worried about that failure mode.

Iris: So they introduce three criteria—repair recovery concerns, entire-target coverage concerns, and structural-value coverage concerns—to guide their approach to avoiding that failure.

Ruth: And PR specifically targets the entire-target coverage concern first, which seems like the most comprehensive way to ensure you don't miss anything.

The paper's summary: Warren: To summarize what they did, they developed Post-Relaxation Projection to handle model refutation by considering all minimal repairs simultaneously.

Iris: They show that PR carries one simultaneous confidence event through each repair and builds a rectangular band based on the specific restrictions of that repair.

Ruth: This method captures exactly the post-relaxation targets compatible with that band, which is key when you have multiple candidates for model fixes.

Warren: They also derived limits on learning the repair itself, its exact perturbation radius, oracle equivalent adaptation, and versions for instrumental variables without a first-stage relevance bound.

Iris: This is important because it addresses the question of how much information you need about the repair to perform inference reliably.

Ruth: It means they’ve also found limits on learning the repair, how big its error can be, and what kind of adaptation you need for your instrumental variables.

Warren: They are looking at this in cross-country data on institutions and income where the data don't settle which historical instrument to drop yet every admissible minimum repair implies a positive effect.

Iris: This is the scenario where multiple repairs support different structural parameters, making it hard to decide which one you should trust.

Ruth: So, the paper essentially says that even when the data doesn't tell you which instrument to drop, every viable minimum repair still suggests a positive effect survives sizable direct effects of that instrument.

The paper's improvements: Warren: The authors suggest several improvements to this initial approach to make it more robust and practical in real-world settings.

Iris: They propose Resolution-Adaptive Inference, which uses an oracle procedure for a certified layer and Post-Relaxation Projection otherwise.

Ruth: So when the data gives you a high resolution on feasibility bounds, the system can switch to that exact oracle inference instead of the more conservative projection.

Warren: When uncertainty is high, like with low resolution, it switches over to this more conservative, data-driven projection method.

Iris: This allows them to adapt their confidence procedures based on how statistically resolved their feasibility bounds are using the oracle procedure for a certified layer and PR otherwise.

Ruth: It sounds like a smart way to manage uncertainty—using the best tool available depending on what the data can actually tell you.

Warren: It also discusses uniform reduced-form inference without a relevance bound, allowing estimation even when first stages are weak.

Iris: This means they can get a set that is uniformly valid without any lower bound on the magnitude of p i j including cases where p i j equals zero.

Ruth: That prevents the system from getting overly conservative just because it’s unsure about how relevant an instrument is.

Conclusion: Warren: So to wrap up, the main implication is that this paper provides a uniform way to get inferences when models are rejected by data and multiple repairs are possible.

Iris: It shows that even with uncertainty about which repair you pick, you can still establish a robust conclusion because every admissible minimum repair implies something.

Ruth: The Post-Relaxation Projection method gives us an interval conclusion that is robust to uncertainty over the menu of repairs.

Warren: The RST data they use demonstrate this idea, showing that the conclusion holds under either one of those two possibilities.

Iris: It really shows how institutions matter for income under every repair that the data cannot rule out, and this minimum effect is economically large when instruments are held close to exact validity.

Ruth: The final word on "Inference After Model Refutation: Uniform Post-Relaxation Inference for Partially Identified Models" is that it’s a method for robust inference in these tough identification scenarios.

More episodes

← Home