Sensitivity Analysis for Linear Estimators

summary

The gist

Sensitivity analysis for linear estimators proposes a novel framework that measures identification failure by comparing an observed distribution to a hypothetical target distribution that would…

In short

This framework measures identification failure for linear estimators by comparing observed data to a hypothetical target distribution that would perfectly identify the causal parameter. It provides generalized bounds for Average Potential Outcome (APO), Regression Discontinuity (RD), and instrumental variables (IV) exclusion failures, consistently yielding sharp bounds under plausible conditions.

Key concepts

Target Distribution
This is a specific distribution designed to point-identify the exact causal parameter you are interested in. The framework compares your actual observed data distribution against this ideal target. The difference between them reveals how much uncertainty or 'failure' exists in identifying the true causal effect.
Radon-Nikodym Derivative
This is a statistical measure that describes how one probability distribution relates to another. In this context, it measures the restriction on the relationship between the target population and your observed population. Analyzing its largest and smallest values helps define sharp boundaries for what can be reliably identified.
Identification Failure Bounds
These are mathematical limits that define the set of possible causal estimates you can reliably calculate. The framework uses restrictions on these derivatives to establish sharp upper bounds, showing how much uncertainty is inherent in estimating parameters like APO or CATE.

Terminology used across episodes

This episode discusses

The paper

Sensitivity Analysis for Linear Estimators · Read on arXiv

Jacob Dorn, Luther Yap

We propose a framework for building sensitivity analyses for linear estimators. We measure the degree of identification failure by the change of measure between the observed distribution and a hypothetical target distribution under which a standard estimator would identify the parameter of interest. Under L infinity bounds on that derivative, the identified set has closed-form sharp bounds that require no numerical optimization and that are computable and continuous when the derivative is unbounded, nesting Manski-type worst-case bounds and existing characterizations that require bounded changes of measure. The framework nests several selection models for average potential outcomes, yielding the established worst-case bounds of each design. The framework yields a new model family for regression discontinuity (RD) that delivers the first sensitivity analysis on the conditional average treatment effect under manipulation. We show that a bootstrap procedure yields valid confidence intervals. An empirical application shows that a modest restriction on manipulation delivers bounds far more informative than worst-case bounds.

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "Sensitivity Analysis for Linear Estimators".

Warren: Here's the gist of the paper. Sensitivity analysis for linear estimators proposes a novel framework that measures identification failure by comparing an observed distribution to a hypothetical target distribution that would identify the causal parameter of interest.

Ruth: This approach yields generalized bounds for Average Potential Outcome (APO), Regression Discontinuity (RD), and instrumental variables (IV) exclusion failure designs, consistently estimating sharp bounds under plausible conditions.

Warren: Framework Overview The proposed framework proceeds by defining a target distribution that point-identifies the causal estimand of interest.

Ruth: It then considers a structural model implying restrictions on the divergence between the target population and the observed population.

Paper summary: Ruth: We're digging deeper into how this framework works, specifically looking at its core thesis outlined in "Sensitivity Analysis for Linear Estimators" >

Warren: They are proposing a novel sensitivity analysis framework where you measure identification failure by comparing the observed distribution to a hypothetical target distribution that would point to the causal parameter of interest >

Iris: The authors claim this approach yields generalized bounds for Average Potential Outcome, Regression Discontinuity, and instrumental variables exclusion failure designs >

Ruth: They suggest that under plausible conditions, this method consistently estimates sharp bounds for these designs, even when things are a bit challenging >

Warren: What's important is that their partial identification results extend the APO context to allow even unbounded likelihood ratios, which is a big step in what they can handle >

Iris: They also offer a simple closed form characterization of the identified set, which means you get an explicit way to define what's identifiable, even with those unbounded changes in measure >

Ruth: And they link this work to the literature on sensitivity analysis for IPW estimators, suggesting it's an extension of previous ideas where you start by assuming point identification and then relax those assumptions >

Warren: It seems like they are building a unified way to handle different types of distributional restrictions that cause identification problems across several major causal inference setups >

Conclusion: Ruth: So, looking at this work by Dorn and Yap, "Sensitivity Analysis for Linear Estimators", it seems to be about developing a systematic way to quantify how far an estimator can be from being perfectly identified >

Warren: Essentially, they're giving us a sensitivity analysis that tells us the degree of identification failure by measuring the gap between what we observe and what would identify the causal parameter >

Iris: The implication for me is that it gives us a concrete way to check our models, not just assume they work well under some nice conditions >

Ruth: Right. It moves identification failure from being an abstract problem to something you can actually measure with bounds on the outcome, like Average Potential Outcome or RD designs >

Warren: And for someone who only listens to the show, it means that when you use these common estimators, you can get a bound on how much your estimate might swing based on how far your actual data is from what would be perfectly identifying >

Iris: It's about moving beyond just checking if an estimator works under standard asymptotics and instead giving us a measure of its reliability across different levels of identification failure >

Ruth: That's the gist of it. It’s a new tool for understanding the limits of our statistical tools when we apply them to real-world data on causal questions >

More episodes

← Home