Debiased Conditional Independence Tests for Causal Effects

arXiv:2611.05608 · econ.EM · Submitted 2026-11-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Debiased Conditional Independence Tests for Causal Effects".

Warren: Here's the gist of the paper. The paper proposes robust tests for causal effects on potentially high-dimensional outcomes where selection into treatment may occur, specifically designed to measure conditional independence between outcomes and treatment assignment by determining whether treatment can be predicted from outcomes more accurately than from covariates alone.

Ruth: The core null hypothesis is that the distributions of potential outcomes are equal under treated and control states: H 0: Y i(1) d= Y i(0) X i (1) almost surely, which is equivalent to testing conditional independence: H 0: D i Y i X 2 almost surely, or no predictability: H 0: P(D i = 1Y i, X) = P(D i = 1X) almost surely.

Warren: Three primary procedures are introduced to test this conditional independence: 1.

Ruth: A conditional randomization test based on an inverse propensity weighted mean squared error (IPW) test statistic, assuming propensity scores are known.

Paper summary: Ruth: So we're looking at this paper now called Debiased Conditional Independence Tests for Causal Effects. It addresses those tricky situations where you have high-dimensional outcomes and selection into treatment is possible.

Warren: Exactly. The core idea here is to measure if the treatment assignment can be predicted from the outcomes more accurately than just using the covariates alone. That's what they’re testing for conditional independence between outcomes and treatment assignment.

Iris: So, to put that in simpler terms, the null hypothesis they are testing is that the distributions of potential outcomes are equal under treated and control states given those covariates. That means if you know the outcomes and the covariates, you can't predict whether someone got treated better than someone who didn't get treated based on those outcomes.

Ruth: Right. They frame it as testing conditional independence, specifically checking if treatment assignment is independent of the outcomes when conditioned on the covariates. It’s basically asking if getting a specific outcome tells you something new about whether you were assigned that treatment or not beyond what the covariates already tell us.

Warren: And they propose three main procedures to test this conditional independence. First, there's a conditional randomization test based on an inverse propensity weighted mean squared error test statistic, but that assumes we already know the propensity scores.

Iris: That first one is interesting because it controls the size in finite samples if you actually know those propensity scores. Then they have a double-residualization approach, which involves partially removing the covariate effects from both the treatment predictions and the outcome predictions and checking if those remaining residuals are uncorrelated.

Ruth: And that second approach is built around testing whether the correlation between those residuals is zero. That's a way to see if there's any left over information that isn't captured by just looking at the covariates.

Warren: Finally, they have this third test based on an augmented inverse propensity weighted estimator, which uses doubly robust, Neyman-orthogonal population moments that are necessary and sufficient for that conditional independence.

Iris: That third one is built around these moments being necessary and sufficient for conditional independence. It’s a technical way to check the required structure of the distributions under the null hypothesis.

Ruth: The paper claims these procedures are designed to test for the joint distributional causal effect of treatment on outcomes under unconfoundedness, which means we're assuming no selection into treatment based on those potential outcomes is happening.

Paper summary: Warren: They claim these methods inherit nice properties like Neyman orthogonality and double robustness when the null hypothesis holds true. That’s important because it means they should behave well in large samples and even if our models are slightly wrong.

Iris: And a big practical advantage they point to is computational efficiency, which allows for generic machine learning applications and only requires estimating a small number of nuisance functions even in high-dimensional settings.

Ruth: When propensity scores have to be estimated from observational data, they construct three specific statistics: Augmented inverse propensity weighting, double residualization DR, and double residualization with augmentation DR star.

Warren: These three statistics are based on efficient, Neyman-orthogonal, doubly robust scores that measure the predictability of treatment from outcomes conditional on covariates. They test for this conditional independence between outcomes and treatment assignment.

Iris: The paper shows these tests achieve asymptotic normality under assumption four, and corollary one confirms they reach the desired asymptotic size alpha when we are testing the null hypothesis.

Ruth: The results show that AIPW, DR, and DR star tests are systematically more powerful than the conventional joint Wald test when we're in low-signal regimes. They also outperform the joint Wald test when there's covariate-driven heterogeneity or a nonzero mean effect present.

Warren: But they are consistent for the true conditional treatment effect even if our outcome model is misspecified. The AIPW, however, needs those propensity scores to be known beforehand.

Iris: That’s a key distinction: AIPW requires you to know the propensity scores, while DR and DR star can handle observational settings where you have to estimate those scores.

Ruth: The paper details how these statistics converge under assumption four. Part one shows that the estimators for AIPW, DR star, and DR all converge in probability to their respective null values under assumption four.

Warren: Part two establishes the consistency of the variance estimators, showing they converge to the asymptotic variances derived from theorem five, which depend on terms involving nuisance function errors that vanish under assumption four.

Iris: And part three proves central limit theorems for these test statistics. For AIPW and DR star, we get a distribution that converges to a normal distribution with mean zero and a specific variance term as n goes to infinity.

Paper summary: Ruth: Crucially, under the null hypothesis, the estimated value of these estimators goes to zero almost surely for all three tests. This means the test statistics converge in distribution to a standard normal distribution with mean zero and variance one.

Warren: That’s powerful because it confirms that all three proposed tests have asymptotically correct sizes when nothing is happening. They also confirm the asymptotic size alpha under the null hypothesis in part four of their analysis.

Iris: Under the alternative, if our true outcome model is well-specified, consistency means power converges to one when both parameters related to the treatment effect and outcomes are positive. If we misspecify the model, consistency still holds as long as those parameters stay positive.

Ruth: Comparing this to other tests, AIPW, DR, and DR star are systematically more powerful than the conventional joint Wald test in low-signal regimes. They also outperform the Wald test when there's covariate-driven heterogeneity or a nonzero mean effect present.

Warren: The paper benchmarks show that AIPW, DR, and DR star tests can achieve consistently high power across different outcome distributions, reaching near one hundred percent even with as few as five hundred observations. The Wald test stays below twenty percent in complex designs like Mixtures and Heavy Tails.

Iris: But the small sample performance shows that while DR/DR star and AIPW dominate when the treatment effect averages to zero, other tests like KCI, GCM, and the joint Wald test dominate when that average treatment effect is not zero.

Ruth: So, wrapping up this paper titled Debiased Conditional Independence Tests for Causal Effects. The authors are Susan Athey, Maria Nareklishvili, and Jann Spiess. They propose these three tests to robustly measure conditional independence between outcomes and treatment assignment in high-dimensional settings with selection into treatment possible.

Warren: Essentially, they are providing tools to see if the treatment can actually be predicted from the outcomes better than just looking at the covariates alone. The implication is that we have a more reliable way to assess the joint distributional causal effect of an intervention under unconfoundedness.

Iris: The paper suggests these methods are very useful because they offer good properties like double robustness and Neyman orthogonality when they hold true, which means they should work reliably even if our functional forms are a bit off.

Ruth: And the practical side is that for observational data where propensity scores need estimation, the DR and DR star tests provide a way to do it consistently. It’s about making causal inference more accessible when the data is messy.

Conclusion: Ruth: So we're wrapping up on "Debiased Conditional Independence Tests for Causal Effects." This paper is about building these three tests to check if treatment assignment is actually predictable from the outcomes, beyond just using the covariates.

Warren: Right, so they're not just looking at correlation; they're trying to isolate that specific causal effect by measuring conditional independence between those two things. It’s about seeing if you can predict getting treated better based on what happens to your outcome.

Iris: What I find important is how they handle the messy part—when you don't know the propensity scores, which is almost always the case in real observational data. They give us these three specific tools, AIPW, DR, and DR star, which are designed to work even when we have to estimate those scores.

Ruth: And what this means for us is that we’ve got a much more robust way of assessing the joint causal effect when selection into treatment might be happening in high-dimensional settings. It gives us more confidence than just running the standard regression tests.

Warren: The real power here is that these methods keep good properties like double robustness even when our models are slightly wrong about the outcome structure. That’s huge for practical applications, because we don't always have perfect data or perfect models.

Iris: And they show that under certain conditions, like when the true treatment effect is positive and the outcome model is pretty good, these tests can actually get really powerful with as few as five hundred observations. That’s a big deal for getting reliable estimates in smaller studies.

Ruth: Exactly, so it shifts our approach from just testing if a relationship exists to testing if that relationship has causal meaning under those complex conditions. But we still have to be careful about what these tests can and cannot do, and where they might stop working.

Stanford University · University of Glasgow

econ.EM

Submitted: 2026-11-03

Updated: 2026-11-15

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 100/100

The gist: The paper proposes robust tests for causal effects on potentially high-dimensional outcomes where selection into treatment may occur…

Key concepts

Conditional Independence Null Hypothesis
This hypothesis assumes that the distribution of potential outcomes under treated and control states is identical when conditioned on covariates. In simpler terms, it tests whether treatment assignment can be predicted from the outcomes more accurately than just using the covariates X.
Augmented Inverse Propensity Weighting (AIPW)
This statistic is used when propensity scores are known. It measures the predictability of treatment based on outcomes, adjusting for covariates and treating both as predictors. It is efficient and provides strong statistical guarantees under certain assumptions.
Double Residualization (DR) / DR∗
These approaches involve partial out-covariate effects from both the treatment and outcome predictions. They test if the correlation between these residuals is zero, which signifies conditional independence. They are useful for observational data where propensity scores need to be estimated.
Double Robustness
This desirable statistical property means that a test remains valid even if there is misspecification in one of the models used (e.g., the outcome model). It ensures that if either the treatment or outcome model is correct, the test still provides a consistent estimate of the causal effect.

Terminology

Summary

The paper proposes robust tests for causal effects on potentially high-dimensional outcomes where selection into treatment may occur, specifically designed to measure conditional independence between outcomes and treatment assignment by determining whether treatment can be predicted from outcomes more accurately than from covariates alone. p. 0 The core null hypothesis is that the distributions of potential outcomes are equal under treated and control states: H 0: Y i(1) d= Y i(0) X i (1) almost surely, which is equivalent to testing conditional independence: H 0: D i Y i X 2 almost surely, or no predictability: H 0: P(D i = 1Y i, X) = P(D i = 1X) almost surely. p. 7

Three primary procedures are introduced to test this conditional independence:

  1. A conditional randomization test based on an inverse propensity weighted mean squared error (IPW) test statistic, assuming propensity scores are known. p. 0

  2. A double-residualization approach, which involves partial out covariate effects from both treatment and outcome-based treatment predictions and tests whether the correlation between the residuals is zero. p. 0

  3. A test based on an augmented inverse propensity weighted (AIPW) estimator built around doubly robust, Neyman-orthogonal population moments necessary and sufficient for conditional independence. p. 0

These proposed measures are designed to test for the joint distributional causal effect of treatment on outcomes under unconfoundedness (Assumption 1). p. 2 They inherit desirable large-sample properties, including Neyman orthogonality and double robustness under the null hypothesis. p. 6 The primary advantage is computational efficiency, allowing for generic machine learning applications and requiring estimation of only a small number of nuisance functions even in high-dimensional settings. p. 1

When propensity scores must be estimated from observational data, three specific statistics are constructed: Augmented inverse propensity weighting (AIPW), double residualization (DR), and double residualization with augmentation (DR). p. 6 These statistics are based on efficient, Neyman-orthogonal, doubly robust scores that measure the predictability of treatment D i from outcomes Y i conditional on covariates X i. p. 4 The tests achieve asymptotic normality under Assumption 4, and Corollary 1 confirms they attain the desired asymptotic size alpha. p. 4

The results demonstrate that AIPW, DR, and DR tests are systematically more powerful than the conventional joint Wald test in low-signal regimes. p. 4 Furthermore, these methods outperform the joint Wald test when covariate-driven heterogeneity or nonzero mean effects are present. p. 0 They are consistent for the true conditional treatment effect even under misspecification of the outcome model. p. 1 The AIPW requires known propensity scores, whereas DR and DR accommodate observational settings where propensity scores must be estimated. p. 3

The analysis details convergence properties:

Part 1 shows that theta hat AIPW theta f0, theta hat DR theta f0, and theta hat DR theta f0 under Assumption 4.

Part 2 establishes the consistency of the variance estimators, showing that the estimated variances converge in probability to the asymptotic variances derived from Theorem 5, which depend on terms involving nuisance function errors that vanish under Assumption 4. p. 4

Part 3 proves Central Limit Theorems (CLTs) for the test statistics T k n. p. 4 For AIPW and DR, sqrt n theta hat k - theta k f N(0, sigma 2 k + delta 2), where sigma 2 k are the asymptotic variances. p. 3 Crucially, under the null hypothesis, theta hat k f = 0 a. p. 1s., leading to T k n N(0, 1) for all three tests (k in AIPW, DR, DR). p. 0

Part 4 confirms asymptotic size alpha under the null. p. 4 Under the alternative, if the true outcome model is well-specified (f tilde 0 = f 0), consistency (power converging to one) is achieved when theta f 0 > 0 and theta f 0 > 0. p. 1 If the true model is misspecified, consistency requires these parameters to remain positive. p. 1

In comparison, the Joint Wald Test tests H 0: tau = 0 in a seemingly unrelated regression setting. p. 0 Performance comparisons show that AIPW, DR, and DR tests achieve consistently high power across various outcome distributions (Normal and Mixtures), reaching near 100% even with as few as 500 observations, whereas the Wald test remains below 20% in complex designs like Mixtures and Heavy Tails. p. 0 Small sample performance benchmarks indicate that while DR/DR and AIPW dominate when the treatment effect averages to zero (E[tau(X i)] = 0), KCI, GCM, and the joint Wald test dominate when E[tau(X i)] not equal to 0. p. 4

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Conditional independence testing for causal effects can be used as a crucial first-stage diagnostic for estimating treatment effects by determining whether outcomes are informative about treatment assignment after controlling for observed confounders.

  2. The proposed tests allow for generic machine learning and only require estimation of a small number of nuisance functions, even when both outcomes and covariates are high-dimensional, which improves computational efficiency compared to methods requiring quadratic or cubic time complexity.

  3. The augmented inverse propensity weighted estimator (AIPW) test statistic is doubly robust under the null hypothesis, meaning that its asymptotic distribution remains normal even if the outcome model or propensity score is misspecified, ensuring size control in finite samples regardless of estimation errors in nuisance functions.

  4. The double residualization approach provides a way to measure the causal link between treatment and the treatment probability predicted from outcomes by removing the set of confounders, which is robust to limited overlap between treated and control units compared to simpler estimators.

  5. The tests can be applied in settings where propensity scores must be estimated from data by using efficient, Neyman-orthogonal, doubly robust scores such as augmented inverse propensity weighting (AIPW) and double residualization (DR).

  6. The Monte Carlo simulations demonstrate that the proposed conditional independence tests significantly outperform the joint Wald test in designs where the unconditional treatment effect is small or zero but covariate-driven heterogeneity remains, maintaining non-trivial power even under weak signal-to-noise ratios.

Sources

Related papers