Good Controls Gone Bad: Difference-in-Differences with Covariates

summary

The gist

Good Controls Gone Bad: Difference-in-Differences with Covariates This paper addresses a critical methodological challenge in Difference-in-Differences (DiD) analysis when covariates are included…

In short

This research introduces DID-INT to solve a major problem in Difference-in-Differences analysis when covariates are included. It developed Integrated Parallel Trends (IPT) to handle violations of common causality assumptions and successfully created an estimator that consistently identifies the Average Treatment Effect on the Treated (ATT), even when standard methods fail due to complex covariate structures.

Key concepts

Common Causality of Covariates (CCC)
This refers to how covariates relate across regions and time. The paper distinguishes between different types, such as region-invariant or two-way CCC. Violations of these assumptions are a core challenge in DiD analysis, as they can invalidate standard identification methods.
Integrated Parallel Trends (IPT)
IPT is a weaker assumption than Conditional Parallel Trends (CPT). Instead of requiring trends to be parallel across regions and time individually, IPT requires that the difference in untreated potential outcome trends between treated and control regions equals zero after integrating over the covariate distribution. This allows for covariate effects to vary.
Integrated Difference-in-Differences (DID-INT)
This is a novel estimator designed to consistently estimate the ATT even when CCC assumptions are violated, provided IPT holds. DID-INT is flexible, accommodating treatment effect heterogeneity and compositional changes in covariates while remaining robust against forbidden comparisons in staggered adoption settings.

Terminology used across episodes

This episode discusses

The paper

Good Controls Gone Bad: Difference-in-Differences with Covariates · Read on arXiv

Sunny Karim, Matthew D. Webb

Carleton University

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: We're looking at the paper: "Good Controls Gone Bad".

Warren: Here's the gist of the paper. Detailed Research Summary: Good Controls Gone Bad: Difference-in-Differences with Covariates This paper addresses a critical methodological challenge in Difference-in-Differences (DiD) analysis when covariates are included, specifically focusing on the common causality of covariates (CCC) assumption and the parallel trends assumptions required for identification.

Ruth: The central contribution is the introduction of a novel estimator, Integrated Difference-in-Differences (DID-INT), designed to consistently estimate the Average Treatment Effect on the Treated (ATT) even when CCC assumptions are violated.

Warren: Core Theoretical Framework: CCC and Parallel Trends The paper meticulously distinguishes between three types of common causality of covariates (CCC): region-invariant CCC, time-invariant CCC, and two-way CCC.

Ruth: It derives the necessary and sufficient conditions for Conditional Parallel Trends (CPT) under a linearity assumption on the untreated potential outcome model.

Title and authors: Ruth: So we've touched on the title and authors of this paper, "Good Controls Gone Bad: Difference-in-Differences with Covariates," and where it comes from. The authors are Sunny Karim and Matthew D. Webb, published in October two thousand twenty-six <ref:2412.14447#pg1>.

Warren: They’re tackling the problem that when you try to use difference-in-differences with covariates, those common causality assumptions can break down in real data scenarios.

Iris: And they're saying that instead of getting stuck on those strict assumptions, we should look at something called Integrated Parallel Trends as a more flexible alternative.

Ruth: That's right, and the main idea they push is that you don't always need the strong two-way common causality assumption to get a valid difference-in-differences estimate.

Warren: They show that there are conditions for conditional parallel trends even when the common causality assumption is violated, provided those changes in covariate effects meet certain criteria across regions and periods.

Iris: It's interesting because they derive these necessary and sufficient conditions for conditional parallel trends under a linearity assumption on the untreated potential outcome model.

Ruth: And they point out that two-way common causality is sufficient but not necessary for the second condition of conditional parallel trends to hold, which is a significant finding.

The paper's summary: Warren: To sum up what this paper is about, it introduces Integrated Difference-in-Differences, or DID-INT. This estimator is designed to consistently estimate the average treatment effect on the treated even when common causality assumptions are violated, provided integrated parallel trends holds.

Ruth: So where we were before, we focused on how they're changing the identification problem itself by suggesting this new estimator can work in more messy situations with covariates.

Iris: And they explain that Integrated Parallel Trends requires that the difference in untreated potential outcome trends between treated and control regions equals zero after you integrate over the covariate distribution of the treated region.

Warren: That integration step is key because it allows for covariate effects to vary across regions and periods, which is what lets them accommodate those common causality violations.

Ruth: This means the estimator can handle situations where treatment effects depend on covariates, or where the way covariates change over time or between regions is not uniform.

Iris: They also show that DID-INT can successfully accommodate treatment-effect heterogeneity with respect to covariates and compositional changes in those variables, which is a big win for real-world applications.

The paper's improvements: Ruth: The paper suggests several improvements over the standard difference-in-differences estimators we usually use when we add covariates. It specifically critiques the standard two way fixed effects estimator and the imputation estimator.

Warren: They point out that those older methods often fail to handle common causality violations or integrated parallel trends, and they can be susceptible to things like forbidden comparisons in staggered adoption settings.

Iris: The paper shows that the standard TWFE is generally not able to accommodate those violations, and even when it seems okay on the surface, it can be biased if you have a common covariate coefficient.

Ruth: And they show that the CS-DID estimator can avoid some of those issues with forbidden comparisons and negative weighting in staggered settings, but its ability to handle common causality is conditional on how you implement the outcome regression part.

Warren: The DID-INT estimator seems to be the main improvement because it's shown to be able to identify the average treatment effect on the treated when integrated parallel trends holds and common causality is violated.

Conclusion: Ruth: So, wrapping up, this paper on "Good Controls Gone Bad: Difference-in-Differences with Covariates" suggests that we don't need those very restrictive parallel trends assumptions to get a consistent estimate when you have covariates in your model.

Warren: The main result is the DID-INT estimator, which remains unbiased even under common causality violations, provided that integrated parallel trends holds. This gives us a much more robust tool for staggered adoption settings.

Iris: From my side, the key is that they provide a graphical assessment of integrated parallel trends based on period-toperiod differences in expected outcomes evaluated over the covariate distribution of the treated regions to check for this assumption.

Ruth: And what this means for us is that we have a way to model situations where treatment effects depend on covariates and where the covariate structure changes over time or across regions without immediately running into identification problems.

Warren: It also shows that standard estimators like TWFE can be biased even when one of the common assumptions seems to hold, highlighting why this new approach is more necessary for complex data.

Iris: So we move from needing perfect conditions to needing a weaker, but more realistic condition that still lets us estimate what we actually want to know about the treatment effect.

More episodes

← Home