How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis
econ.EM
Submitted: 2026-10-29
Updated: 2026-11-03
Comments: 35 pages, 8 figures, 4 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation…
Abstract
I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation, analyzing 1,430 staggered-adoption specifications from 116 economics papers. TWFE estimates are highly correlated with modern difference-in-differences estimates, with sign reversals in 12.0-14.9% of comparisons and greater precision for TWFE. Hierarchical models accounting for estimation noise imply estimand correlations of 91-97% and a 0.7-2.4% probability of a statistically significant TWFE estimate opposing the modern estimand's sign. These findings suggest negative weights may be a less serious problem in practice and illustrate how AI can support systematic, large-scale empirical assessments of new econometric methods.
Related papers
- Stationarity with Occasionally Binding Constraints
- Backtesting High-Dimensional Expected Shortfall
- Higher-Order Neyman Orthogonality in Moment-Condition Models
- Online Bayesian Inference for a Dirichlet Process Mixture of Binary Logit Models
- Debiased Conditional Independence Tests for Causal Effects
- Sharp Testable Implications of Encouragement Designs