How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis

arXiv:2610.39995 · econ.EM · Submitted 2026-10-29 · Read on arXiv

econ.EM

Submitted: 2026-10-29

Updated: 2026-11-03

Comments: 35 pages, 8 figures, 4 tables

License: http://creativecommons.org/licenses/by/4.0/

The gist: I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation…

Abstract

I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation, analyzing 1,430 staggered-adoption specifications from 116 economics papers. TWFE estimates are highly correlated with modern difference-in-differences estimates, with sign reversals in 12.0-14.9% of comparisons and greater precision for TWFE. Hierarchical models accounting for estimation noise imply estimand correlations of 91-97% and a 0.7-2.4% probability of a statistically significant TWFE estimate opposing the modern estimand's sign. These findings suggest negative weights may be a less serious problem in practice and illustrate how AI can support systematic, large-scale empirical assessments of new econometric methods.

Related papers