Improving causal inference in interrupted time series analysis: the triple difference design
summary
The gist
Improving causal inference in interrupted time series analysis: the triple difference design formalizes the DDD-ITSA design for health research by extending multiple-group ITSA with an additional…
In short
The triple difference interrupted time series analysis (DDD-ITSA) design extends existing methods to improve causal inference in health research. It uses three groups—a treatment group, a primary control group, and a secondary control group—to isolate treatment effects. This allows researchers to 'difference out' remaining biases after standard comparisons and assess effect heterogeneity across different comparison dimensions.
Key concepts
- Triple Difference Design (DDD-ITSA)
- This design extends multiple-group ITSA by adding a second control group for comparison. It mirrors the DDD framework in econometrics, using an additional difference to remove confounding that might remain after a standard two-group comparison, capturing changes along an extra dimension.
- Model Specification
- The regression model is extended to include multiple interaction terms involving the treatment time and comparisons between the different control groups. The validity of the estimator depends on identifying assumptions that ensure the difference between two-component difference-in-differences comparisons remains constant over time in the absence of intervention.
- Identifying Assumptions
- For DDD-ITSA to yield valid results, specific assumptions must hold. Crucially, any bias affecting the primary comparison (treatment vs. first control) must also affect the comparison between the second control group and the first control group in an equivalent manner. These assumptions are essential for isolating causal effects.
- Practical Guidance
- Researchers must carefully select primary and secondary control groups based on content expertise to avoid spillover effects. Additionally, accounting for autocorrelation is vital because multiple interaction terms require high statistical resolution, and baseline balance must be assessed before interpretation.
Terminology used across episodes
This episode discusses
- Improving causal inference in interrupted time series analysis: the triple difference design · Paper Radio
The paper
Improving causal inference in interrupted time series analysis: the triple difference design · Read on arXiv
Ariel Linden, DrPH
University of California, San Francisco
Background: Interrupted time series analysis (ITSA) is widely used to evaluate health policy and intervention effects. While multiple-group ITSA (MG-ITSA) improves causal inference by incorporating a control group, residual confounding from unmeasured time-varying factors may remain. The triple-difference interrupted time series (DDD-ITSA) design extends this approach by adding a second control group to further isolate treatment effects, but it remains underutilized and lacks formal guidance. Methods: We formalize the DDD-ITSA framework, specify the regression model, define key parameters for estimating level and trend effects, and clarify interpretation of the triple-difference estimand. We illustrate the approach using a worked example evaluating California's Proposition 99 cigarette tax and its impact on per-capita cigarette sales. Results: In the example, all groups were balanced on pre-intervention level and trend. The triple-difference estimand indicated a statistically significant annual reduction of-1.76 per-capita cigarette packs in California relative to the secondary control (P = 0.020; 95 percent CI: -3.24, -0.28), consistent with results from the primary comparison. Differences between control groups were not significant. Conclusions: DDD-ITSA strengthens causal inference when two-group comparisons may be confounded by leveraging an additional control group to remove remaining biases and assess heterogeneity. Implementation is facilitated by updates to the itsa Stata package. Careful attention to control selection, baseline balance, and autocorrelation remains essential.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Improving causal inference in interrupted time series analysis".
Jane: Here's the gist of the paper. Improving causal inference in interrupted time series analysis: the triple difference design formalizes the DDD-ITSA design for health research by extending multiple-group ITSA with an additional control group to isolate treatment effects, yet the design remains underutilized and lacks formal methodological guidance<ref:2603.17281#pg2> The Core Design and Motivation The triple difference interrupted time series (DDD-ITSA) design extends the logic of the MG-ITSA by adding a second control group for comparison<ref:2603.17281#pg4> This approach mirrors the difference-in-differences-in-differences (DDD) framework in the econometrics literature, where an additional difference is used to remove remaining confounding that may persist after a standard two-group comparison<ref:2603.17281#pg4> The DDD-ITSA design captures whether the post-intervention change in level and/or trend for the treated group differs along an additional dimension of comparison (such as between groups or comparison dimensions within a population or relative to an additional control group) beyond what would be expected based on concurrent changes in the primary comparison groups<ref:2603.17281#pg4> Model Specification and Identification The DDD-ITSA regression model is an extension of the MG-ITSA
5, 7, 14: and assumes the following form<ref:2603.17281#pg8> The underlying regression model is specified as Yt = β0 + β1Tt + β2Xt + β3XtTt + β4Z1 + β5Z1Tt + β6Z1Xt + β7Z1XtTt + β8Z2 + β9Z2Tt + β10Z2Xt + (β8 represents the difference in the level (intercept) of the dependent variable between the second control group and the first control group prior to the intervention, according to Table 3<ref:2603.17281#pg9> The validity of this estimator rests on identifying assumptions that require that, in absence of the intervention, the difference between two-component differencein-differences comparisons would have remained constant over time<ref:2603.17281#pg6> Specifically, identification requires that any bias affecting the primary comparison between the treatment and first control group must also affect the comparison between the second control group and the first control group in an equivalent way<ref:2603.17281#pg6> Key Results from Application In a worked example evaluating California’s Proposition 99 cigarette tax increase on per-capita cigarette sales, all groups demonstrated balance on pre-intervention level and trend<ref:2603.17281#pg4> The triple-difference estimand revealed a statistically significant annual net reduction of-1.76 per-capita cigarette packs in California relative to the secondary control (P = 0.020; 95% CI: -3.24, -0.28), consistent with the treatment effect observed against the primary control<ref:2603.17281#pg4> Practical Guidance for Implementation Researchers face several issues with this design that require careful consideration<ref:2603.17281#pg5> Key considerations include:: The choice of primary and secondary control groups is substantively important and should be guided by content expertise and empirical balance assessments<ref:2603.17281#pg5> Spillover effects or interference between groups must be considered, as the validity relies on the assumption that the intervention does not directly or indirectly affect outcomes in the control groups<ref:2603.17281#pg5> 2: When the pool of potential controls is sufficiently large, increasing the number of units assigned to each control group should be considered to enhance precision<ref:2603.17281#pg5>: Researchers should carefully assess and account for autocorrelation, which can lead to biased standard errors and invalid statistical inference<ref:2603.17281#pg5>: Because the DDD-ITSA model includes multiple interaction terms, it requires substantially greater statistical resolution than SG-ITSA or MG-ITSA designs<ref:2603.17281#pg5>: The design is particularly well-suited when researchers suspect that two-group comparisons may be confounded by concurrent policy changes, economic shocks, or secular trends that differentially affect groups<ref:2603.17281#pg5> The DDD-ITSA design offers a useful extension of the interrupted time series framework that can improve causal inference when its identifying assumptions are plausible<ref:2603.17281#pg3> By leveraging an additional control group, researchers can difference out remaining biases and assess effect heterogeneity across groups or comparison dimensions<ref:2603.17281#pg6> The updated itsa package for Stata facilitates implementation, making this design accessible for routine application in health policy research<ref:2603.17281#pg6> Careful attention to control group selection, baseline balance, and autocorrelation remains essential for valid interpretation<ref:2603.17281#pg6> The DDD-ITSA design is particularly well-suited to several common scenarios in health services and policy research<ref:2603.17281#pg5> References
1: Donald T.
Tom: Campbell and Julian C.
Jane: Stanley.
Tom: Experimental and Quasi-Experimental Designs for Research.
Title and authors: Tom: So we're diving into the title and authors again for this paper, "Improving causal inference in interrupted time series analysis: the triple difference design." What does that actually mean for a listener who isn't reading?
Jane: It sounds super technical, right? Interrupted time series analysis is just looking at how something changes over time after an event. But the authors are proposing this "triple difference design" as a way to make those causal conclusions more reliable.
Lu: They’re taking the existing logic of multiple group ITSA and adding that second control group to isolate the treatment effect even more cleanly than before > Page three of THIS PAPER. It’s about going deeper into the comparison structure.
Meng: So, instead of just one comparison, you're comparing a treated group against a baseline, and then you’re adding another control group for comparison to figure out if the change in that second control group also tells us something useful > Page four of THIS PAPER <ref:2603.17281#pg1>.
Tom: It’s about refining the comparison structure itself to remove some of that leftover confounding noise that plagues standard two-group comparisons in time series data > Page four of THIS PAPER <ref:2603.17281#pg1>. That’s what they are aiming for.
Jane: Right, so if you have a standard comparison and you suspect there’s something else lurking in the data—maybe an unmeasured factor changing over time—this design helps you "difference out" that bias > Page six of THIS PAPER <ref:2603.17281#pg1>.
Lu: Exactly. It allows you to assess if the post-intervention change in the treated group differs along an extra dimension, maybe between different subpopulations or comparison dimensions > Page three of THIS PAPER.
Meng: From a practical standpoint, this means you can get a cleaner signal if those extra groups are truly comparable and not just adding noise because they’re totally different > Page five of THIS PAPER.
Tom: So the paper is formalizing the DDD-ITSA design, which is basically building on existing ideas but giving it a proper mathematical definition for health research use > Page one of THIS PAPER <ref:2603.17281#pg1>.
Jane: And that formalization helps researchers move past just having an idea and start applying this tool rigorously in their policy work > Page two of THIS PAPER.
Lu: It’s a step toward better causal inference by giving researchers a clear mathematical framework to follow when they want to test those more complex research questions > Page one of THIS PAPER <ref:2603.17281#pg1>.
Meng: I just hope the implementation isn't so messy that it becomes impossible to deploy this on actual live data streams > Page five of THIS PAPER.
Tom: We’ll get into the summary next, where they lay out exactly what they are trying to accomplish in plain language > Page one of THIS PAPER <ref:2603.17281#pg1>.
The paper's summary: Jane: So, we’re looking at the summary of the paper, "Improving causal inference in interrupted time series analysis: the triple difference design." What are they saying they actually achieved?
Lu: They start by saying that this design is a useful extension of the interrupted time series framework that can improve causal inference when its identifying assumptions are plausible > Page two of THIS PAPER. It’s not a magic fix, just better structure.
Tom: So, what’s the key mechanism they're focusing on? They focus on leveraging an additional control group to help them "difference out remaining biases" > Page two of THIS PAPER.
Jane: They explain that by adding this second control group, researchers can isolate the treatment effects even better than they could with just a standard two-group comparison > Page three of THIS PAPER. It relaxes the strong assumption about stable counterfactual trends within a single unit > Page three of THIS PAPER.
Lu: They align it with difference-in-differences estimators, which is great because it lets them remove secular trends and common shocks that hit all units similarly > Page three of THIS PAPER.
Meng: So they’re essentially cleaning up the data by removing those big, general shifts that affect everything at once so they can see the specific treatment effect > Page three of THIS PAPER.
Tom: That’s a good way to put it. And it lets them assess if the post-intervention change in level and trend for the treated group differs along an additional dimension > Page four of THIS PAPER <ref:2603.17281#pg1>.
Jane: So they are looking for heterogeneity across groups or comparison dimensions, not just a single average effect across everything > Page four of THIS PAPER <ref:2603.17281#pg1>.
Lu: That’s the main payoff: isolating that specific type of effect that standard methods might miss when treatment effects aren't uniform > Page three of THIS PAPER.
Meng: It means if you're looking for an effect that varies by group, this design is specifically built to help you find it > Page four of THIS PAPER <ref:2603.17281#pg1>.
Tom: So the summary boils down to: use a third comparison point to remove residual confounding and see if the effect changes across different groups or dimensions > Page two of THIS PAPER.
Jane: And that’s a very clear way to explain the core improvement for our audience > Page four of THIS PAPER <ref:2603.17281#pg1>.
The paper's improvements: Lu: The authors suggest they improve causal inference by using the triple difference estimand to assess effect heterogeneity across groups or comparison dimensions > Page six of THIS PAPER <ref:2603.17281#pg1>. That’s a key improvement for complex research questions.
Tom: It's about being able to look beyond just the main effect and see if it varies based on group characteristics > Page six of THIS PAPER <ref:2603.17281#pg1>.
Jane: They also suggest that by leveraging an additional control group can enhance precision when you have a large pool of potential controls, leading to more stable estimates across those dimensions > Page five of THIS PAPER.
Meng: And they also mention that increasing the number of units assigned to each control group should be considered for that reason, because it helps get more stable results from the comparison > Page five of THIS PAPER.
Tom: So, in short, the improvements are using that extra control group to clean up biases and improve precision by having more stable estimates across different dimensions > Page six of THIS PAPER <ref:2603.17281#pg1>.
Jane: That’s a neat way to explain it for someone who isn't reading the paper > Page five of THIS PAPER. It moves the focus from just finding one average effect to finding variation across groups or dimensions > Page six of THIS PAPER <ref:2603.17281#pg1>.
Lu: And they also point out that this design is particularly well-suited when you suspect two-group comparisons might be confounded by concurrent policy changes, economic shocks, or secular trends that affect different groups differently > Page five of THIS PAPER.
Meng: If you have a scenario where those things are happening simultaneously across different groups, this design is really the one to deploy > Page five of THIS PAPER.
Tom: So the improvements boil down to using that third comparison point to clean up biases and improve precision by having more stable estimates across different dimensions > Page six of THIS PAPER <ref:2603.17281#pg1>.
Jane: It’s about moving past just finding one average effect and seeing if it varies by group or dimension > Page six of THIS PAPER <ref:2603.17281#pg1>.
Conclusion: Tom: Alright, let's wrap up with the conclusion for this paper on "Improving causal inference in interrupted time series analysis: the triple difference design." What’s the final word?
Jane: The bottom line is that this DDD-ITSA design offers a useful extension of the interrupted time series framework that can improve causal inference when its identifying assumptions are plausible > Page six of THIS PAPER <ref:2603.17281#pg1>. It makes it accessible for routine application in health policy research > Page six of THIS PAPER <ref:2603.17281#pg1>.
Lu: And they emphasize that careful attention to control group selection, baseline balance, and autocorrelation remains essential for valid interpretation > Page six of THIS PAPER <ref:2603.17281#pg1>. You can’t just run it without checking those things first.
Meng: I think the biggest thing is making sure you don't ignore autocorrelation because that’s where all the technical problems hide when you try to deploy this on real data > Page six of THIS PAPER <ref:2603.17281#pg1>.
Tom: So, we have a solid tool here for improving causal inference by adding that extra control group to difference out residual biases and assess effect heterogeneity across groups or comparison dimensions > Page six of THIS PAPER <ref:2603.17281#pg1>.
Jane: It’s a useful addition to the health policy researcher’s toolkit because it lets them test their causal questions more robustly > Page five of THIS PAPER. We'll keep an eye on this design as researchers start using the updated itsa package for Stata to get this running > Page six of THIS PAPER <ref:2603.17281#pg1>.
Lu: The DDD-ITSA design is particularly well-suited to several common scenarios in health services and policy research > Page five of THIS PAPER. It’s a tool you need to have in your toolkit if your assumptions hold up.
Meng: And just remember that the validity ultimately depends on the plausibility of those identifying assumptions in the specific context where you apply it > Page six of THIS PAPER <ref:2603.17281#pg1>. That’s the reality for any quasi-experimental method.
Tom: So, that’s our take on this paper, "Improving causal inference in interrupted time series analysis: the triple difference design." Thanks for joining us today.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates