Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients

summary

The gist

Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients is proposed to allow for therapist learning and centre too…

In short

The paper proposes orthogonal factorial designs to separate therapist effects from intervention effects in trials of therapist-delivered treatments. It suggests randomizing therapists to patients or concealing the intervention during assignment for unbiased estimation of key variance components, leading to better evidence.

Key concepts

Orthogonal Factorial Designs
These are specific experimental structures used when testing two factors—like an intervention and a therapist—to ensure that the effects of one factor do not interfere with estimating the other. This formal approach helps separate the influence of therapists from treatments.
Randomized Block Design
This design introduces 'blocking' factors, such as time periods or different clinical centers, to account for known sources of variation. Blocking helps reduce noise in the data so that researchers can more accurately measure the true effects of the intervention and therapist.
Randomizing Therapists to Patients
The authors recommend randomizing which therapist is assigned to which patient. This method is statistically preferred because it helps achieve an unbiased estimate of how much variation comes from therapists versus how much comes from the intervention itself.

Terminology used across episodes

This episode discusses

The paper

Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients · Read on arXiv

Rebecca EA Walwyn, Rosemary A Bailey, Arpan Singh, Neil Corrigan, Steven G Gilmour

University of Leeds · University of St Andrews · King’s College London

It is recognised that treatment-related clustering should be allowed for in the sample size and analyses of individually-randomised parallel-group trials that evaluate therapist-delivered interventions such as psychotherapy. Here, interventions are a treatment factor, but therapists are not. If the aim of a trial is to separate effects of therapists from those of interventions, we propose that interventions and therapists should be regarded as two potentially interacting treatment factors (one fixed, one random) with a factorial structure. We consider the specific design where each therapist delivers each intervention (crossed therapist-intervention design), and the resulting therapist-intervention combinations are randomised to patients. We adopt a classical Design of Experiments (DoE) approach to propose a family of orthogonal factorial designs and their associated data analyses, which allow for therapist learning and centre too. We set out the associated data analyses using ANOVA and regression and report the results of a small simulation study conducted to explore the performance of the proposed randomisation methods in estimating the intervention effect and its standard error, the between-therapist variance and the between-therapist variance in the intervention effect. We conclude that more purposeful trial design has the potential to lead to better evidence on a range of complex interventions and outline areas for further methodological research.

DOI: 10.1002/sim.70813

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Orthogonal factorial designs for trials of therapist-delivered interventions".

Jane: Here's the gist of the paper. Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients is proposed to allow for therapist learning and centre too Here, interventions are a treatment factor, but therapists are not If the aim of a trial is to separate effects of therapists from those of interventions, they should be regarded as two potentially interacting treatment factors with a factorial structure The authors propose a family of orthogonal factorial experimental designs and their associated data analyses for trials of therapist-delivered interventions They outline three types of designs: completely randomised, randomised block, and multicentre randomised block factorial designs This formal approach leads to recommendations about how to randomise and analyse such trials.

Tom: Design Frameworks The paper introduces three specific orthogonal factorial experimental designs for therapist-delivered interventions, namely the completely randomised factorial design, the randomised block factorial design, and the multicentre randomised block factorial design The completely randomised factorial design considers all combinations of levels of two treatment factors, Interventions (I) and Therapists (T), with each treatment replicated a certain number of times The randomised block factorial design introduces a blocking factor B, representing categorical time periods or batches, to account for variation in when interventions commence Finally, the multicentre randomised block factorial design incorporates a centre factor C as another blocking factor to account for variations across different centres.

Jane: Statistical Analysis and Modeling The analysis employs an approach based on the Design of Experiments (DoE) methodology, utilizing Hasse diagrams to describe the randomisation structure The paper details the construction of ANOVA tables based on these diagrams, where strata represent subspaces associated with each factor For the completely randomised factorial design, the appropriate residual for testing the fixed effect of I is identified by finding the supremum of all random factors which occur below factor I in a combined Hasse diagram The regression models are presented as random coefficient models, defining variance components such as between-therapist variance and between-therapist variance in the intervention effect.

Tom: Simulation Study Findings A simulation study was conducted to explore the performance of the proposed randomisation methods on estimating key parameters like the intervention effect and its standard error The simulations compared five different randomisation methods, including our proposed method of randomising therapist-intervention combinations to patients The results indicated that the intervention effect was estimated without bias across all five randomisation methods, regardless of the size of the baseline patient covariate The between-therapist variance in the intervention effect was estimated to be about 0.15 when therapist allocation was randomized to patients The paper concludes that more purposeful trial design has the potential to lead to better evidence on a range of complex interventions The authors suggest that randomising therapists to patients (or at least concealing the intervention when assigning therapists to patients) is preferable for unbiased estimation of the between-therapist variance and between-therapist variance in the intervention effect They recommend that the primary data analysis should fully reflect the randomisation, including blocking by time Furthermore, they suggest that treatment contrasts are orthogonal to estimate main effects and interactions.

Paper summary: Tom: Okay, so to summarize what we've heard about "Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients", the authors are proposing a formal way to design these trials. They're suggesting that instead of just looking at the intervention, you need to consider both the intervention and the therapist as potentially interacting treatment factors with a factorial structure.

Jane: So, they aren't just looking at one thing; they are setting up a system where you can isolate what the therapy does from what the therapist does. They introduce three specific types of designs: completely randomized, randomized block, and multicentre randomized block factorial designs.

Lu: The completely randomized factorial design considers all combinations of levels for both factors, interventions and therapists, with each combination being replicated a certain number of times. It’s a broad starting point for setting up the structure.

Meng: And then you have the randomized block design where they introduce a blocking factor representing categorical time periods or batches to account for when interventions start happening. That’s more realistic than just one big trial, right?

Lalam: And finally, there's the multicentre randomized block factorial design which adds another blocking factor for variations across different centers. It’s like they are accounting for where the trial is happening geographically or institutionally.

Tom: The statistical analysis they use involves a Design of Experiments methodology, using Hasse diagrams to describe the randomization structure, and then constructing ANOVA tables based on those diagrams. They even identify specific residual values to test the fixed effect of the intervention.

Jane: That sounds very technical for someone who isn't in statistics. What does that actually mean for us when we think about how they analyze this data?

Lu: It means they are using these diagrams to visualize exactly how things are randomized, which helps them build models, like random coefficient models, that define variance components like the between-therapist variance in the intervention effect.

Meng: So if I were running this, I’d want to know what those variance components look like so I can budget for the necessary sample size. What are they saying about those estimates?

Lalam: A simulation study they did explored how these randomization methods perform when estimating key parameters like the intervention effect and its standard error. They found that the intervention effect was estimated without bias across five different randomization methods they tested, no matter how big the baseline patient covariate was.

Conclusion: Tom: So to wrap up on this paper, "Orthogonal factorial designs for trials of therapist-delivered interventions: Randomising intervention-therapist combinations to patients", the main implication is that more purposeful trial design can lead to better evidence on complex interventions. The authors strongly recommend randomizing therapists to patients directly, or at least concealing the intervention when assigning therapists because they think this helps get an unbiased estimate of the between-therapist variance and between-therapist variance in the intervention effect.

Jane: That sounds like a practical suggestion for people running these kinds of studies. It’s telling us that how you set up the randomization sequence has a real impact on what you can actually measure about therapist effects versus intervention effects.

Lu: The authors are suggesting that treatment contrasts should be orthogonal to estimate main effects and interactions in the regression models, which is a specific statistical tool for making sure the model estimates those things correctly.

Meng: From my side, I see this as a way to make sure we aren't just seeing noise when we look at therapist variation. If we can properly block for time and centers, that gives us more clean data on the intervention effect itself.

Lalam: It’s about making sure the statistical analysis fully reflects the randomization you did, including blocking by time, which is a key recommendation from this paper.

Tom: So in simple terms, they are pushing for a more structured way to randomize things so we can get clearer evidence on how much the therapist matters versus how much the therapy itself matters. That’s what this paper is all about.

More episodes

← Home