Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials
summary
The gist
In platform trials, multiple intervention arms can enter or leave the study at any time, which brings the challenge of how to control the type I error rate In a platform trial…
In short
The ADDIS-Spending procedure is a group sequential algorithm designed to control Type I error rates in online multi-stage platform trials where new intervention arms can be added at any time. It dynamically updates the significance level for each hypothesis based on decisions made about other hypotheses, offering a more flexible and efficient method than static alpha spending.
Key concepts
- Platform Trials
- These are clinical trials where multiple intervention arms can enter or leave the study at any point during the trial. This introduces complexity because the total number of possible treatments is unknown initially, making standard error control difficult.
- Family-wise Error Rate (FWER)
- This is a statistical measure that controls the probability of making at least one Type I error (false positive) across all multiple hypotheses tested simultaneously. Controlling FWER ensures that the overall risk of falsely claiming a treatment effect is kept low.
- Group Sequential ADDIS-Spending
- This is an adaptive method where the testing level (alpha) for each hypothesis changes dynamically during interim analyses. It uses a conflict set to manage hypotheses without decisions, ensuring that the error rate remains controlled even as new arms are added or data accumulates.
Terminology used across episodes
This episode discusses
- Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials · Paper Radio
- Principles of Conditionality and Layering of Error Rates with Application to Platform Trials
- Online control of the false discovery rate in biomedical research
- Asymptotic Online FWER Control for Dependent Test Statistics
The paper
Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials · Read on arXiv
Jiangyue Yao, David S. Robertson, Thomas Jaki
Faculty of Informatics and Data Science, University of Regensburg · MRC Biostatistics Unit, University of Cambridge
In platform trials, multiple intervention arms can enter or leave the study at any time, which brings the challenge of how to control the type I error rate. Online testing procedures, where a stream of hypotheses are tested over time and the total number of hypotheses is unknown, might be a viable choice for error rate control in platform trials. Interim analyses in a multi-stage design offer the opportunity of early stopping for success or futility in clinical trials by testing the same hypothesis at different time points. Some methods targeting false-discovery rate (FDR) or family-wise error rate (FWER) control for multi-stage platform trials have been proposed. However, a method which controls the FWER and allows new arms to be added at any time without prior information is yet to be established. The ADDIS-Spending (ADaptive DIScarding Alpha-spending) procedure controls the FWER in an online setting. We propose the group sequential ADDIS-Spending algorithm to enable interim analyses for each hypothesis in platform trials. The developed method can update the error level of an individual hypothesis at all (interim) analyses whenever new decisions on other hypotheses are made. We compare alternative approaches with our method in a simulation study. The proposed method yields competitive results in the conducted simulations.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials".
Jane: Here's the gist of the paper. In platform trials, multiple intervention arms can enter or leave the study at any time, which brings the challenge of how to control the type I error rate In a platform trial, multiple intervention arms can enter or leave the study at any time, which brings the challenge of how to control the type I error rate.
Tom: The ADDIS-Spending (ADaptive DIScarding Alpha-spending) procedure controls the FWER in an online setting Problem Definition and Motivation Platform trials present a challenge regarding error rate control because multiple hypotheses are tested sequentially over time, and the total number of hypotheses is unknown The paper addresses the lack of a method that controls the FWER and allows new arms to be added at any time without prior information While standard two-arm trials focus on a single intervention, platform trials allow multiple treatments to enter at any time point, introducing multiplicity problems since the total number of arms is not known initially.
Jane: The family-wise error rate (FWER) control is usually the standard requirement in a confirmatory clinical trial.
Tom: Proposed Method Development The authors propose the group sequential ADDIS-Spending algorithm to enable interim analyses for each hypothesis in platform trials This method updates the error level of an individual hypothesis at all (interim) analyses whenever new decisions on other hypotheses are made.
Title and authors: Tom: The title itself, "Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials," tells you right away it's about managing multiple treatments over time without knowing the whole set beforehand.
Jane: And they are led by researchers from Regensburg and Cambridge, who are clearly deep in the statistical weeds of how to handle sequential testing under these complex conditions.
Lu: What’s interesting is that they point out that while other papers have looked at false discovery rate control or multiplicity issues with shared control arms, this paper specifically aims for FWER control while allowing new arms to be added without prior information.
Meng: So, the core problem they are solving isn't just about testing more hypotheses; it’s about maintaining a valid error rate when the structure of the trial itself is changing dynamically.
Lalam: It sounds like they are addressing a gap where previous methods either controlled the false discovery rate or only looked at specific types of shared control, but not this general online environment where arms are added unpredictably.
The paper's summary: Tom: The authors summarize that platform trials naturally create an online multiple testing procedure because the total number of hypotheses is unknown and they are tested sequentially over time.
Jane: They emphasize that family-wise error rate control is the standard requirement for confirmatory clinical trials, so this work is important because it provides a method to achieve that under these dynamic conditions.
Lu: The proposed ADDIS-Spending procedure works by updating the error level of an individual hypothesis at every interim analysis whenever a decision about another hypothesis is made, which prevents that error rate from getting inflated at later stages.
Meng: It seems they are using a specific rule involving a conflict set to decide which hypotheses are being tested and which ones are being discarded or kept for later testing.
Lalam: So, the summary points toward an adaptive approach where the testing level for one hypothesis changes based on what has happened with others in the trial timeline.
The paper's improvements: Tom: The main improvement they suggest is moving away from static alpha spending procedures, which usually have a fixed sequence like a gamma sequence that sums to one, because that just doesn't work well when new arms are entering.
Jane: They introduce the ADDIS-Spending procedure which improves power by using the fraction of non-null hypotheses and specifically exploiting those null hypotheses that turn out to be strictly non-significant.
Lu: The paper claims that this group sequential approach is more powerful than the usual alpha spending because it constrains the alpha cost to specific hypotheses using parameters like lambda and tau, which are different from a simple fixed sequence.
Meng: I'm interested in the claim about power gains when there's a high proportion of non-null hypotheses; that suggests this method is particularly useful when you have many potential treatments to test.
Lalam: And they suggest that there should exist specific values for lambda and tau where the power of this ADDIS-Spending procedure actually beats the performance of a standard alpha spending procedure with the same sequence.
Conclusion: Tom: So, to wrap up, this paper presents the group sequential ADDIS-Spending algorithm as a flexible way to manage multiplicity in online multi-stage platform trials while ensuring FWER control.
Jane: Essentially, it’s an improvement over static methods because it adapts the error level of each hypothesis based on decisions made about other hypotheses at earlier time points.
Lu: The implication is that you can have more efficient interim analyses and better power when you are adding treatments sequentially, even when the total number of arms remains unknown initially.
Meng: From a deployment standpoint, this means you could potentially run longer or test more arms while still keeping the error rate under control without needing perfect upfront planning for every single arm.
Lalam: This whole paper on "Online Control of the Family-wise Error Rate (FWER) for Multi-stage Platform Trials" shows that adaptive methods can provide a more robust way to handle sequential hypothesis testing in these complex trial designs.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates