Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
summary
The gist
Subgroup analyses within randomized controlled trials are often underpowered due to limited sample sizes…
In short
This research proposes two advanced statistical methods, TMLE-PR and A-TMLE, to improve subgroup analysis in randomized controlled trials when sample sizes are small. These estimators use participants outside the subgroup to borrow information from the main trial data, enhancing precision without needing external real-world data.
Key concepts
- Targeted Maximum Likelihood Estimators (TMLE)
- A statistical method used to estimate a specific parameter of interest while incorporating auxiliary information. It works by fitting an initial model and then iteratively refining the estimate to target a desired outcome, ensuring that the final result is focused precisely on the subgroup effect.
- TMLE with Pooled Regression (TMLE-PR)
- This estimator uses a preliminary regression based on all trial participants to build an initial model. It then evaluates this model specifically for the subgroup of interest before applying TMLE to target the desired subgroup-specific treatment effect, leveraging information from everyone in the trial.
- Adaptive Targeted Maximum Likelihood Estimator (A-TMLE)
- This method decomposes the target effect into a pooled part and a bias part. It adaptively adjusts how much information is borrowed from non-subgroup participants based on the size and complexity of that bias term, aiming for greater efficiency than simpler methods.
- Subgroup Membership Mechanism
- This refers to the underlying process or model used to determine which patients belong to a specific subgroup. Since this mechanism is not directly observed in the data, accurately estimating it is crucial for correctly identifying and analyzing subgroup-specific treatment effects.
Terminology used across episodes
This episode discusses
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE · Paper Radio
- Data fusion for efficiency gain in ATE estimation: A practical review with simulations
- Highly Adaptive Empirical Risk Minimization with Principal Components
- On Weighted Orthogonal Learners for Heterogeneous Treatment Effects
- An Estimator-Robust Design for Augmenting Randomized Controlled Trials with External Real-World Data
- Lassoed Tree Boosting
- Adaptive debiased machine learning using data-driven model selection techniques
- Highly Adaptive Principal Component Regression
- Unified implementation and comparison of Bayesian shrinkage methods for treatment effect estimation in subgroups
- Investigating Targeting Strategies and Truncation in TMLE for the Average Treatment Effect under Practical Positivity Violations
The paper
Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE · Read on arXiv
Sky Qiu, Nerissa Nance, Rachael Phillips, Jens Tarp, Maya Petersen, Mark van der Laan
University of California, Berkeley
Subgroup analyses within randomized controlled trials are often underpowered due to limited sample sizes. We address this challenge by leveraging trial participants outside the subgroup of interest to augment estimation within the subgroup. Specifically, we study two Targeted Maximum Likelihood Estimators (TMLEs) that borrow information from non-subgroup participants within the same trial: a TMLE with pooled regression (TMLE-PR) and an Adaptive Targeted Maximum Likelihood Estimator (A-TMLE). Both estimators enable information sharing without relying on any external real-world data, thereby capitalizing on key strengths of the trial: most importantly, the protection against bias afforded by the randomized treatment, but also harmonized data collection, and consistent treatment and outcome definitions. The general strategy proposed here directly advances the priorities of key regulatory agencies, including the FDA, by improving the precision of subgroup-specific treatment effect estimates without introducing external sources of bias, thereby facilitating rigorous inference to support equitable labeling, access, and post-market evaluation. In a case study based on analysis of data from a cardiovascular outcome trial (LEADER, NCT01179048), we estimate the risk reduction of major adverse cardiac events (MACE) under liraglutide treatment among Black and Asian subgroups -- each comprising less than 10% of the trial population -- using the proposed estimators that borrow information from the remainder of the trial. Using A-TMLE, in particular, we find estimated absolute MACE risk reductions of 1.6, 1.5, and 1.5 percentage points among Asian participants and 2.1, 2.0, and 2.1 percentage points among Black participants at 365, 540, and 730 days, respectively, with 95% confidence intervals excluding the null at each time point.
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE".
Jane: Here's the gist of the paper. Subgroup analyses within randomized controlled trials are often underpowered due to limited sample sizes, and this paper addresses this challenge by leveraging trial participants outside the subgroup of interest to augment estimation within the subgroup Specifically, we study two Targeted Maximum Likelihood Estimators (TMLEs) that borrow information from non-subgroup participants within the same trial: a TMLE with pooled regression (TMLE-PR) and an Adaptive Targeted Maximum Likelihood Estimator (A-TMLE) Both estimators enable information sharing without relying on any external real-world data, thereby capitalizing on key strengths of the trial: most importantly, the protection against bias afforded by the randomized treatment, but also harmonized data collection, and consistent treatment and outcome definitions The general strategy proposed here directly advances the priorities of key regulatory agencies, including the FDA, by improving the precision of subgroup-specific treatment effect estimates without introducing external sources of bias Problem Setup and Causal Estimand Estimating treatment effects within specific patient subgroups in randomized controlled trials (RCTs) is important for characterizing subgroup-specific average treatment effects, which may differ from the population average treatment effect (Kravitz et al., 2004; Kent et al., 2010) However, RCTs often lack sufficient representation of certain subpopulations of interest, limiting the statistical power of subgroup-specific analyses (Wang et al., 2021) When restricting attention to RCT data alone, one approach to conduct subgroup analysis is to consider whether information from patients outside the subgroup, yet still enrolled in the trial, can be leveraged to improve estimation of the subgroup-specific treatment effect The idea of improving subgroup-specific treatment effect estimates by borrowing information from the broader trial population has a long history in Bayesian and empirical-Bayes subgroup analysis TMLE with Pooled Regression (TMLE-PR) Our first proposed estimator works by fitting an initial regression of the outcome on the trial arm and baseline covariates on the pooled trial sample, evaluating that regression on the subgroup sample, and then targeting the subgroup-specific estimand with a standard TMLE This estimator is referred to as TMLE with pooled regression (TMLE-PR) The scientific target remains the same subgroup-specific estimand Ψ The construction involves fitting an initial outcome regression Qn(s, w, a) of Q0(s, w, a) = E0(Y S = s, W = w, A = a) using the full RCT sample The resulting TMLE-PR estimator is then computed as ΨTMLE-PRn = Pn h S Q⋆ n(1, W, 1) − Q⋆ n(1, W, 0) i Pn(S = 1) Adaptive TMLE (A-TMLE) Our second proposed estimator is inspired by data integration methods (Shi et al., 2023; Colnet et al., 2024; Lin et al., 2024) This A-TMLE decomposes the target estimand into a pooled estimand and a bias estimand The pooled estimand Ψ(˜P) is defined as EPWS=1
Q¯P (W, 1) − Q¯P (W, 0): The bias estimand Ψ(P) is defined as Ψ(P) = Ψ(˜P) − Ψ(P), which captures precisely the difference between the pooled estimand and our target estimand Where the Gains Come From The simulation study suggests that both TMLE-PR and A-TMLE improve efficiency for estimation of subgroup effects in finite samples, compared to both conventional unadjusted estimator and semi-parametrically efficient subgroup-only estimators In particular, ATMLE tends to offer larger gain in efficiency compared to TMLE-PR in settings where baseline covariates are strongly predictive of subgroup membership The case study on the LEADER trial demonstrates that A-TMLE estimates are remarkably stable, at about a 2-percentage-point reduction at 365, 540, and 730 days in the Black or African American subgroup Practical Considerations and Limitations A key feature of A-TMLE is its decomposition Ψ(P) = Ψ(˜P) − Ψ(P), which makes explicit that the benefit of borrowing information depends on the size and complexity of the subgroup-specific bias term However, when the subgroup-bytreatment interaction is highly complex, projection bias may be non-negligible, and the efficiency gains from borrowing strength may be reduced (van der Laan et al., 2026) A related limitation concerns the current working-model selection strategy for the bias component, which currently restricts model selection to a class of working models that always include the treatment variable A Case Study Application We applied A-TMLE to estimate subgroup-specific treatment effects using data from the LEADER cardiovascular outcome trial (Marso et al., 2013, 2016) Across the first 730 days of follow-up, the subgroup analyses generally suggest that liraglutide is associated with a lower event risk than placebo in both Asian and Black subgroups The A-TMLE again suggests a modest but consistent protective effect of about 1.5 to 1.6 percentage points at each time horizon Both the simulation study and the LEADER case study suggest that this approach can improve precision, and our LEADER analysis provides subgroupspecific estimates in racial groups for which the original trial was not powered to draw definitive conclusions Future work will focus on more targeted working-model selection strategies, extensions to settings with right-censoring and more complicated data structures References Laura B Balzer, Mark J van der Laan, Maya L Petersen, and SEARCH Collaboration.
Tom: Adaptive prespecification in randomized trials with and without pair-matching.
Jane: Statistics in medicine, 35(25):4528–4545, 2016 David Benkeser and Mark van der Laan.
Tom: The highly adaptive lasso estimator.
Title and authors: Tom: So we're looking at this paper called "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE". It sounds like they're trying to solve a problem where subgroup analyses in these trials are often underpowered because there aren't enough participants in those specific groups.
Jane: That makes sense. So, the main idea here is that they want to use information from people *outside* the subgroup of interest within the same trial to get better estimates for that subgroup, without bringing in any outside data like real-world patient records.
Lu: Exactly. They're proposing two specific methods called estimators: a TMLE with pooled regression, which they call TMLE-PR, and an Adaptive Targeted Maximum Likelihood Estimator, or A-TMLE. These methods are designed to share information internally within the trial data structure itself.
Meng: Internally? So they're not using some external database of patients who didn't participate in the original trial, right? They are only looking at people already enrolled in this specific clinical trial. I like that constraint because it keeps things consistent and avoids those nasty distributional shifts we worry about with real-world data.
Lalam: Lalam here. The paper summarizes the core strategy: they use these two estimators to improve the precision of subgroup-specific treatment effect estimates, which is what regulators like the FDA are really focused on when looking at these kinds of analyses.
Tom: Right, so they're showing that by using this internal information sharing, you can get more accurate numbers for a specific subgroup compared to just looking at that small group alone or using a standard estimator.
Jane: And the paper points out how both TMLE-PR and A-TMLE work by breaking down the target estimate into two parts: one that combines info from everyone, and another part that corrects for the difference between that combined info and what you actually want for your subgroup.
Lu: That decomposition is key, especially with the A-TMLE which they describe as performing a bias correction to the pooled estimate. This lets them explicitly see how much benefit comes from borrowing strength from the rest of the trial population.
Meng: But I wonder about that bias correction part, Lu. If you're correcting for it, how much do you actually gain? Does it just complicate things with more modeling, or does it actually make a measurable difference in terms of efficiency?
Title and authors: Jane: That's a good question. The simulation study they ran suggests that both estimators are better than the standard methods when you have a limited sample size, and the A-TMLE seems to offer an even bigger boost in efficiency than the TMLE-PR in certain situations.
Tom: I saw that part about A-TMLE actually showing larger gains in efficiency when baseline covariates are very predictive of whether someone belongs to the subgroup, which makes sense because you're leveraging those known differences.
Lu: And they also found that for a specific trial, like the LEADER trial, the A-TMLE estimates were remarkably stable across different time points—about a two percentage point reduction in event risk at three hundred sixty-five days and seven hundred thirty days in the Black or African American subgroup.
Meng: Stability is good for deployment because it means you're not getting wildly different results depending on minor fluctuations in the data collection process over time, right? But what about when those baseline covariates are just weakly predictive? Does the benefit disappear then?
Jane: The paper touches on that complexity because they note that when the subgroup-by-treatment interaction is very complex, projection bias can become a problem and might reduce those efficiency gains. That's one of the limitations they flag.
Tom: So, so it's not a magic fix for every single subgroup analysis scenario. The method works best when the baseline factors are strong predictors of subgroup membership, but it gets trickier with very messy interactions.
Lu: And another thing they mention is that the current strategy for choosing the working model for that bias component is too restrictive; it always has to include the treatment variable itself, which limits how flexibly they can select what's causing that bias.
Meng: That sounds like a practical deployment hurdle. If you have to restrict your model choices just because of how you built the algorithm, it makes scaling up much harder than I'd like for production.
Jane: So, while the theory suggests strong gains in efficiency by using trial data alone to fix subgroup estimation issues, the authors are being honest about where their current methodology has limitations regarding complex interactions and model selection.
Title and authors: Tom: It really boils down to this: they're showing a way to improve precision for regulators by using what's already inside the trial data, but you have to be careful about how you apply it because of those complex modeling details.
Jane: So, as we wrap up on this paper "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE", these estimators offer a concrete way to boost subgroup precision using internal trial data structures.
Lu: The core contribution is showing that these methods can improve efficiency in finite samples compared to conventional estimators, and the A-TMLE shows particular promise when baseline covariates are highly predictive of subgroup membership.
Meng: From my side, it means if you're running these analyses on a massive scale, you need to be careful about those model selection constraints they mentioned because deploying a model that only works under strict assumptions is risky.
Lalam: Lalam here. The paper concludes that for the LEADER trial, the A-TMLE provided consistent protective effect estimates for racial subgroups where the original trial wasn't powered to draw definitive conclusions, suggesting it has real utility in these specific scenarios.
Jane: So, to wrap up on this paper about "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE", the main point is that leveraging participants outside a subgroup within the same trial can significantly improve precision without needing external real-world data.
Tom: It's about making sure those subgroup-specific treatment effect estimates are as accurate as possible while respecting the structure of the randomized controlled trial data.
Lu: And we see that while there are efficiency gains, you have to contend with things like projection bias if the interaction is too complex and the way they select their working models for bias correction needs more flexibility.
Meng: The practical implication is that this isn't a plug-and-play solution; it requires careful handling of those model assumptions when you try to deploy it at scale, which is something I need to keep in mind.
Lalam: Lalam here. This work on "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE" shows how internal data sharing can help estimate subgroup effects, and future work will focus on making those working-model selection strategies more targeted for real-world use.
The paper's summary: Tom: So, we're looking at this paper called "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE." The main idea is that they want to fix a big problem where subgroup analyses in these trials are usually underpowered because there aren't enough patients in those specific groups.
Jane: That makes sense. So, the core strategy is using information from people already inside the trial, but who aren't in your specific subgroup of interest, to get better estimates for that subgroup, without needing any outside data from the real world.
Tom: Right. They propose two estimators called TMLE with pooled regression and Adaptive Targeted Maximum Likelihood Estimator or A-TMLE. These methods aim to improve the precision of those subgroup treatment effect estimates without introducing bias from external sources.
Jane: The paper shows that both these estimators can actually be more efficient than the standard ways you currently run subgroup analyses, especially when you have a limited number of participants in your specific group.
Tom: I saw that they found A-TMLE tends to offer bigger efficiency gains than TMLE with pooled regression when the baseline factors are strongly predictive of whether someone belongs to the subgroup. That makes sense because you're leveraging those known differences.
Jane: And they have a case study on a trial called LEADER, where for certain racial subgroups, A-TMLE estimates were remarkably stable across different time points in their follow-up period.
Tom: Stability is good for deployment because it means you're not getting wildly different results just because of minor fluctuations in how the data was collected over time. But they do point out some hurdles, though.
Jane: They flag that when the subgroup by treatment interaction is really complicated, there can be projection bias that might reduce those efficiency gains. Plus, they note that their current way of picking which working model to use for the bias correction part is a bit too restrictive.
Tom: That means if you try to deploy this at scale, you have to be careful with those modeling assumptions and the way you select your working models for that bias term. It's not just plug-and-play.
Jane: Exactly. The paper concludes that while this approach can boost precision using internal trial data, future work needs to focus on making those working model selection strategies much more targeted for actual use cases in the real world.
Tom: So, it’s showing a way to get better subgroup numbers using what’s already inside the trial structure, but you have to manage the modeling complexity carefully. This paper really gives us concrete estimators that might help regulators get more accurate subgroup effect estimates from these trials.
The paper's improvements: Tom: So, we’re talking about how these new estimators fix the efficiency issues they found in subgroup analysis using randomized controlled trials. We’re looking at how they suggest you can get better results without needing a massive sample size for every single subgroup you want to look at.
Jane: Right. The paper suggests that by borrowing information from people outside your specific group, the estimators get more precise estimates than standard methods, especially when data is limited.
Tom: They’re talking about two main things here: first, the TMLE with pooled regression and second, the Adaptive Targeted Maximum Likelihood Estimator or A-TMLE. These are these new ways to do the math on top of the original trial results.
Jane: The improvement is that these methods can get you a better estimate for a specific subgroup effect by using data from participants who weren't in that exact subgroup, all while staying within the confines of the original randomized controlled trial.
Tom: They found that A-TMLE actually beats TMLE with pooled regression when your baseline characteristics are really good at predicting whether someone belongs to the subgroup you’re interested in. That means it focuses its borrowing power where it actually makes sense.
Jane: And they show that this can lead to a reduction in event risk estimates, like we saw in that LEADER trial, specifically for certain racial subgroups over time. The numbers point toward a more consistent effect size estimate rather than just a noisy one.
Tom: It means for people who only listen to the show, it’s about getting more trustworthy numbers when the original study was too small to draw firm conclusions about that specific group.
Jane: And they did lay out the caveats clearly: you have to be careful because if the subgroup interaction is really complex, you might run into projection bias that cuts into those efficiency gains. Plus, they have to stick with a very specific way of choosing the working model for that bias term.
Tom: So, it’s not a magic fix; it’s a tool that works better under certain conditions and requires careful tuning of the model assumptions. It’s about improving precision, but you can't ignore how complex the underlying data structure is making things tricky.
Jane: Exactly. The paper isn't just giving you a new formula; it's giving you a roadmap for how to apply these statistical ideas in a way that respects the limitations of real trial data and makes those subgroup results more reliable for decision-making.
Conclusion: Tom: So, to wrap up this discussion on "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE," we’re seeing that these new methods offer a way to get more reliable subgroup numbers using only the data already collected during the trial itself.
Jane: That’s right. The big implication is that for regulators and researchers, this means they have a more precise tool to characterize effects in smaller patient populations where traditional power was lacking.
Tom: Exactly. We see it as a way to boost the precision of those subgroup-specific treatment effect estimates without needing to introduce external data sources like real-world records. It keeps everything consistent with the randomized controlled trial structure.
Jane: The authors show that when you use these estimators, you can get better efficiency than traditional methods, particularly in cases where the baseline patient factors are very strong predictors of being in a specific subgroup.
Tom: And they did point out that A-TMLE is actually more efficient than the other method we looked at when those baseline factors were highly predictive. That’s a key number for us as engineers trying to decide which model to use.
Jane: It changes things for someone listening because it suggests that even if you don't have enough people in a niche subgroup, you can still make a pretty strong, stable estimate of the treatment effect for that group using internal data.
Tom: Well said. But we do have those caveats about projection bias if the interaction is really messy and the way they select their working model for that correction isn't flexible enough. That’s where deployment gets tricky in my book.
Jane: It means while the math looks solid on paper, you can't just take it and run it everywhere without being careful about how complex those interactions are.
Tom: So, this paper is a strong piece for anyone working in causal inference trying to get subgroups out of RCT data, but you have to be ready to tune the model selection strategy carefully.
Jane: It’s a solid contribution because it gives us concrete estimators that can actually improve precision for groups where we usually run into statistical trouble.
Lu: If this A-TMLE approach works as suggested, it shifts the way we think about how much information we can extract from a trial without needing to collect more participants just for one niche group. It’s ambitious because it leverages existing data structures in a new way.
Meng: From an engineering standpoint, I still worry about the complexity of those working model selections they mentioned; if the setup requires too much manual tuning or restrictive assumptions, running this on a massive scale becomes a real headache for production.
Lalam: I’m pretty sure these results hold up because they are built on solid causal inference principles from the literature. The A-TMLE method is very sure about its ability to provide better estimates when the underlying data structure allows it to borrow strength effectively.
Tom: So, in summary, this paper "Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE" gives us two concrete methods, TMLE-PR and A-TMLE, that use internal trial data to get better subgroup estimates without needing external information.
Jane: It’s a helpful tool for anyone trying to characterize treatment effects in smaller patient groups where traditional analysis might be underpowered.
Tom: And while it has limitations regarding model complexity, the efficiency gains are real when you have predictive baseline covariates and you manage that modeling aspect correctly.
Jane: We need to keep an eye on how future work targets those working model selections so this method becomes a more straightforward tool for researchers.
More episodes
- Daily Summary for 2026-11-19
- 2310.07973-Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning
- 2408.09155-Learning Robust Treatment Rules for Censored Data
- Daily Summary for 2026-11-19
- 2603.23397-Kinetic Langevin Splitting Schemes for Constrained Sampling
- 2606.09906-An information-geometric framework for mapping maximum potential biodiversity
- 2603.27718-Induced replication and the assessment of models
- 1307.0898-Entropy of a Zipfian Distributed Lexicon
- 2406.19974-Generalizing self-normalized importance sampling with couplings
- 2412.10213-Collaborative Optimal Design of Controlled Experiments with Subject Covariates