A causal framework for evaluating the total effect of strategies aiming to expand screening and to improve outcomes
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.
Tom: We're looking at the paper: "A causal framework for evaluating the total effect of strategies aiming to expand screening and to improve outcomes".
Jane: Here's the gist of the paper. For many health conditions, there are highly efficacious treatment and prevention products.
Tom: Maximizing their impact requires strategies that improve the reach of health screening in order to establish who could benefit The paper proposes a novel framework using Two-Stage targeted minimum loss-based estimation (TMLE) to estimate the total effect of strategies aiming to expand screening and improve outcomes, addressing complex multilevel-mediation-missing data problems in cluster randomized trials.
Jane: The Causal Framework The authors introduce Counterfactual Strata Effects as causal estimands where the outcome is only relevant for a group whose membership is subject to missingness and/or impacted by the exposure.
Tom: This approach allows for the estimation of the total effect, which includes both direct and indirect effects, prior to investigating mediating pathways.
Title and authors: Tom: So we’re talking about this paper called "A causal framework for evaluating the total effect of strategies aiming to expand screening and to improve outcomes," by Nakato, Litunya, Beesiga, Kabami, Ayieko, Kamya, Chamie, and Balzer. It’s a big effort because it tries to figure out how much better things get when you focus on getting more people screened.
Jane: It sounds complicated because it deals with multilevel problems where the intervention happens at one level but the outcome is measured at another. The authors are trying to build a formal way to define this total effect in these complex settings.
Lu: What’s interesting is their use of Counterfactual Strata Effects, which allows them to estimate what the outcome would be for a specific group that's missing data or affected by the strategy, rather than just assuming everyone participates.
Meng: So instead of just measuring how many people got screened, they are estimating the total benefit when you change how screening is done across different communities.
Lalam: Exactly, and they use this framework to estimate the difference between what happens if you do a certain strategy versus another one aimed at expanding screening or improving outcomes.
The paper's summary: Tom: The paper says that for things like HIV prevention, where testing is the first step, health screening isn't always one hundred percent <ref:2506.06267#pg2>. This means we don't know the full focus population for a treatment effort.
Jane: They use this setup to define what they call Counterfactual Strata Effects as their causal estimands, which lets them look at the outcome for groups whose membership is subject to missing data or impacted by the strategy.
Lu: They set up a hierarchical, non-parametric structural equation model to define the causal relationship between variables like the intervention strategy and the outcome.
Meng: The key part is that they structure it so that you can estimate the total effect—that’s both direct and indirect effects—before you even get into figuring out how those effects work through other steps.
Lalam: They define this total effect by calculating the expected difference in uptake of pre-exposure prophylaxis among people with HIV risk if all clusters received multi-disease messaging versus HIV-focused messaging.
The paper's improvements: Tom: The authors point out a real problem with simpler methods, where single-stage approaches are biased because they condition on the mediator or something downstream of it.
Jane: So they show that this two-stage framework, using Targeted Minimum Loss-based Estimation, addresses that by fully stratifying on each cluster in Stage one to control for differential missingness without blocking the indirect effect.
Lu: And they mention that in Stage two you can adapt how you adjust the cluster level covariates based on the cluster size to maximize empirical efficiency, which could give a power boost of about two point eight percent to eight point three percent <ref:2506.06267#pg1>.
Meng: So if we were actually deploying this at scale, being able to choose between standard generalized linear models and these more complex adaptive approaches in Stage two based on the cluster size would be helpful for deployment.
Lalam: They also extended the framework to handle time-varying covariates by assuming that within values of baseline and post-baseline covariates, health status among those screened is representative among health status among those missed.
Conclusion: Tom: So, to wrap up, this paper on "A causal framework for evaluating the total effect of strategies aiming to expand screening and to improve outcomes" shows how to use two-stage Targeted Minimum Loss-based Estimation to estimate the total effect when dealing with those tricky multilevel missing data problems.
Jane: It gives us a solid tool for understanding strategies that aim to improve health by expanding screening, allowing us to see the full picture of direct and indirect effects without having to make overly strong assumptions about participation.
Lu: The way they handle the causal structure and the estimation in both stages is what makes this framework useful for complex cluster randomized trials where the intervention isn't uniform across all groups.
Meng: From an engineering standpoint, it’s good that they show how to maximize efficiency by adapting the adjustment strategy in Stage two based on cluster size, which helps us run these analyses faster on real data.
Lalam: This work gives us a concrete method for estimating the total effect of screening strategies while acknowledging the missingness inherent in real-world health systems.
Joy Z. Nakato, Janice Litunya, Brian Beesiga, Jane Kabami, James Ayieko, Moses R Kamya, Gabriel Chamie, Laura B Balzer
University of California, Berkeley Division of Biostatistics Department of HIV Infectious Diseases & Global Medicine Department of Medicine Makerere University
stat.ME
Submitted: 2025-06-06
Updated: 2026-06-17
Comments: 20 pages, 3 figures, accepted at "Statistics in Medicine"
Journal ref: Statistics in Medicine, 2026; 45:e70648
DOI: 10.1002/sim.70648
Code: https://github.com/jznakato/mult
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Importance score: 91/100
The gist: For many health conditions, there are highly efficacious treatment and prevention products.
Key concepts
- Counterfactual Strata Effects
- This concept allows researchers to study groups whose membership is affected by missing data or the intervention. It lets you estimate the total effect on this specific subgroup, even if not everyone in the cluster actually received the screening strategy.
- Two-Stage TMLE
- This is a two-step statistical method used to estimate causal effects in complex settings with missing data. Stage 1 estimates an expected outcome for each cluster, and Stage 2 uses this information to calculate the total effect with high precision.
- Mediation Analysis
- This involves understanding how an exposure (like a screening strategy) influences an outcome through an intermediate step (like health screening). The framework is designed to estimate these complex pathways, including both direct and indirect effects.
- Cluster Randomized Trials
- These are trials where entire groups or clusters receive the intervention. The framework specifically handles the challenges of analyzing data from these trials, which often involves missing information at the cluster level.
Terminology
Summary
For many health conditions, there are highly efficacious treatment and prevention products. p. 1 Abstract
For many health conditions, there are highly efficacious treatment and prevention products. Maximizing their impact requires strategies that improve the reach of health screening in order to establish
who could benefit. Maximizing their impact requires strategies that improve the reach of health screening in order to establish
who could benefit. For example, HIV prevention strategies aim to expand risk screening and to improve
uptake of pre-exposure prophylaxis (PrEP) among those experiencing risk.Where the paper says it
Where the paper says it
The Causal Framework
The authors introduce Counterfactual Strata Effects as causal estimands where the outcome is only relevant for a group whose membership is subject to missingness and/or impacted by the exposure. This approach allows for the estimation of the total effect, which includes both direct and indirect effects, prior to investigating mediating pathways. p. 2 These estimands correspond to hypothetical interventions to change
the distribution of the mediator, and many are reviewed in Rudolph et al. p. 2 Examples include
controlled direct effects, natural direct and indirect effects, stochastic direct and indirect effects, and separable
direct and indirect effects [12, 17–20]. These estimands correspond to hypothetical interventions to change
the distribution of the mediator, and many are reviewed in Rudolph et al. 1 Structural causal model
Causal relationships between these variables are specified through a hierarchical, non-parametric structural
equation model [22, 23]:
E
c “ fEc pUEc q
W “ fW pE
c
, UW q
A
c “ fAc pUAc q
Y 1
˚ “ fY 1˚ pE
c
,W, UY 1˚ q
∆ “ f∆pE
c
,W, Ac
, U∆q
Y 1 “ fY 1p∆,Y 1
˚
q “ ∆ ˆ Y 1
˚
Y 2 “ fY 2pE
c
,W, Ac
,Y 1, UY 2q
In this model, the values of the random variables on the left-hand side are determined by the structural
equations pfEc , fW , fAc , fY 1˚ , f∆, fY 1, fY 2q, which specify each variable’s “parents”, including unmeasured
factors pUEc , UW , UAc , UY 1˚ , U∆, UY 2q. Due to randomization, the unmeasured factors contributing to the
intervention assignment UAc are independent of the others.Where the paper says it
Where the paper says it
Where the paper says it
Data Generation and Variables
The data generating process involves a hierarchical causal model where individuals are nested in clusters, and health screening acts as a mediator on the causal pathway between the exposure (the strategy) and the outcome. Key variables include cluster-level baseline covariates, intervention assignment, health screening status, underlying HIV risk status, observed HIV risk status, and the outcome variable. p. 4 We now present our novel approach
for estimating total effects, while addressing the multilevel-mediation-missing data problem. Cluster- & individuallevel baseline
characteristics
Individual-level
prevention use
Individual-level
risk screening
Cluster-level
intervention
Unobserved
individual-level
HIV risk status
Observed
individual-level
HIV risk status
Figure 1: Simplified causal graph for the OPAL trial to increase HIV risk screening and PrEP use among
persons with HIV risk: individuals are nested within clusters; HIV risk screening mediates the intervention
effect, and HIV risk status is missing (unobserved) if an individual does not screen. To match the running example where PrEP initiation occurs at the health
clinic, we assume complete measurement of the outcome, but this can easily be relaxed. Altogether, we
denote the observed data for individual i in cluster j as Oij “ pEc
j
, Wij , Ac
j
, ∆ij , Y 1ij , Y 2ij q.Where the paper says it
Where the paper says it
Estimation Procedure
The estimation is conducted in two stages using Two-Stage TMLE to address the multilevel-mediation-missing data problem.
-
Stage 1 focuses on identifying and estimating the expected outcome among the underlying focus population for each cluster, defining the cluster-level endpoint Yc “ PpY2 “ 1q. p. 8
This involves estimating the denominator using G-computation formula under MAR assumptions. p. 9Where the paper says it
3 Two-Stage TMLE for Estimation and Inference Two-Stage TMLE is a general approach to identifying and estimating causal effects in a settings with missing data and dependent data [26, 27]. In the first stage, we stratify on each cluster and focus on controlling for differential missingness/censoring of individual-level data.
Where the paper says it
Throughout, the denominator defines the focus population and is subject to missingness: PpY 1 ˚ “ 1q. To identify the denominator from the observed data distribution, we need to consider the plausibility of various missing data assumptions.
-
Stage 2 focuses on evaluating the total effect Ψ˚ “ ErYc˚p1q ´ Yc˚p0qs with maximum precision by implementing a cluster-level TMLE. p. 9
Where the paper says it
In the following, we propose an implementation of Two-Stage TMLE to address unique challenges when evaluating the total effect of strategies to expand health screening and to improve health outcomes. As detailed below, fully stratifying on each cluster in the first stage simplifies the multilevel-mediation-missingness problem (Figure 1).
Simulation Results and Performance
Simulations demonstrated that Single-Stage approaches were meaningfully biased towards the null because they conditioned on a mediator or a descendant of the mediator. The Two-Stage framework, particularly when using TMLE in Stage 1 to estimate Yc, resulted in negligible bias and nominal confidence interval coverage across all sample sizes. p. 3 For identification and estimation of the corresponding statistical
estimand, we use Two-Stage targeted minimum loss-based estimation (TMLE), which separates control for
missing data from effect evaluation [26, 27]. Our simulation study evaluates the finite sample performance
of our approach and compares with common alternatives, including Single-Stage approaches. 4 Stage 1: Identifying and estimating in each cluster
In Stage 1, we consider each cluster in turn. Within each cluster, the cluster-level covariates and intervention
are constant. The term “cascade”
highlights the drop-offs occurring at each step of the process. Indeed, the coverage of health screening is
rarely 100%.Where the paper says it
Where the paper says it
Where the paper says it
Extensions and Future Work
The framework is extended to accommodate time-varying covariates and allow the exposure to impact the focus population. The authors plan to extend this work to observational studies where the group-level strategy is delivered in a non-randomized strategy. p. 1 This scenario creates a complex, multilevel-mediation-missing data problem for the
following reasons. First, the strategy is delivered at the cluster-level, while health screening and outcomes are at the individual-level. Balzer1
1Division of Biostatistics, University of California, Berkeley, California, USA
2Kenya Medical Research Institute (KEMRI), Kisumu, Kenya
3
Infectious Disease Research Collaboration (IDRC), Kampala, Uganda
4Division of HIV, Infectious Diseases & Global Medicine, University of California, San
Francisco, California, USA
5Department of Medicine, Makerere University, Kampala, Uganda
∗Corresponding author: jznakato@berkeley. edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. In
other words, our goal is to evaluate the total effect of the public health strategy among all persons in the
group of interest, including those not reached for health screening. We now propose developing, evaluating,
and applying Two-Stage TMLE for this effect.Where the paper says it
Where the paper says it
Where the paper says it
Acknowledgments
We gratefully acknowledge the participants and communities for taking part in the OPAL study. We also gratefully acknowledge the larger OPAL study team and the SEARCH collaboration. p. 3 In the following, we provide a novel framework to address the multilevel-mediation-missing data problem,
which commonly arises when public health strategies aim to improve both health screening and health
outcomes. First, we give an overview of OPAL, our motivating study, which was designed to improve PrEP
uptake among persons with HIV risk in Kenya and Uganda. A causal framework for evaluating the total effect of strategies
aiming to expand screening and to improve outcomes
Joy Z. Nakato∗1
, Janice Litunya2
, Brian Beesiga3
, Jane Kabami3
, James Ayieko2
, Moses R. , did not screen or screened but was not at risk). Finally, we define the outcome vector for cluster j as Y 2j “ pY 2ij : i “ 1, .Where the paper says it
Where the paper says it
Where the paper says it
Conflict of Interest and Funding
The authors declare no potential conflict of interests. This work was supported, in part, by the National Institutes of Health under awards: R01AA030464, K24AA031211, U01AI150510 and R01MH140685. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
References
[1] UNAIDS. The Urgency of Now: AIDS at a Crossroads — global AIDS report, 2024 <ref:2506.06267#pg2>
[2] UNAIDS. Creating hiv prevention cascades: Operational guidance on a tool for monitoring programmes, 2021 <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[3] UNAIDS. The HIV test-and-treat cascade – AIDS. p. 8 tion without enforcing 100% compliance on screening. Therefore, Ψ˚ can be classified as an intent-to-treat
parameter. edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
Where the paper says it
[4] UNAIDS. AIDS, crisis and the power to transform: UNAIDS global AIDS update, 2025 <ref:2506. p. 0 Kamya3,5, Gabriel Chamie4
, and Laura B. Balzer1
1Division of Biostatistics, University of California, Berkeley, California, USA
2Kenya Medical Research Institute (KEMRI), Kisumu, Kenya
3
Infectious Disease Research Collaboration (IDRC), Kampala, Uganda
4Division of HIV, Infectious Diseases & Global Medicine, University of California, San
Francisco, California, USA
5Department of Medicine, Makerere University, Kampala, Uganda
∗Corresponding author: jznakato@berkeley.Where the paper says it
[5] Maryam Shahmanesh, T. Nondumiso Mthiyane, Carina Herbsst, Melissa Neuman, Oluwafemi Adeagbo, Paul Mee, Natsayi Chimbindi, Theresa Smit, Nonhlanhla Okesola, Guy Harling, Nuala McGrath, Lorraine Sherr, Janet Seeley <ref:2506.06267#pg2>
[6] Elijah R. Kakande et al. A community-based dynamic choice model for HIV prevention improves PrEP and PEP coverage in rural Uganda and Kenya: a cluster randomized trial <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[7] Matthew D. Hickey et al. Community health worker–facilitated telehealth for moderate–severe hypertension care in Kenya and Uganda: A randomized controlled trial <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[8] Katrina F. Ortblad et al. Stand-alone model for delivery of oral HIV pre-exposure prophylaxis in Kenya: a single-arm, prospective pilot evaluation <ref:2506. p. 0 Balzer1
1Division of Biostatistics, University of California, Berkeley, California, USA
2Kenya Medical Research Institute (KEMRI), Kisumu, Kenya
3
Infectious Disease Research Collaboration (IDRC), Kampala, Uganda
4Division of HIV, Infectious Diseases & Global Medicine, University of California, San
Francisco, California, USA
5Department of Medicine, Makerere University, Kampala, Uganda
∗Corresponding author: jznakato@berkeley. edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation.Where the paper says it
[9] Jane Kabami et al. A Multi-Component Integrated HIV and Hypertension Care Model Improves Hypertension Screening and Control in Rural Uganda: A Cluster Randomized Trial <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[10] S. Greenland, J. Pearl, and J. M. Robins. Causal diagrams for epidemiologic research <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[11] Maya L Petersen et al. Estimation of direct causal effects <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[12] Vanessa Didelez et al. Direct and indirect effects of sequential treatments <ref:2506. p. 2 Several mediated effects may be of potential interest. Examples include
controlled direct effects, natural direct and indirect effects, stochastic direct and indirect effects, and separable
direct and indirect effects [12, 17–20].Where the paper says it
[13] David P. MacKinnon, Amanda J. Fairchild, and Matthew S. Fritz. Mediation Analysis <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[14] J. M. Robins and S. Greenland. Identifiability and exchangeability for direct and indirect effects <ref:2506. p. 2 Several mediated effects may be of potential interest. Examples include
controlled direct effects, natural direct and indirect effects, stochastic direct and indirect effects, and separable
direct and indirect effects [12, 17–20].Where the paper says it
[15] Judea Pearl. Direct and indirect effects <ref:2506. p. 2 Several mediated effects may be of potential interest. Examples include
controlled direct effects, natural direct and indirect effects, stochastic direct and indirect effects, and separable
direct and indirect effects [12, 17–20].Where the paper says it
[16] Peter C. Austin et al. Introduction to the Analysis of Survival Data in the Presence of Competing Risks <ref:2506. p. 2 1 Introduction
Health screening is the first step to initiating a new treatment or prevention product. For example, HIV
testing is a prerequisite for initiation of antiretroviral therapy (ART) for persons with HIV and for initiation
of pre-exposure prophylaxis (PrEP) for persons at risk of HIV acquisition.Where the paper says it
[17] Jessica G. Young et al. p. 8 However, our goal is to evaluate
the overall effectiveness of the health strategy among the entire focus population: persons with HIV risk. Counterfactual Strata Effects are also related but distinct from the framework proposed by Young and
colleagues for time-to-event outcomes with competing events [17, 20, 45, 46]. However, our goal is to evaluate
the overall effectiveness of the health strategy among the entire focus population: persons with HIV risk. Counterfactual Strata Effects are also related but distinct from the framework proposed by Young and
colleagues for time-to-event outcomes with competing events [17, 20, 45, 46].Where the paper says it
Where the paper says it
[18] Kara E. Rudolph et al. Robust and Flexible Estimation of Stochastic Mediation Effects: A Proposed Method and Example in a Randomized Trial Setting <ref:2506. p. 1 Simulations demonstrate the practical performance of our approach as well as the limitations
of existing approaches. Key Words: counterfactual strata effects; cluster randomized trials; group randomized trials;
measurement; mediation; missing data; screening; targeted minimum loss-based estimation
2Where the paper says it
[19] Tyler J. VanderWeele and Eric J. Tchetgen Tchetgen. Mediation analysis with time varying exposures and mediators <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[20] Mats J. Stensrud et al. A generalized theory of separable effects in competing event settings <ref:2506. p. 2 Health
screening can also be considered to be a competing event, because the outcome can only occur if the person
participates in screening [16, 17]. Several mediated effects may be of potential interest.Where the paper says it
[21] Kara E Rudolph et al. Causal Mediation Analysis With Observational Data: Considerations and Illustration Examining Mechanisms Linking Neighborhood Poverty to Adolescent Substance Use <ref:2506. p. 4 We now present our novel approach
for estimating total effects, while addressing the multilevel-mediation-missing data problem. Cluster- & individuallevel baseline
characteristics
Individual-level
prevention use
Individual-level
risk screening
Cluster-level
intervention
Unobserved
individual-level
HIV risk status
Observed
individual-level
HIV risk status
Figure 1: Simplified causal graph for the OPAL trial to increase HIV risk screening and PrEP use among
persons with HIV risk: individuals are nested within clusters; HIV risk screening mediates the intervention
effect, and HIV risk status is missing (unobserved) if an individual does not screen.Where the paper says it
[22] Judea Pearl. Causality <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[23] Laura B Balzer et al. A new approach to hierarchical data analysis: Targeted maximum likelihood estimation for the causal effect of a clusterlevel exposure <ref:2506. p. 3 We formally define the causal estimand with Counterfactual Strata
Effects, occurring when the outcome is only relevant for a group whose membership is subject to missingness
and/or impacted by the exposure [24–29]. For identification and estimation of the corresponding statistical
estimand, we use Two-Stage targeted minimum loss-based estimation (TMLE), which separates control for
missing data from effect evaluation [26, 27].Where the paper says it
[24] Laura Balzer et al. Evaluation of Progress Towards the UNAIDS 90-90-90 HIV Care Cascade: A Description of Statistical Methods Used in an Interim Analysis of the Intervention Communities in the SEARCH Study <ref:2506.06267#pg2>
[25] Laura B. Balzer et al. Far from MCAR: Obtaining population-level estimates of HIV viral suppression <ref:2506. p. 8 In
the running example, PrEP uptake is only relevant for persons who are at risk of HIV acquisition, and the
population with HIV risk is generally unknown. Counterfactual Strata Effects have previously been applied
to study intervention effects on population-level HIV viral suppression (the proportion of all persons with
HIV who are suppressing viral replication) and the incidence of latent tuberculosis infection (the proportion of persons at risk of tuberculosis who acquire it) [24–27, 39–42].Where the paper says it
[26] Laura B Balzer et al. Two-Stage TMLE to reduce bias and improve efficiency in cluster randomized trials <ref:2506. p. 9 the first stage to evaluate the total effect and focus on improving efficiency. In the following, we propose
an implementation of Two-Stage TMLE to address unique challenges when evaluating the total effect of
strategies to expand health screening and to improve health outcomes.Where the paper says it
[27] Joshua R Nugent et al. Blurring cluster randomized trials and observational studies: Two-Stage TMLE for subsampling, missingness, and few independent units <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[28] Shalika Gupta. Mechanism and Mediation: Counterfactual Strata Effects for Perinatal Epidemiology. p. 1 Simulations demonstrate the practical performance of our approach as well as the limitations
of existing approaches. Key Words: counterfactual strata effects; cluster randomized trials; group randomized trials;
measurement; mediation; missing data; screening; targeted minimum loss-based estimation
2 Kamya3,5, Gabriel Chamie4
, and Laura B. Balzer1
1Division of Biostatistics, University of California, Berkeley, California, USA
2Kenya Medical Research Institute (KEMRI), Kisumu, Kenya
3
Infectious Disease Research Collaboration (IDRC), Kampala, Uganda
4Division of HIV, Infectious Diseases & Global Medicine, University of California, San
Francisco, California, USA
5Department of Medicine, Makerere University, Kampala, Uganda
∗Corresponding author: jznakato@berkeley.Where the paper says it
Where the paper says it
[29] Maya Petersen. The Causal Roadmap in the Age of AI: From All Wheel Drive to Formula 1, April 2024 <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[30] Noah Kiwanuka et al. Population attributable fraction of incident HIV infections associated with alcohol consumption in fishing communities around Lake Victoria, Uganda <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[31] Marilyn N. Nyabuti et al. Characteristics of HIV seroconverters in the setting of universal test and treat: Results from the SEARCH trial in rural Uganda and Kenya <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[32] Mtumbi Goma et al. Predicting harmful alcohol use prevalence in Sub-Saharan Africa between 2015 and 2019: Evidence from population-based HIV impact assessment <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[33] Martin Mbonye et al. Alcohol consumption and high risk sexual behaviour among female sex workers in Uganda <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[34] Jennifer Velloza et al. HIV-risk behaviors and social support among men and women attending alcohol-serving venues in South Africa: Implications for HIV prevention <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[35] Demetria Cain et al. HIV risks associated with patronizing alcohol serving establishments in South African Townships, Cape Town <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[36] Janice Litunya et al. Impact of alcohol drinking venue characteristics on yield of hiv status-neutral testing in rural east africa, 2024 <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[37] Brian Beesiga et al. Higher levels of alcohol use associated with increased hiv risk among persons at alcohol-serving venues in the opal trial in kenya and uganda, 2025 <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[38] Alejandra Benitez et al. Defining and estimating effects in cluster randomized trials: A methods comparison <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[39] Maya Petersen et al. Association of Implementation of a Universal Testing and Treatment Intervention With HIV Diagnosis, Receipt of Antiretroviral Therapy, and Viral Suppression in East Africa <ref:2506. p. 2 1 Introduction
Health screening is the first step to initiating a new treatment or prevention product. For example, HIV
testing is a prerequisite for initiation of antiretroviral therapy (ART) for persons with HIV and for initiation
of pre-exposure prophylaxis (PrEP) for persons at risk of HIV acquisition. 1 Introduction
Health screening is the first step to initiating a new treatment or prevention product. For example, HIV
testing is a prerequisite for initiation of antiretroviral therapy (ART) for persons with HIV and for initiation
of pre-exposure prophylaxis (PrEP) for persons at risk of HIV acquisition.Where the paper says it
Where the paper says it
[40] Diane V. Havlir et al. HIV Testing and Treatment with the Use of a Community Health Approach in Rural Africa <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[41] Rachel Abbott et al. Incident Tuberculosis Infection Is Associated With Alcohol Use in Adults in Rural Uganda <ref:2506. p. 3 In sub-Saharan Africa, attending alcoholserving venues as a patron or an employee has been associated with increased HIV risk [33–37]. OPAL (Outreach and Prevention at ALcohol venues) is an ongoing cluster randomized trial (CRT)
whose primary objective is to evaluate the effectiveness of community-based, recruitment strategies on PrEP
uptake among adults at alcohol-serving venues in rural Kenya and Uganda (NCT05862857).Where the paper says it
[42] Carina Marquez et al. Association of Implementation of a Universal Testing and Treatment Intervention With HIV Diagnosis, Receipt of Antiretroviral Therapy, and Viral Suppression in East Africa <ref:2506.06267#pg2>
[43] Leonardo Grilli and Fabrizia Mealli. Nonparametric Bounds on the Causal Effect of University Studies on Job Opportunities Using Principal Stratification <ref:2506. p. 6 Finally, the only effect of screening (∆q on
the outcome (Y 2) is through establishing that the individual is in the focus population (Y 1 “ 1). Using
the causal model, we can generate counterfactuals and formally define the causal estimand of interest, as
described next.Where the paper says it
[44] Constantine E. Frangakis and Donald B. Rubin. Principal Stratification in Causal Inference <ref:2506. p. 8 We now propose developing, evaluating,
and applying Two-Stage TMLE for this effect. 3 Two-Stage TMLE for Estimation and Inference
Two-Stage TMLE is a general approach to identifying and estimating causal effects in a settings with missing
data and dependent data [26, 27].Where the paper says it
[45] Torben Martinussen and Mats Julius Stensrud. Estimation of separable direct and indirect effects in continuous time <ref:2506. p. 2 Several mediated effects may be of potential interest. Examples include
controlled direct effects, natural direct and indirect effects, stochastic direct and indirect effects, and separable
direct and indirect effects [12, 17–20].Where the paper says it
[46] Matias Janvin et al. Causal inference with recurrent and competing events <ref:2506. p. 8 However, our goal is to evaluate
the overall effectiveness of the health strategy among the entire focus population: persons with HIV risk. Counterfactual Strata Effects are also related but distinct from the framework proposed by Young and
colleagues for time-to-event outcomes with competing events [17, 20, 45, 46].Where the paper says it
[47] DONALD B. RUBIN. Inference and missing data <ref:2506. p. 8 We now propose developing, evaluating,
and applying Two-Stage TMLE for this effect. 3 Two-Stage TMLE for Estimation and Inference
Two-Stage TMLE is a general approach to identifying and estimating causal effects in a settings with missing
data and dependent data [26, 27].Where the paper says it
[48] James Robins. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect <ref:2506. p. 8 We now propose developing, evaluating,
and applying Two-Stage TMLE for this effect. 3 Two-Stage TMLE for Estimation and Inference
Two-Stage TMLE is a general approach to identifying and estimating causal effects in a settings with missing
data and dependent data [26, 27].Where the paper says it
[49] Mark J. Van Der Laan and Sherri Rose. Targeted Learning: Causal Inference for Observational and Experimental Data <ref:2506. p. 8 We now propose developing, evaluating,
and applying Two-Stage TMLE for this effect. 3 Two-Stage TMLE for Estimation and Inference
Two-Stage TMLE is a general approach to identifying and estimating causal effects in a settings with missing
data and dependent data [26, 27].Where the paper says it
[50] Mark J. van der Laan, Eric C. Polley, and Alan E. Hubbard. Super learner <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[51] Mitchell H. Gail et al. Design considerations for studies of intervention effects on recurrence <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[52] Anastasios A. Tsiatis et al. Covariate adjustment for twosample treatment comparisons in randomized clinical trials: A principled yet flexible approach <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[53] R. A. Fisher. Statistical Methods for Research Workers <ref:2506. p. 4 New methods are needed
to evaluate overall effectiveness, while accounting for the multilevel-mediation-missing data problem. 3 Methods
We first introduce the notation that will be used to formalize the data generating process, specify the causal
estimand, and derive the statistical estimand.Where the paper says it
[54] K. L. Moore and M. J van der Laan. Covariate adjustment in randomized trials with binary outcomes: Targeted maximum likelihood estimation <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[55] Michael Rosenblum and Mark J van der Laan. Simple, Efficient Estimators of Treatment Effects in Randomized Trials Using Generalized Linear Models to Leverage Baseline Variables <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[56] Laura B Balzer et al. Adaptive Pre-specification in Randomized Trials With and Without Pair-Matching <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[57] Laura B Balzer et al. Adaptive selection of the optimal strategy to improve precision and power in randomized trials <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[58] Laura B Balzer et al. Machine learning to optimize precision in the analysis of randomized trials: A journey in pre-specified, yet data-adaptive learning <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[59] Daniel B. Rubin and Mark J. van der Laan. Empirical efficiency maximization: improved locally efficient covariate adjustment in randomized experiments and survival analysis <ref:2506.06267#pg2>
[60] Ori M. Stitelman and Mark J. van der Laan. Collaborative targeted maximum likelihood for time to event data <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[61] A. W. Van Der Vaart. Asymptotic Statistics <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506. edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
Where the paper says it
[62] Richard J Hayes and Lawrence H Moulton. Cluster Randomised Trials <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[63] Mallorie H. Fiero et al. Statistical analysis and handling of missing data in cluster randomized trials: a systematic review <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[64] Mireille E Schnitzer et al. EFFECT OF BREASTFEEDING ON GASTROINTESTINAL INFECTION IN INFANTS: A TARGETED MAXIMUM LIKELIHOOD APPROACH FOR CLUSTERED LONGITUDINAL DATA <ref:2506. p. 3 For identification and estimation of the corresponding statistical
estimand, we use Two-Stage targeted minimum loss-based estimation (TMLE), which separates control for
missing data from effect evaluation [26, 27]. Our simulation study evaluates the finite sample performance
of our approach and compares with common alternatives, including Single-Stage approaches.Where the paper says it
[65] KUNG-YEE LIANG and SCOTT L. ZEGER. Longitudinal data analysis using generalized linear models <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[66] Fan Li et al. Model-Robust Standardization in Cluster-Randomized Trials <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[67] Joshua Schwab et al. ltmle: Longitudinal Targeted Maximum Likelihood Estimation, April 2023 <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[68] Joy Z Nakato and Laura B. Balzer. Adaptive pooling to minimize bias and maximize power in clusterrandomized trials, 2025 <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
[69] Aad W. van der Vaart. Asymptotic Statistics <ref:2506.06267#pg2>
[70] Anastasios A. Tsiatis. Semiparametric Theory and Missing Data <ref:2506. p. 0 edu
Abbreviations: ART, Antiretroviral Therapy; CRTs, Cluster Randomized Trials; PrEP, Pre-Exposure Prophylaxis;
TMLE, Targeted Minimum Loss-based Estimation. 1
arXiv:2506.Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Counterfactual Strata Effects allow for causal estimands where
the outcome is only relevant for a group whose membership is subject to missingness and/or impacted by the exposure of interest,
enabling evaluation of total effects without enforcing 100% compliance on screening. -
Two-Stage TMLE can be implemented to address unique challenges when evaluating the total effect of strategies to expand health screening and improve health outcomes, as it
fully stratif[ies] on each cluster in Stage 1
to control for differential missingness without blocking the indirect effect. -
The system can estimate the total effect of a strategy by calculating
the expected difference in the counterfactual uptake of PrEP (among persons with HIV risk) if all clusters received multi-disease messaging versus HIV-focused messaging.
-
Time-varying covariates can be incorporated into Stage 1 estimation by assuming that
within values of baseline and post-baseline covariates, health status among those screened is representative among health status among those missed: Y 1˚ K ∆ W, L.
-
The improved AI system can perform adaptive selection of the adjustment approach in Stage 2 to
maximize empirical efficiency,
choosing between working generalized linear models or more complex adaptive approaches based on cluster size.
Abstract
For many health conditions, there are highly efficacious treatment and prevention products. Maximizing their impact requires strategies that improve the reach of health screening in order to establish who could benefit. For example, HIV prevention strategies aim to expand risk screening and to improve uptake of pre-exposure prophylaxis (PrEP) among those experiencing risk. Often, these strategies induce changes at the group-level (e.g., health clinics or communities) and are evaluated through cluster randomized trials. This scenario creates a complex, multilevel-mediation-missing data problem for the following reasons. First, the strategy is delivered at the cluster-level, while health screening and outcomes are at the individual-level. Second, the strategy improves health outcomes directly and indirectly through improved health screening. Third, everyone has an underlying status, which is only observed among those screened. To formally define the total effect in such settings, we use Counterfactual Strata Effects: causal estimands where the outcome is only relevant for a group whose membership is subject to missingness and/ or impacted by the exposure of interest. To identify and estimate the corresponding statistical estimand, we propose a novel extension of Two-Stage targeted minimum loss-based estimation (TMLE). Simulations demonstrate the practical performance of our approach as well as the limitations of existing approaches.
Sources
Related papers
- Learning Robust Treatment Rules for Censored Data
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Spectral divide-and-conquer MCMC for long stationary time series
- Improving the Efficiency of Subgroup Analysis in Randomized Controlled Trials with TMLE
- Personalized Federated Vector Autoregression with Personalization Diversity
- Doubly Robust and Efficient Calibration of Prediction Sets for Right-Censored Time-to-Event Outcomes