Sensitivity Analysis for Trials with Irregular and Informative Assessment Times: Marginal Mean Models and Sensitivity Parameter Selection

In randomized trials, outcome assessment times often vary across participants and may depend on the underlying outcome process, so outcomes observed near a target time need not represent the target population. Whether assessment is informative cannot be determined from the observed data alone. The paper extends the sensitivity analysis framework of Smith et al. (2024), which is anchored in an explainable assessment assumption and was restricted to continuous outcomes through an identity-link marginal mean model, to a broader class of link functions that accommodates binary and count outcomes. Within this framework, it develops an influence function-based augmented inverse intensity-weighted estimator that is root-n consistent under standard regularity conditions. It also proposes a data-driven procedure for selecting the range of the sensitivity parameter from plausible bounds on the target mean outcome and for constructing a confidence set for this range, which accounts for uncertainty in sensitivity parameter selection. The method is illustrated with data from the ARC study and evaluated in simulation studies.

The framework involves:

Assumption 1 (Tilting assumption): For each time t such that λt O(t) > 0, dF y(t) ∆N(t) = 0, O(t) = dF y(t) ∆N(t) = 1, O(t) exp[αy(t)]/c n O(t); αo 0 such that, for all t in [t1, t2], ρt O(t), Y (t) > c for all values of Y (t) and O(t).

Assumption 3 (Generalized marginal mean assumption): For all t in [t1, t2], µ(t) = s s B(t)'β, where B(t) = (B1(t), · · ·, Bp(t))' is a specified vector of basis functions such that R t2 t=t1 B(t)B(t)'dt is positive definite, s ∈ C2 is a known inverse link function with strictly positive derivative ∂s(z)/∂z > 0, and β ∈ R p is a parameter vector.

Assumption 4 (Non-future dependence assumption): Let L = Y (t), X(t): 0 ≤ t ≤ τ. Then limϵ→0+ h P N(t +ϵ) − N(t−) = 1 O(t), L /ϵi = limϵ→0+ h P N(t +ϵ) − N(t−) = 1 O(t), Y (t)/ϵi.

The estimation procedure involves:

Estimation of the Mean Outcome:

The paper models the intensity function λt O(t) using an Andersen-Gill model stratified by assessment number k, and models dF Y (t) = y(t) ∆N(t) = 1, O(t) using a single-index model. Estimators βb and µb(t) are constructed through steps involving estimating the conditional mean outcome Eb Y (Tik) Oi(Tik), computing the estimated inverse intensity weight ρb Tik Oi(Tik), and solving a system of equations for βb j.

Influence Functions:

The observed-data influence function for β is given by:

φj (O) = Z t2 t=t1 Wj (t; β) h Y (t) − E n Y (t) O(t) o ρn t O(t), Y (t) o dN(t)+Z t2 t=t1 Wj (t; β) h E n Y (t) O(t) o − s B(t)'β i dt where Wj (t; β) has the form:

W1(t; β) = V−1 1 B(t) ∂g(µ)/∂µ, V1 = Z t2 u=t1 B(u)B(u)' du; W2(t; β) = V2 (β−1 B), V2(β) = Z t2 u=t1 B(u)B(u)' ∂s(z)/∂z z=B'(u)'β du.

Sensitivity Parameter Selection:

The true range of the sensitivity parameter αa is defined by the interval (αl a, αu a): Wa = α ∈ [α1 a, α2 a]: min t∈[t1,t2] E Y(t; α) > µla, max t∈[t1,t2] E Y(t; α) < µua. The bounds are estimated using optimization problems and bias-corrected via multiplier bootstrap procedures (Algorithm 2).

Confidence Set Construction:

A 95% confidence set for the range of the sensitivity parameter is constructed using either a Basic Bootstrap CI or a Wald CI based on the bias-corrected point estimates and variance estimators.

The paper applies this method to analyze data from the ARC study, examining both continuous (ACQ score) and binary (AC indicator) outcomes. Simulation studies demonstrate that while finite-sample performance can be challenging, the proposed bias-corrected estimators consistently exhibit smaller bias than original estimators as sample size increases, and Wald confidence intervals often achieve better empirical coverage than basic bootstrap confidence intervals. The analysis suggests that statistically significant treatment effects emerge when informative assessment mechanisms differ between treatment arms.

The key findings regarding the ARC study are:

For the ACQ score:

When assuming α0 = α1, there is insufficient evidence to conclude a treatment effect at either 6 or 12 months for either outcome. However, if α0 = 0 and α1 = −0.8 (implying unobserved ACQ scores tend to be lower than observed scores in the intervention arm), there is evidence that the home visits intervention improves the population mean ACQ score at both 6 and 12 months compared to portal training alone.

For the AC indicator:

If α0 = 0 and α1 = 0.8 (implying unobserved AC indicators tend to be higher than observed values in the intervention arm), there is evidence that portal training alone results in better asthma control than the home visits intervention at both 6 and 12 months.

Overall, when α0 and α1 are similar with a difference no more than 0.8, there is insufficient evidence to conclude a treatment effect at either follow-up time. When α0 > α1 + 0.8, the analyses frequently provide evidence that the home visits intervention improves the population mean asthma control outcome relative to portal training alone at both 6 and 12 months.

The paper concludes by demonstrating that when informative assessment mechanisms are allowed to differ between treatment arms, statistically significant treatment effects emerge under some plausible sensitivity parameter values. It also details a data-driven procedure for identifying plausible ranges of the sensitivity parameters and constructing confidence sets based on the plausible bounds for the target mean outcome. The methodology is extended to accommodate binary and count outcomes through appropriate link functions. The authors note limitations regarding non-future dependence assumptions and informative censoring, suggesting future research directions.

Relevant quotes include:

"We extend the sensitivity analysis framework of Smith et al. (2024), which is anchored in an explainable assessment assumption and was restricted to continuous outcomes through an identity-link marginal mean model, to a broader class of link functions that accommodates binary and count outcomes."

Within this generalized framework, we develop an influence function-based augmented inverse intensity-weighted estimator that is root-n consistent under standard regularity conditions.

"We also propose a data-driven procedure for selecting the sensitivity parameter from plausible bounds on the target mean outcome and for constructing a confidence set for this range, which accounts for uncertainty in sensitivity parameter selection."

"The proposed methodology substantially broadens the scope of the sensitivity analysis framework. Our approach accommodates a broad class of outcome types, including continuous, binary, and count outcomes, through appropriate link functions."

Compared with the previous sensitivity analysis framework of Smith et al. (2024), the proposed methodology substantially broadens the scope of the sensitivity analysis framework.

Our approach accommodates a broad class of outcome types, including continuous, binary, and count outcomes, through appropriate link functions.

We develop a data-driven procedure for identifying plausible ranges of the sensitivity parameters and constructing confidence sets based on the plausible bounds for the target mean outcome.

"The results for both outcomes indicate that when α0 and α1 are similar, with the difference in values no more than 0.8, there is insufficient evidence to conclude a treatment effect at either follow-up time. In contrast, when α0 > α1 + 0.8, the analyses frequently provide evidence that the home visits intervention improves the population mean asthma control outcome relative to portal training alone at both 6 and 12 months."

"Overall, the results for both outcomes indicate that when α0 and α1 are similar, with the difference in values no more than 0.8, there is insufficient evidence to conclude a treatment effect at either follow-up time. In contrast, when α0 > α1 + 0.8, the analyses frequently provide evidence that the home visits intervention improves the population mean asthma control outcome relative to portal training alone at both 6 and 12 months."

"In contrast, for the AC indicator, if we assume α0 = 0 and α1 = 0.

1. Sensitivity analysis for treatment effect estimation can be generalized to binary and count outcomes by extending frameworks from Smith et al. (2024) beyond continuous outcomes through a broader class of link functions. This allows AI systems to perform more accurate inference when dealing with non-normal data types, such as the ACQ score or asthma control indicator mentioned in the ARC study.

2. Development of an influence function based augmented inverse intensity-weighted estimator provides a flexible semiparametric modeling approach that achieves root-n consistency under standard regularity conditions. This enables AI systems to estimate treatment effects at clinically meaningful target times even when assessment patterns are irregular and outcomes are informative, addressing limitations in existing methods like inverse intensity weighting.

3. A data driven procedure for selecting the range of the sensitivity parameter from plausible bounds on the target mean outcome allows AI systems to construct confidence sets for this range, accounting for uncertainty in parameter selection. This provides a principled way to quantify uncertainty when clinical experts provide prior ranges for outcome means, leading to more interpretable sensitivity analysis results.

4. The proposed estimation procedure for the treatment effect at time t involves estimating intensity functions using an Andersen-Gill model stratified by assessment number and conditional distributions using a single-index model. This allows AI systems to account for informative assessment times by modeling both the assessment process and outcome distribution separately, which is crucial when outcomes are not observed at target times.

5. The framework enables inference for treatment effects under a range of plausible assumptions, providing a more comprehensive representation of the overall uncertainty in study conclusions compared to methods that rely on single assumptions. This multi-assumption approach allows AI systems to quantify how different informativeness mechanisms affect treatment effect estimates across various outcome types and time points.

6. The simulation studies demonstrate that the proposed augmented inverse intensity-weighted estimator achieves satisfactory coverage performance, even when the explainable assessment assumption is violated. This suggests that AI models can be trained and validated using this method to ensure their reported confidence intervals accurately reflect true coverage rates under realistic, informative assessment scenarios.