A response-adaptive multi-arm design for continuous endpoints based on a weighted information measure

arXiv:2409.04970 · stat.ME, cs.IT, math.IT, stat.AP · Submitted 2024-09-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "A response-adaptive multi-arm design for continuous endpoints based on a weighted information measure".

Jane: Here's the gist of the paper. Multi-arm trials are gaining interest in practice given the statistical and logistical advantages they can offer The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target We establish conditions under which the procedure consistently selects the target arm and derive the corresponding limiting allocation ratios The practical implementation of the proposed criterion and its potential advantage over currently used alternatives are illustrated in the context of early Phase IIa proof-of-concept oncology trials.

Tom: Response-Adaptive Design Framework The authors present a response-adaptive design for continuous endpoints which explicitly allows to control the trade-off between the number of patients allocated to the "optimal" arm and the statistical power This balance is achieved through the calibration of a tuning parameter, and they explore robust procedures to select it.

Jane: The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target.

Tom: Information Gain Measure The authors propose to use the information gain (i.e.

Paper summary: Jane: So, wrapping up this discussion on "A response-adaptive multi-arm design for continuous endpoints based on a weighted information measure." The main idea here is moving away from fixed allocation ratios to a system that picks the next patient based on how much new information that arm provides about the true endpoint <ref:2409.04970#pg1>.

Tom: The authors establish conditions under which this procedure consistently selects the target arm, and they show how to choose a tuning parameter, kappa, robustly without knowing all the underlying parameters upfront <ref:2409.04970#pg1>. This whole framework is about balancing patient benefit against statistical power in continuous endpoint trials.

Lu: It implies that for continuous data where you want to explore multiple treatment arms, you can build a design that dynamically adapts its allocation based on observed responses, aiming for the most informative path forward <ref:2409.04970#pg3>.

Meng: From my side, it means we need to be very careful about how we set up these adaptive rules because the performance heavily depends on choosing that kappa correctly in practice <ref:2409.04970#pg1>. It's not just a theoretical model anymore; it’s a decision you have to make based on your risk tolerance and available data.

Lalam: The paper suggests exploring kappa values between zero point five and one point five, with the suggestion to use one for when there's expected variation in response among the different arms Supplementary Material. They are looking at joint optimization of kappa and some other parameter B for future work Supplementary Material.

Jane: So, it’s a framework that lets you control the trade-off between getting patients on the best arm and maintaining enough statistical power to make a real decision based on those continuous measurements <ref:2409.04970#pg1>. It’s about making smarter allocation decisions as you go.

Conclusion: Tom: So, we've seen how they build this whole response adaptive design for continuous endpoints based on that weighted information measure paper.

Jane: Right, so basically, they’re taking a trial where you don't know the exact outcome numbers yet and they are changing which arm gets the next patient based on how much new info that arm gives them about the true target.

Lu: The authors set up conditions where this switching process consistently picks the right arm and then they derive these allocation ratios, which tells us exactly how much of our total sample size should go to each treatment group.

Meng: What I'm looking at is how they choose that tuning parameter kappa, because that’s the knob you turn to control if the design explores a lot or it just sticks to what it knows.

Lalam: The paper suggests exploring kappa values between zero point five and one point five in practical settings, and they also mention using one when you expect different response variations between your arms.

Tom: It sounds like they are proposing a way to get a good balance between getting patients on the best arm and making sure we still have enough statistical power for a real conclusion.

Jane: Exactly. It’s about moving past fixed ratios, where you just stick to what you planned at the start, and letting the trial adapt to give you better results over time.

Lu: The methodology uses this information gain measure, which is like measuring how much more knowing that arm's result tells you about the true target mean compared to a simple average.

Meng: But what does that actually mean for us on the ground? It means if we're running a long trial, we can actually be smarter about where we put our patients next instead of just following a rigid plan.

Tom: It's interesting because they test this across five hundred different scenarios with known and unknown variances, and they found these response adaptive designs get good performance compared to other methods.

Jane: So, it seems like the authors are showing that this approach is solid for continuous data where you want to make smarter allocation choices as the trial goes on.

Lu: They also propose a way to test if one arm is significantly better than another by focusing only on the two best performers and checking if they are truly closer to the target mean.

Meng: The implication here is that for complex, continuous data settings, we might be able to design trials that actually work better in terms of patient benefit while keeping statistical power reasonable.

Tom: It's a shift from a static plan to a dynamic one that learns and adapts during the process.

Jane: And it opens up possibilities for how we run these types of experiments in clinical settings where you need to keep optimizing the trial as you go.

University of Cambridge

stat.ME, cs.IT, math.IT, stat.AP

Submitted: 2024-09-08

Updated: 2026-11-17

Comments: Accepted for publication in Journal of the Royal Statistical Society Series B: Statistical Methodology

DOI: 10.1093/jrsssb/qkag148

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Multi-arm trials are gaining interest in practice given the statistical and logistical advantages they can offer The proposed criterion is based on a context-dependent information measure which gives…

Key concepts

Response-Adaptive Design Framework
This is a design strategy for trials where decisions about which treatment arm to assign change based on the patient's response. It controls the trade-off between allocating more patients to the most promising arm and maintaining sufficient statistical power by calibrating a tuning parameter.
Information Gain Measure
This measure quantifies how much additional information is gained by estimating a specific treatment arm's mean response. It is calculated as the difference between Shannon differential entropy and its weighted version, giving more importance to arms near the desired clinical target.
Allocation Rule (WE)
The allocation rule assigns the current patient to the arm with the highest information gain. This creates a deterministic, response-adaptive scheme where patients are sequentially assigned based on which arm currently offers the most informative data about its true mean response.
Robust Parameter Selection ($\kappa$)
$\kappa$ is a crucial tuning parameter that balances exploration (trying different arms) and exploitation (focusing on the best known arm). The authors propose selecting a robust value for $\kappa$ by considering many plausible scenarios to ensure good performance regardless of unknown response distributions.

Terminology

Summary

Multi-arm trials are gaining interest in practice given the statistical and logistical advantages they can offer The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target We establish conditions under which the procedure consistently selects the target arm and derive the corresponding limiting allocation ratios The practical implementation of the proposed criterion and its potential advantage over currently used alternatives are illustrated in the context of early Phase IIa proof-of-concept oncology trials. p. 0

Where the paper says it

The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target. We establish conditions under which the procedure consistently selects the target arm and derive the corresponding limiting allocation ratios.

Response-Adaptive Design Framework

The authors present a response-adaptive design for continuous endpoints which explicitly allows to control the trade-off between the number of patients allocated to the optimal arm and the statistical power This balance is achieved through the calibration of a tuning parameter, and they explore robust procedures to select it. The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target. p. 0

Where the paper says it

Such a balance is achieved through the calibration of a tuning parameter, and we explore robust procedures to select it. The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target.

Information Gain Measure

The authors propose to use the information gain (i.e. the difference between the Shannon differential entropy and its weighted version) as a measure for the decision-making in a trial. p. 1

Where the paper says it

Drawing from the theory of weighted information measures [Kelbert and Mozgunov, 2015], we propose to use the information gain (i. the difference between the Shannon differential entropy and its weighted version) as a measure for the decision-making in a trial.

This quantity is defined as ∆(πnj) = h(πnj) − hϕγ(πnj), which represents the amount of additional information needed to estimate µj when the degree of interest in µj is not uniform across the whole parametric space. p. 2
Where the paper says it

Following Mozgunov and Jaki [2020], the information gain from considering the estimation problem in this sensitive area is given by ∆(πnj ) = h(πnj ) − h ϕγ (πnj ). (2) This quantity is the amount of additional information needed to estimate µj when the degree of interest in µj is not uniform across the whole parametric space (i.

Theorem 1 provides a closed-form expression for the information gain when a family of weight functions centred around the target γ is considered. p. 2
Where the paper says it

some values of µj are more desirable than others). Theorem 1 provides a closed-form expression for the information gain when a family of weight functions centred around the target γ is considered, and insights on its asymptotic behaviour.

Allocation Rule and Consistency

The allocation rule adopts the scheme where for the t-th patient, one assigns patient t to arm j such that j(t) = argmax j=1,..., K ∆njj. The WE(p,κ) allocation rule may be seen as a deterministic response-adaptive criterion where the next patient is almost surely allocated to the arm associated with the highest information gain. p. 4

Where the paper says it

In what follows, we refer to this class of designs based on the weighted-entropy measure as WE(p,κ). This allocation rule may be seen as a deterministic response-adaptive criterion, where the next patient is almost surely allocated to the arm associated with the highest information gain.

Theorem 2 establishes conditions under which each arm continues to be sampled indefinitely and the estimated ranking of arms (from closest to γ to furthest) converges to the true ranking. p. 4
Where the paper says it

3 provides a reliable selection of the best arm as the sample size increases. Theorem 2 establishes conditions under which each arm continues to be sampled indefinitely and the estimated ranking of arms (from closest to γ to furthest) converges to the true ranking.

Robust Parameter Selection

The choice of κ is crucial to control the trade-off between exploration and exploitation. The authors propose a general approach to select the robust optimal value of κ which does not require the prior knowledge on the mean responses and the standard deviations. p. 9

Where the paper says it

Apart from the vectors of the true means and true standard deviations, we suppose that all the other parameters are fixed in advance. Following Mozgunov and Jaki [2020], we discuss a general approach to select the robust optimal value of κ which does not require the prior knowledge on the mean responses and the standard deviations.

They suggest considering a large variety of plausible alternative scenarios, each one characterized by mean responses (and standard deviations if assumed unknown), and to select the value of κ associated with the average best performance over all the considered scenarios. p. 9
Where the paper says it

Following Mozgunov and Jaki [2020], we discuss a general approach to select the robust optimal value of κ which does not require the prior knowledge on the mean responses and the standard deviations. The idea is to consider a large variety of plausible alternative scenarios, each one characterized by mean responses (and standard deviations if assumed unknown), and to select the value of κ associated with the average best performance over all the considered scenarios.

Hypothesis Testing Procedure

The authors propose a testing procedure which only compares the two best performing arms, by also taking into account their correct identification. They define a test procedure which assesses whether µj∗ is sufficiently much closer to the target γ than µj∗∗. p. 7

Where the paper says it

,K, j̸=j ∗ |x¯Nj − γ|. Denoting µj ∗ and µj ∗∗ the true mean responses of arm j ∗ and arm j ∗∗, respectively, we propose a test procedure which assesses whether µj ∗ is sufficiently much closer to the target γ than µj ∗∗ .

Theorem 5 shows that, for κ > 0. p. 5
Where the paper says it

5 < κ < 1 (γ − µj ) 2 σ 1−p j 1+σ 2−p j −1 κ = 1 γ−µj σj −2 κ > 1 . Theorem 3 shows that, for κ > 0.

5 and under H1, the hypothesis testing procedure based on πbbj∗,bj∗∗ is consistent as the trial size increases. p. 8
Where the paper says it

Theorem 5 shows that, for κ > 0. 5 and under H1, the hypothesis testing procedure based on πbbj ∗,bj ∗∗ is consistent as the trial size increases.

Simulation Study Results

The simulation study investigates the performance of the proposed method over S = 500 random scenarios under various conditions for known and unknown variances. The results show that WE designs achieve good performance in terms of patient benefit and identification of the best arm than their competitors, with WE(1, 0. p. 0

Where the paper says it

The practical implementation of the proposed criterion and its potential advantage over currently used alternatives are illustrated in the context of early Phase IIa proof-of-concept oncology trials. Keywords information gain, best arm identification, type-I error rate control, phase II trial, patient benefit 1 Introduction Clinical trials are medical research studies involving patients with the aim of identifying the best treatment for future recommendation.

8) and WE(2, 1.1) achieving a power almost comparable to FR. p. 9
Where the paper says it

8 relaxes the requirement that the WE design must outperform the FR design in terms of power, a condition that can be overly stringent. Analogous practical compromises can be found in dose-response settings, where the dose achieving 80% of the maximum efficacy is often considered a reasonable target.

Conclusion and Recommendations

The class of WE(p,κ) designs offers a good balance between patient benefit and statistical power by making the correct decision in a remarkable proportion of replicated trials. They recommend exploring κ ∈ (0.5, 1.5] in practical applications. p. 0

Where the paper says it

We also introduce a simulation-based hypothesis testing procedure which focuses on selecting the target arm and discuss strategies to effectively control the type-I error rate. The practical implementation of the proposed criterion and its potential advantage over currently used alternatives are illustrated in the context of early Phase IIa proof-of-concept oncology trials.

The choice of p = 1 is suggested when heteroscedasticity among different arms is expected. p. 2
Where the paper says it

A specific value of this marker is being targeted, as lower values would indicate insufficient efficacy, while higher values could lead to severe adverse effects. The aims of the trial are (i) to find the most promising combination among the four, (ii) check whether the difference between the most promising and the second promising arm is “significant” and (iii) collect the most information about the most promising arms.

The authors adopt a hybrid approach which combines Bayesian and frequentist methodologies. p. 7
Where the paper says it

compute |µ (l) bj ∗ − γ| and |µ (l) bj ∗∗ − γ|; 3) approximate (9) with 1 L PL l=1 I |µ (l) bj ∗ − γ| < |µ (l) bj ∗∗ − γ| . 2 Frequentist properties of πbbj ∗,bj ∗∗ Although conditioning on the realized data is standard in Bayesian trials, it does not generally ensure control of the type-I error rate, so the frequentist properties of the resulting Bayesian test must be assessed separately [Spiegelhalter et al.

Supplementary Material

A Proofs section provides the proofs for the theorems, including A.1 Proof of Theorem 1, A. p. 6

Where the paper says it

5, 1) is sufficient to ensure that ρ˜j ∗ ≥ ρ˜j , ∀j ̸= j ∗ . The proof of Corollary 1 follows directly from substituting p = 1 in the asymptotic allocation ratio of Theorem 3, when κ ∈ (0.

2 Proof of Theorem 2, A. p. 6
Where the paper says it

5, 1) is sufficient to ensure that ρ˜j ∗ ≥ ρ˜j , ∀j ̸= j ∗ . The proof of Corollary 1 follows directly from substituting p = 1 in the asymptotic allocation ratio of Theorem 3, when κ ∈ (0.

3 Proof of Theorem 3, and A. p. 6
Where the paper says it

5, 1) is sufficient to ensure that ρ˜j ∗ ≥ ρ˜j , ∀j ̸= j ∗ . The proof of Corollary 1 follows directly from substituting p = 1 in the asymptotic allocation ratio of Theorem 3, when κ ∈ (0.

4 Proof of Theorem 4. p. 6
Where the paper says it

5, 1) is sufficient to ensure that ρ˜j ∗ ≥ ρ˜j , ∀j ̸= j ∗ . The proof of Corollary 1 follows directly from substituting p = 1 in the asymptotic allocation ratio of Theorem 3, when κ ∈ (0.

The Supplementary Material also includes details on multivariate generalization and the derivation of the information gain in the bivariate case. p. 6
Where the paper says it

However, this limiting share ρ˜j ∗ ∝ [(γ − µj ∗ ) 2 ] −1/(2κ−1) decreases as κ increases, reflecting a reduced exploitation of the best arm and a greater exploration of the other arms. 6 Multivariate generalization Consider now the case where q > 1 endpoints are measured.

The paper concludes with a discussion on future directions, suggesting joint optimization of κ and B. p. 7
Where the paper says it

This is consistent with the Bayesian principle that inference depends on the current study only through the data actually observed [Berry et al. Conditional on these data, the joint posterior of (µbj ∗ , µbj ∗∗ ) is bivariate normal with independent marginal components given by (1).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Context-dependent information measure for continuous endpoints allows allocation to be weighted by desirability near a clinical target gamma, controlling trade-offs between patient benefit and statistical power. This enables AI systems to prioritize learning about treatment arms whose mean responses are close to the desired clinical outcome.

  2. Allocation rule WE(p, kappa) can be implemented as a deterministic response-adaptive criterion where the next patient is assigned to arm j such that j(t) = argmax j=1,..., K ∆nj, ensuring sequential allocation maximizes contextual information gain.

  3. Robust selection of the tuning parameter kappa allows AI systems to optimize operating characteristics like patient benefit or statistical power across a wide set of plausible scenarios using an objective function approach. This enables the AI to select an allocation strategy that is nearly optimal regardless of the true underlying treatment parameters.

  4. Simulation-based hypothesis testing procedure provides a rigorous way for AI systems to assess whether the estimated best arm is sufficiently closer to the target than the second best, controlling type-I error rates through Monte Carlo procedures. This allows for more reliable decision-making under uncertainty compared to classical frequentist tests.

  5. Multivariate generalization of information gain handles multiple continuous endpoints simultaneously by using a vector of sample means and covariance matrices. This enables AI systems to make decisions in complex settings with multiple measurable outcomes, taking into account correlations between different endpoints.

Abstract

Multi-arm trials are gaining interest in practice given the statistical and logistical advantages they can offer. The standard approach uses a fixed allocation ratio, but there is a call for making it adaptive and skewing the allocation of patients towards better-performing arms. However, it is well-known that these approaches might suffer from lower statistical power. We present a response-adaptive design for continuous endpoints which explicitly allows to control the trade-off between the number of patients allocated to the "optimal" arm and the statistical power. Such a balance is achieved through the calibration of a tuning parameter, and we explore robust procedures to select it. The proposed criterion is based on a context-dependent information measure which gives greater weight to treatment arms with characteristics close to a pre-specified clinical target. We establish conditions under which the procedure consistently selects the target arm and derive the corresponding limiting allocation ratios. We also introduce a simulation-based hypothesis testing procedure which focuses on selecting the target arm and discuss strategies to effectively control the type-I error rate. The practical implementation of the proposed criterion and its potential advantage over currently used alternatives are illustrated in the context of early Phase IIa proof-of-concept oncology trials.

Sources

Related papers