Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators

arXiv:2511.19905 · math.ST, stat.ME, stat.TH · Submitted 2025-11-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Mathematics. The day in mathematics: new theorems, new proofs, and what they open.

Ada: We're looking at the paper: "Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators".

Emmy: Here's the gist of the paper. Detailed Research Summary: Sigmoid-FTRL for Adaptive Neyman Allocation in AIPW Estimation This research paper presents Sigmoid-FTRL, a novel adaptive experimental design tailored for Adaptive Neyman Allocation within the framework of AIPW (Augmented Inverse Probability Weighting) estimators.

Ada: The primary objective is to construct an adaptive procedure that minimizes the Neyman Regret, defined as the difference between the variance of the adaptive procedure and that of an oracle estimator utilizing optimal nonadaptive assignment probabilities and linear predictors.

Emmy: Core Methodology: Sigmoid-FTRL The design, Sigmoid-FTRL, addresses inherent non-convexity in adaptive allocation problems by employing a simultaneous minimization strategy targeting two distinct convex regrets: probability regret and prediction regret.

Ada: The algorithm operates by applying the Follow-the-Regularized Leader (FTRL) principle separately to these two objectives.

Paper summary: Ada: So we're looking at a paper called "Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators," and the main idea is building an adaptive experimental design for when you have those AIPW estimators.

Emmy: The core thesis of this paper is constructing an adaptive procedure that minimizes the Neyman Regret, which they define as the difference between how much variance their adaptive method has compared to what you'd get from a perfect nonadaptive design using optimal assignment probabilities and linear predictors.

Kolya: What I find interesting is that previous work on this problem showed a connection between Neyman Regret and online convex optimization for the Horvitz–Thompson estimator, but the AIPW estimator presents a non-convex optimization challenge that this paper aims to tackle directly.

Ada: Exactly. The paper proposes something called Sigmoid-FTRL, which is an adaptive experimental design that specifically addresses that non-convexity in the setup.

Emmy: They do this by using a simultaneous minimization strategy targeting two different types of regrets: probability regret and prediction regret, applying the Follow-the-Regularized Leader principle separately to each one.

Kolya: So, they are splitting the optimization into two parts: one for choosing how likely you assign a treatment, and another for picking the best linear predictors to use.

Ada: Right. Specifically, they choose linear predictors to minimize estimated squared prediction errors and select the treatment probability to minimize a weighted sum of online residuals with a sigmoid regularizer added in.

Emmy: That sigmoid regularizer is what helps them navigate that non-convex landscape effectively when they are choosing the treatment probability.

Kolya: And they state that this design has some computational tractability, with complexity scaling as O of d cubed q per iteration and O of d squared q total storage.

Ada: The key theoretical result they establish is the convergence rate for the Neyman Regret, which they prove converges at a rate of T to the power of one over two R, where T is the number of subjects in the experiment and R is the maximum norm of those covariate vectors.

Emmy: That specific exponent, one over two R, is significant because it establishes that this rate is actually the minimax rate for Neyman regret under their stated regularity conditions.

Kolya: It means they're showing that no adaptive design can do better than a T to the power of one over two R worst-case performance when those assumptions hold true.

Ada: And they decompose the total Neyman Regret into two parts, expected probability regret and expected prediction regret, both of which are bounded by O of T to the power of one over two R.

Emmy: That decomposition is neat because it shows that both components contribute equally to the overall convergence rate, even though they represent different types of errors.

Kolya: Beyond just the convergence rate, they establish that under those conditions, the asymptotic variance of their adaptive AIPW estimator converges exactly to its oracle variance.

Ada: That means when you run the experiment for a large number of subjects, the variance estimate from their method settles down right onto what you'd expect from a perfect nonadaptive design.

Emmy: Plus, they prove the central limit theorem holds for this AIPW estimator using the martingale central limit theorem and controlling conditional variance convergence.

Kolya: So what that means for someone just listening to the show is that if you take your adaptive estimate and standardize it, it behaves like a standard normal distribution asymptotically.

Ada: And they provide tools for practical inference by showing that the estimated variance bound is consistent, with its asymptotic coverage bounded by O of T to the power of five over twelve R to the power of five over six.

Emmy: This consistency allows them to build Wald-type confidence intervals that are guaranteed to cover the true parameter at least at their nominal level.

Kolya: So, in simple terms, they've got a design that converges fast enough, and it gives you reliable ways to make inferences about your treatment effect using those AIPW estimators.

Ada: To wrap up the paper "Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators," the authors show that their Sigmoid-FTRL design achieves a Neyman Regret convergence rate of T to the power of one over two R, which is minimax optimal under those conditions.

Emmy: The implication is that this design gives us a solid framework for adaptive experimental designs when we are dealing with AIPW estimators where the underlying optimization problem is non-convex.

Kolya: It shows that even though things are non-convex, we can still get good performance bounds and reliable statistical inference using this specific approach.

Ada: So, the paper tackles a hard problem in adaptive experimental design by providing a concrete algorithm and proving its theoretical guarantees on convergence and asymptotic normality of the resulting estimator.

Conclusion: Ada: So we're wrapping up on Sigmoid-FTRL: Design-Based Adaptive Neyman Allocation for AIPW Estimators, and what this whole thing really means for how we do those AIPW estimators in the real world.

Emmy: It sounds like the authors are showing that even when the optimization problem gets tricky and non-convex, there's a way to handle it adaptively using this Sigmoid Follow-the-Regularized Leader method.

Kolya: They’ve done some heavy lifting by proving that their design achieves a Neyman Regret convergence rate of T to the power of one over two R, which they state is the minimax rate under their specific assumptions.

Ada: That’s the big number here, Kolya. It means that for this type of problem, you can't really hope to get better worst-case performance than what they've found with this method.

Emmy: And it’s not just about the theoretical rate; they also confirm that their estimator actually converges to the correct variance when you run a lot of subjects.

Kolya: That’s important for practical use because it means if you are trying to build confidence intervals around your AIPW estimate, those intervals will behave correctly as you get more data.

Ada: Exactly. So, the main thing is that they’ve built a complete framework: a design that works well under tough conditions and gives us reliable statistical inference tools like Wald-type confidence intervals.

Emmy: The authors are essentially taking a complex, non-convex problem and giving us an adaptive algorithm that solves both the probability regret and the prediction regret simultaneously.

Kolya: It’s a lot of structure there, Emmy. But they did point out their limitations too, so we know where this specific design might not apply perfectly.

Ada: Right. So while they proved it works under certain regularity conditions, we still have to keep those assumptions in mind when deciding if Sigmoid-FTRL is the right tool for our own experiments.

Emmy: If you want to understand how this relates to other experimental designs, or maybe look at what they plan next with this approach, we’ve got some more stuff coming up.

Columbia University · Massachusetts Institute of Technology · New York University

math.ST, stat.ME, stat.TH

Submitted: 2025-11-25

Updated: 2026-11-04

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 100/100

The gist: Sigmoid-FTRL for Adaptive Neyman Allocation in AIPW Estimation This research paper presents Sigmoid-FTRL…

Key concepts

Sigmoid-FTRL
A new algorithm designed for adaptive experimental design. It uses a 'Follow-the-Regularized Leader' approach to simultaneously minimize probability regret and prediction regret, using a sigmoid regularizer to handle complex, non-convex allocation problems effectively.
Adaptive Neyman Allocation
A method for assigning subjects to treatment groups based on their characteristics during an experiment. The goal is to select the best assignment probabilities dynamically as more data becomes available, aiming to minimize the difference between the actual variance and the best possible theoretical variance (Neyman Regret).
AIPW Estimators
Augmented Inverse Probability Weighting estimators are statistical methods used to estimate treatment effects. They incorporate weights derived from both probability of treatment assignment and outcome information, providing more accurate estimates than simpler weighting schemes.
Neyman Regret Convergence Rate
This measures how close an adaptive design's performance is to the best possible performance. The paper proves that Sigmoid-FTRL achieves a convergence rate of $T^{1/2R}$, which is the minimax optimal rate, showing no other adaptive design can perform better in the worst-case scenario.

Terminology

Summary

Sigmoid-FTRL for Adaptive Neyman Allocation in AIPW Estimation

This research paper presents Sigmoid-FTRL, a novel adaptive experimental design tailored for Adaptive Neyman Allocation within the framework of AIPW (Augmented Inverse Probability Weighting) estimators. The primary objective is to construct an adaptive procedure that minimizes the Neyman Regret, defined as the difference between the variance of the adaptive procedure and that of an oracle estimator utilizing optimal nonadaptive assignment probabilities and linear predictors.

Core Methodology: Sigmoid-FTRL

The design, Sigmoid-FTRL, addresses inherent non-convexity in adaptive allocation problems by employing a simultaneous minimization strategy targeting two distinct convex regrets: probability regret and prediction regret. The algorithm operates by applying the Follow-the-Regularized Leader (FTRL) principle separately to these two objectives. p. 2

Where the paper says it

In this paper, we make the following contributions which resolve these open questions: • Optimal Rates and Experimental Design: We present Sigmoid-FTRL, a new adaptive experimental design under which the AIPW Neyman Regret converges at a rate of T ´1{2R, where T is the number of subjects and R is the maximum covariate norm. To overcome the issue of non-convexity, the design simultaneously minimizes two convex objectives corresponding to the selection of treatment assignment probability and linear predictors, respectively.

Specifically:

  1. Linear Predictors: These are chosen to minimize estimated squared prediction errors.

  2. Treatment Probability: This is selected to minimize a weighted sum of online residuals, augmented by a regularization term provided by a sigmoid regularizer, which helps navigate the non-convex landscape effectively.

The design's computational tractability is noted, with complexity scaling as O(d 3q) per iteration and O(d 2q) total storage.

Key Theoretical Results and Performance Guarantees

The analysis rigorously establishes the performance characteristics of Sigmoid-FTRL under standard regularity conditions (Assumptions 1-3 and Condition 1). The central findings are:

1. Neyman Regret Convergence Rate:

The paper proves that the Neyman Regret converges at a rate of O(T 1/2R), where T is the number of subjects in the experiment and R is the maximum norm of covariate vectors. This result is significant because it establishes ** T 1/2R as the minimax rate** for Neyman regret under these regularity conditions, demonstrating that no adaptive design can achieve a better worst-case performance.

2. Regret Decomposition:

The total Neyman Regret (R Neyman T) is decomposed into two convex regret components:

R Neyman T = 1 over T ErR prob T + 1 over T ErR pred T

Both the expected probability regret (ErR prob T) and the expected prediction regret (ErR pred T) are bounded by O(T 1/2R) (as shown in Propositions 4.8 and 4.13, respectively). p. 2

Where the paper says it

In this paper, we make the following contributions which resolve these open questions: • Optimal Rates and Experimental Design: We present Sigmoid-FTRL, a new adaptive experimental design under which the AIPW Neyman Regret converges at a rate of T ´1{2R, where T is the number of subjects and R is the maximum covariate norm. To overcome the issue of non-convexity, the design simultaneously minimizes two convex objectives corresponding to the selection of treatment assignment probability and linear predictors, respectively.

3. Variance Convergence and Asymptotic Normality:

A crucial aspect of the analysis is establishing the convergence of the estimator's variance.

  • Oracle Variance Equivalence: Theorem 5.1 proves that under Assumptions 1-3 and Condition 1, the asymptotic variance of the adaptive AIPW estimator converges exactly to its oracle variance: V hat AIPW V oracle, where V oracle = 2p 1' rho q E[p 1 q] E[p 0 q].

  • Central Limit Theorem (CLT): The CLT for the AIPW estimator is established in Section D.3, relying on the Martingale Central Limit Theorem and controlling conditional variance convergence. p. 0

    Where the paper says it

    Moreover, we show that no adaptive design can improve upon the T ´1{2R rate under our regularity conditions, establishing the minimax rate of Neyman Regret. Finally, we establish a central limit theorem and a consistently conservative variance estimator which facilitate the construction of asymptotically valid Wald-type confidence intervals.

    This leads to the conclusion that the standardized adaptive AIPW estimator (tau hat - tau) is asymptotically standard normal (N(0, V oracle)). p. 6
    Where the paper says it

    4 Adaptive AIPW Estimators We focus on the class of adaptive Augmented Inverse Propensity Weighted (AIPW) estimators for the average treatment effect. The AIPW estimator is widespread in causal inference, where a regression model is used to improve the efficiency of the standard IPW estimator.

4. Confidence Interval Construction:

The analysis provides tools for practical inference:

  • Variance Estimator Consistency: The consistency of the estimated variance bound is verified (Corollary 5.8), with its asymptotic coverage bounded by O(T 5/12R 5/6). p. 3

    Where the paper says it

    More recent work has focused on adaptive experiment design for obtaining efficiency bounds (Laan and Lendle, 2014; Cook, Mishler, and Ramdas, 2024; Kato et al. These results are for the asymptotic variance and do not provide an analysis of how close the finite sample variance is to the efficiency bound.

  • Wald-Type Intervals: Combining the CLT and the consistent variance estimator yields asymptotically valid Wald-type confidence intervals (CI alpha = tau hat + phi-1(alpha)), which are guaranteed to cover the true parameter at least at the nominal level (Corollary 5.9).

Conclusion

In summary, Sigmoid-FTRL successfully achieves a **Neyman Regret convergence rate of T 1/2R **, which is proven to be minimax optimal under the stated regularity conditions. Furthermore, it provides a complete statistical framework by confirming that the estimator is asymptotically normal and its variance estimate is consistent, allowing for reliable inference via Wald-type confidence intervals. p. 3

Where the paper says it

We also construct a consistent estimator for Neyman’s variance bound. Together, these enable the development of Wald-type intervals that asymptotically cover at least at the nominal level.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Adaptive experimental design for AIPW estimators can select both treatment assignment probability and linear predictors simultaneously to minimize Neyman Regret, which is defined as the difference between the variance of an adaptive procedure and an oracle variance which uses the optimal nonadaptive choice of assignment probabilities and linear predictors.

  2. The Sigmoid-FTRL design allows the system to address the non-convexity challenge by simultaneous minimization of two convex regrets: probability regret and prediction regret, leading to a convergence rate for Neyman Regret at T'1/2R.

  3. The system can achieve an asymptotically normal distribution for the AIPW estimator under Sigmoid-FTRL, facilitating the construction of Wald-type confidence intervals where the central limit theorem holds under the same conditions required for non-superefficiency.

  4. A consistent estimator for Neyman’s variance bound can be constructed, allowing the system to create Wald-type intervals that asymptotically cover at least at the nominal level, even when the residual correlation rho is not estimable.

  5. The system can provide a lower bound on Neyman Regret of order T'1/2R, which establishes that no adaptive design can improve upon the T'1/2R rate under our regularity conditions, setting a minimax rate for the regret.

Abstract

We consider the problem of Adaptive Neyman Allocation for the class of AIPW estimators for estimating the average treatment effect in a design-based setting, where potential outcomes and covariates are deterministic. As each subject arrives, an adaptive procedure must select both a treatment assignment probability and a pair of linear predictors to be used in the AIPW estimator. Our goal is to construct an adaptive procedure that minimizes the Neyman Regret, which is the difference between the variance of the adaptive procedure and an oracle variance which uses the optimal non-adaptive choice of assignment probabilities and linear predictors. While previous work has drawn fruitful connections between Neyman Regret and online convex optimization for the Horvitz--Thompson estimator, one of the central challenges for the AIPW estimator is that the underlying optimization is non-convex. In this paper, we propose Sigmoid-FTRL, an adaptive experimental design which addresses the non-convexity via simultaneous minimization of two convex regrets. We prove that under standard regularity conditions, the Neyman Regret of Sigmoid-FTRL converges at a sqrt d/T rate, where d is the dimension of the covariates and T is the number of subjects in the experiment. We establish that this T dependence is optimal by proving a T-1/2 lower bound on the Neyman Regret for any adaptive design under our regularity conditions. Finally, we establish a central limit theorem and a consistently conservative variance estimator which facilitate the construction of asymptotically valid Wald-type confidence intervals.

Sources

Related papers