Approximate minimax regret for conditionally similar tests
summary
The gist
We construct approximately minimax regret optimal tests in a scalar Gaussian instrumental variables experiment This example shows how conditionally similar tests allow a novel approximation approach…
In short
This work constructs minimax regret optimal tests for a scalar Gaussian instrumental variables experiment using conditionally similar tests. By allowing weights to be optimized over alternatives, the method minimizes worst-case power regret. The algorithm uses negative-entropy mirror descent to update weights based on estimated regret, achieving strong approximation guarantees.
Key concepts
- Conditional Similarity
- This condition means that for every possible alternative hypothesis ($ heta$), the test's performance is similar when conditioned on the observed data ($S$ and $U$). This allows the test designer to optimize weights across alternatives instead of just focusing on a single null distribution.
- Worst-Case Power Regret
- This measures the maximum loss in power experienced by any alternative hypothesis under the worst possible conditions. The goal of this method is to minimize this worst-case regret, ensuring robustness even if the true underlying alternative is not perfectly specified.
- Conditional Neyman–Pearson Optimization
- This optimization focuses on finding the best rejection rule for a specific null distribution given a fixed alternative $ heta$. It rejects when the likelihood ratio exceeds a certain threshold derived from the weighted average power, leading to adaptive test construction.
- Negative-Entropy Mirror Descent
- This is an algorithm used to update the weights ($w_i$) in proportion to how much regret each alternative contributes. It iteratively increases the weight of alternatives that have shown higher regret, guiding the search toward a set of weights that balances power across all alternatives.
Terminology used across episodes
This episode discusses
- Approximate minimax regret for conditionally similar tests · Paper Radio
- Approximate Least-Favorable Distributions and Nearly Optimal Tests via Stochastic Mirror Descent
The paper
Approximate minimax regret for conditionally similar tests · Read on arXiv
Andrías Aradillas Fernándeza
Massachusetts Institute of Technology
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: I'm Ruth.
Warren: And I'm Warren, here with Iris.
Ruth: We're looking at the paper: "Approximate minimax regret for conditionally similar tests".
Warren: Here's the gist of the paper. We construct approximately minimax regret optimal tests in a scalar Gaussian instrumental variables experiment This example shows how conditionally similar tests allow a novel approximation approach, allowing us to optimize weights over alternatives to minimize worst-case power regret Testing Problem Setup The testing problem involves observing a pair of reduced-form estimates satisfying the given equation The transformation is invertible, and at theta = (β, π), the variables S and U are independent normals with means m(theta) = π(β − β0) and τ(theta) = π − a m(theta) and variances σ2 S and σ2 U Under the null, S U = u ∼ N(0, σ2 S), independently of π A randomized test is defined by a measurable rejection probability φ(s, u) ∈
0, 1: such that ∫ φ(s, u) f0 (s) ds = α for Lebesgue-a.e.
Ruth: u This restriction is equivalent to conditional similarity for every π Conditional Best Responses and Weight Updates The best response maximizes the weighted average power, defined as max φ∈ Cα ∑ i=1 w i Eθ i
φ: The constraint in (3) separates across u, leading to a conditional Neyman–Pearson optimization that rejects for large values of Lw (s, u) = gw (s, u) / f0 (s) The function F(w) = sup φ∈ Cα Í i=1 w i(Eθ i
φ: − b i) is convex, with subgradient G i(w) = Eθ i
φw: − b i Negative-entropy mirror descent thus increases weights in proportion to estimated regret (see Section 4.2 of Bubeck (2015), as well as Freund and Schapire (1999), after which this algorithm is modeled Algorithm Guarantees Proposition 1 states that for the exact conditional best responses in (5), the maximum grid regret is bounded by max i R (φ¯K, θi) ≤ V I + log I ηK + η8 With simulated updates, for any δ ∈ (0, 1), with probability at least 1 − δ over the simulation draws, the maximum grid regret is bounded by max i R (φ¯K, θi) ≤ V I + log I ηK + η2 + √2 log((I + 1)/δ) N K Numerical Illustration The numerical illustration sets β0 = 0, Σ = I squared, and α = 0.05, resulting in two alternatives with power envelopes of 0.4424 and 0.6388 For these two alternatives, the likelihood ratio in (6) is strictly convex in s The resulting rule has conditional size q + (α − q) = α by construction Table 1 shows that conditioning improves power at both alternatives relative to the equal-tail test After 4,251 iterations, maximum regret is 0.0776, compared with 0.0819 initially and 0.1228 for the equal-tail test Conditional similarity allows us to shift the numerical search from null distributions enforcing size to alternative distributions determining worst-case power regret Relative to EMW’s fixed WAP criterion, the alternative weights are chosen endogenously to balance regret within the conditionally similar class, allowing robustness to WAP misspecification Conditional Neyman–Pearson best responses and SMD deliver approximation guarantees while preserving exact conditional size This work can serve as the basis for future work moving beyond the Gaussian setup of the problem, with the built-in size control allowing for optimization over many alternative hypotheses Data and Code Availability We use simulated data The reproducible Python package, as well as instructions, are available at //github.com/aaf101/SMD-for-Conditional-Similarity Declaration of Generative AI and AI-assisted Technologies in the Manuscript Preparation Process During the preparation of this work, the author used GPT-6 to help with the writing, proof-checking, and code implementation in this paper The author directed this work and subsequently reviewed and verified all output The author takes full responsibility for the content of the article Proofs of Main Results A.
Paper summary: Ruth: So we've seen how they construct these approximate minimax regret optimal tests in a scalar Gaussian instrumental variables experiment, and what that setup allows them to do with the alternative hypotheses.
Warren: The central thesis here is that conditionally similar tests enable a novel approximation approach that lets them optimize weights over alternatives specifically to minimize worst-case power regret.
Iris: They show that stochastic mirror descent and averaging techniques yield finite iteration guarantees for exact and simulated powers when exact conditional similarity is maintained, which is a big deal for theoretical guarantees.
Ruth: The paper establishes bounds for both the exact and simulated powers, and they illustrate this whole procedure numerically to show how it works on the ground.
Warren: They use specific numerical settings like beta zero equal to zero and sigma squared S equal to one squared, with alpha set at point zero zero five for two alternatives with power envelopes of point four four two four and point six three eight eight.
Iris: Because of that convexity, the resulting rule has a conditional size of q plus alpha minus q equal to alpha by construction, which means it still maintains exact size control under these specific conditions.
Ruth: And they show that after a large number of iterations, like four thousand two hundred fifty-one iterations in their simulation, the maximum regret they found was point zero seven seven six compared to point zero eight one nine initially and point zero one two two eight for the equal-tail test.
Iris: The contribution of this work really lies in bringing together conditional Neyman–Pearson best responses with exponential weight updates, which helps preserve the geometry of the probability simplex during the iterations.
Conclusion: Ruth: Thinking about the title, "Approximate minimax regret for conditionally similar tests," it points to a method where you shift your search away from null distributions that enforce size to alternative distributions that determine worst-case power regret.
Warren: Basically, relative to the existing fixed weighted average power criterion used by EMW, these alternative weights are chosen endogenously to balance regret within that conditionally similar class.
Iris: This allows for robustness when you might have a misspecification of the weighted average power criterion because you’re balancing things based on what actually happens in the data under those specific conditions.
Ruth: So, this work shows that conditional similarity lets us shift the numerical search from null distributions enforcing size to alternative distributions determining worst-case power regret.
Warren: The method uses conditional Neyman–Pearson best responses and stochastic mirror descent to deliver approximation guarantees while preserving exact conditional size, which is a tough balance to strike.
Iris: It's a solid foundation for future work that can move beyond the Gaussian setup of the problem because this built-in size control allows for optimization over many alternative hypotheses.
More episodes
- Daily Summary for 2026-11-19
- 2608.21274-Recommendation Quality and the Concentration of Consumption: Experimental Evidence from Netflix
- 2408.02757-A nonparametric test for diurnal variation in spot correlation processes
- 2306.16393-High-Dimensional Canonical Correlation Analysis
- 2605.17299-Geometric Brownian motion with intermittent entries and exits
- 2607.04753-Fooling Yourself: how narratives shape beliefs
- 2408.14671-Inference on Treatment Effects with Repeatedly Mismeasured High-Dimensional Controls
- 2201.06898-Difference-in-Differences Estimators for Treatments Continuously Distributed at Every Period
- 2501.15307-Influence Functions: A Unified Differentiation-Based Framework
- The day in conversation for 2026-11-19