On the Tightness and Computational Tractability of Higher-Dimensional Confidence Sequences
stat.ML, stat.ME
Submitted: 2026-08-21
Updated: 2026-10-08
Comments: Accepted at NeurIPS 2026 (Spotlight)
License: http://creativecommons.org/licenses/by/4.0/
The gist: Modern sequential monitoring problems often involve multiple metrics, where we monitor several data streams simultaneously and may act once the evidence is strong enough.
Terminology
Abstract
Modern sequential monitoring problems often involve multiple metrics, where we monitor several data streams simultaneously and may act once the evidence is strong enough. Confidence sequences (CSs) are a natural tool for such continuous monitoring. However, for bounded vector means, existing multivariate CSs are either tight but computationally intractable, or fast to compute but conservative. To address this, we study three lifts of one-dimensional betting-based CSs to higher dimensions: a weighted Bonferroni region, an equivalent max-wealth form, and a portfolio region. The portfolio is typically much tighter, especially in higher dimensions, but its boundary and properties such as volume are not available in closed form. To make this tighter construction usable, we propose tractable outer approximations of the portfolio region that preserve statistical validity: a bounding box, an p-ellipsoid, and their intersection. We prove set relations among all constructions and show empirically that these approximations (i) achieve regions close to the intractable portfolio, (ii) substantially outperform existing tractable multivariate CSs, and (iii) enable practical use cases such as multi-metric A/B testing and model comparison.
Sources
- Always Valid Inference: Bringing Sequential Analysis to A/B Testing
- Empirical Bernstein in smooth Banach spaces
- Time-Uniform Confidence Spheres for Means of Random Vectors
- Time-Uniform Self-Normalized Concentration for Vector-Valued Processes
- Efficient Post-Hoc Uncertainty Calibration via Variance-Based Smoothing
- Probabilistic Artificial Intelligence
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing
- From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier