Memory--Batch Tradeoffs in Lipschitz Bandits
cs.LG, stat.ML
Submitted: 2026-08-08
Updated: 2026-09-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Adaptive learning needs both a state that preserves what observations imply and opportunities to act on that state.
Terminology
Abstract
Adaptive learning needs both a state that preserves what observations imply and opportunities to act on that state. We study this width--depth tradeoff in stochastic Lipschitz bandits. After each pull, the learner retains at most W bits of live reward-dependent state and organizes its pulls into at most B committed batches. For W d (eT), we characterize minimax expected pseudo-regret up to logarithmic factors; the lower bounds hold for every W. Besides the classical sequential and unrestricted-memory batch costs, the frontier contains the new penalty [ T d+2 over d+3 (1+(B-1)W)- 1 d(d+3),] proving that state width and update depth are not interchangeable. The interaction is an information-routing constraint: at regional scale s, low regret forces the committed action transcript to encode d(s-d) regional decisions, while the collected boundary states carry at most (B-1)W bits of entropy. Matching policies stream and erase verification statistics while retaining a mask of a safe active set, either in memory or fragment by fragment. The theorem recovers the full-dimensional worst-case batch-only frontier and logarithmic-memory achievability in the fully sequential specialization; static batch boundaries match predictable adaptive ones.
Sources
- Few Batches or Little Memory, But Not Both: Simultaneous Space and Adaptivity Constraints in Stochastic Bandits
- Batched Stochastic Linear Bandits with 1-Bit Communication Constraints
- Tight Memory-Regret Lower Bounds for Streaming Bandits
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks