BOCO: Bayesian Online Contextual Optimization for Decision-Focused Online Learning

Decision-focused learning (DFL) trains predictive models to optimize downstream decisions rather than prediction accuracy alone. While recent studies have extended this paradigm to online settings with streaming data, existing online DFL methods generally maintain a point estimate, while their gradient-based updates require either a differentiable optimization layer or a problem-specific surrogate loss. Consequently, they can be unstable under limited data and difficult to apply across heterogeneous optimization problems. We introduce Bayesian Online Contextual Optimization (BOCO), a framework that maintains a decision-focused posterior over model parameters. BOCO aggregates the resulting predictions when prescribing decisions, thereby accounting for parameter uncertainty. To track this posterior in evolving environments, we develop two particle-based inference algorithms: a sequential Monte Carlo sampler for nondifferentiable problems and a function-space Stein variational gradient descent algorithm for differentiable problems. Across both real-world tasks, BOCO reduces running mean regret and temporal regret variability relative to two frequentist online DFL baselines. At the full horizon, relative to the best frequentist baseline, BOCO reduces running mean regret and temporal regret variability by 6.9% and 6.3% on knapsack and by 49.3% and 38.1% on energy scheduling, respectively. The gains are larger early in the data stream, a pattern consistent with a benefit from accounting for parameter uncertainty when data are limited.