Counterfactual Online Conformal Prediction Under Adaptive Logging
cs.LG
Submitted: 2026-09-25
Updated: 2026-09-25
License: http://creativecommons.org/licenses/by/4.0/
The gist: Online conformal prediction can fail when predictions shape actions and actions determine which outcomes enter calibration.
Terminology
Abstract
Online conformal prediction can fail when predictions shape actions and actions determine which outcomes enter calibration. Standard adaptive methods may retain marginal coverage while systematically miscovering the counterfactual outcomes of rarely selected actions. This paper formalizes the failure through counterfactual coverage and introduces Propensity-Weighted Online Conformal Prediction, an inverse-propensity-weighted recursion that debiases calibration. A doubly robust variant further reduces nuisance bias to the product of outcome-model and propensity errors. Under positivity, the resulting coverage rate matches an information-theoretic lower bound up to logarithmic factors. Experiments on synthetic decision tasks, open bandit data, and financial rebalancing show that PW-OCP and DR-OCP improve counterfactual coverage and downstream regret without sacrificing prediction-set sharpness.
Sources
- Optimal Model Selection for Conformalized Robust Optimization
- E-Values Expand the Scope of Conformal Prediction
- Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation
- Conformal e-prediction in the presence of confounding
- Conformal Risk Training: End-to-End Optimization of Conformal Risk Control
- Calibrating Decision Robustness via Inverse Conformal Risk Control
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks