Bias-corrected Cox regression with AI-extracted covariates via calibration summary statistics
Arjun Sondhi
stat.ME, stat.ML
Submitted: 2026-07-28
Comments: Accepted to MLHC 2026
Code: https://github.com/asondhi/cox-bias-cov
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large-scale observational studies increasingly rely on AI pipelines to extract structured variables from unstructured clinical records.
Terminology
Abstract
Large-scale observational studies increasingly rely on AI pipelines to extract structured variables from unstructured clinical records. A common workflow separates the data vendor, who validates extraction accuracy with a gold-standard sample, from the downstream researcher, who receives only the extracted dataset and summary accuracy statistics. We develop a bias-correction framework for the Cox proportional hazards model when covariates are subject to AI extraction error. Within a unified multivariate calibration framework, we show that the naive Cox estimator's bias decomposes into a leading-order calibration term and a second-order residual that vanishes as extraction accuracy improves. The leading-order term yields a corrected estimator that operates as a post-hoc matrix multiplication on the output of any standard Cox software. We further derive bias-adjusted confidence intervals that incorporate calibration uncertainty and a sensitivity diagnostic for assessing whether the neglected residual could materially affect inference. Synthetic data experiments with cross-dependent extraction errors and controlled nonlinear calibration violations confirm that the correction substantially reduces bias and achieves near-nominal coverage even under mild violations of the linear calibration assumption. The framework yields a concrete reporting specification: a short list of summary statistics that data vendors should provide alongside any AI-extracted covariate dataset used in survival analysis.
Related papers
- Doubly robust inference via calibration
- Bayesian Empirical Bayes: Simultaneous Inference from Probabilistic Symmetries
- Weak-Form Recovery of Stochastic Generators
- Optimal Watermark Localization in Mixed-Source Large Language Model Texts
- Optimized Conformal Selection: Powerful Selective Inference After Data-Dependent Score Construction
- Small area learning