Bias-corrected Cox regression with AI-extracted covariates via calibration summary statistics

arXiv:2607.25868 · stat.ME, stat.ML · Submitted 2026-07-28 · Read on arXiv

Arjun Sondhi

stat.ME, stat.ML

Submitted: 2026-07-28

Comments: Accepted to MLHC 2026

Code: https://github.com/asondhi/cox-bias-cov

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large-scale observational studies increasingly rely on AI pipelines to extract structured variables from unstructured clinical records.

Terminology

Abstract

Large-scale observational studies increasingly rely on AI pipelines to extract structured variables from unstructured clinical records. A common workflow separates the data vendor, who validates extraction accuracy with a gold-standard sample, from the downstream researcher, who receives only the extracted dataset and summary accuracy statistics. We develop a bias-correction framework for the Cox proportional hazards model when covariates are subject to AI extraction error. Within a unified multivariate calibration framework, we show that the naive Cox estimator's bias decomposes into a leading-order calibration term and a second-order residual that vanishes as extraction accuracy improves. The leading-order term yields a corrected estimator that operates as a post-hoc matrix multiplication on the output of any standard Cox software. We further derive bias-adjusted confidence intervals that incorporate calibration uncertainty and a sensitivity diagnostic for assessing whether the neglected residual could materially affect inference. Synthetic data experiments with cross-dependent extraction errors and controlled nonlinear calibration violations confirm that the correction substantially reduces bias and achieves near-nominal coverage even under mild violations of the linear calibration assumption. The framework yields a concrete reporting specification: a short list of summary statistics that data vendors should provide alongside any AI-extracted covariate dataset used in survival analysis.

Related papers