Multicalibration for Unbiased Model-Based Prevalence Estimation

arXiv:2604.21549 · cs.AI, stat.ME · Submitted 2026-04-23 · Read on arXiv

cs.AI, stat.ME

Submitted: 2026-04-23

Updated: 2026-10-01

Code: https://github.com/facebookresearch/multicalibrated_llm_measurement

Terminology

• Calibration — 3× in this paper · explained in 7 episodes
• Covariate Shift — 1× in this paper · explained in 2 episodes

Sources

• How should we proxy for race/ethnicity? Comparing Bayesian improved surname geocoding to machine learning methods
• The Llama 3 Herd of Models
• Measuring multi-calibration
• On Causal and Anticausal Learning
• Class Prior Estimation under Covariate Shift: No Problem?
• Invariance assumptions for class distribution estimation

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems