Statistical Inference under Performativity: From Policy Feedback to Self-Evolving Systems
cs.LG, stat.ML
Submitted: 2026-10-30
Updated: 2026-10-30
Comments: Extended version of the NeurIPS 2025 paper "Statistical Inference under Performativity" (arXiv:2505.18493)
License: http://creativecommons.org/licenses/by/4.0/
The gist: Performativity arises when prediction-informed decisions change the data-generating process, creating feedback between deployment and learning.
Terminology
Abstract
Performativity arises when prediction-informed decisions change the data-generating process, creating feedback between deployment and learning. This phenomenon occurs in policy-making and in self-evolving AI systems that shape their own future training data. Statistical inference is challenging because estimation errors in earlier predictors affect subsequent data distributions and propagate into later updates. We develop an end-to-end framework for statistical inference under performativity. For repeated risk minimization and repeated gradient descent, we establish central limit theorems at fixed deployment rounds and derive covariance recursions that capture both current-round sampling variation and uncertainty inherited from earlier deployments. We develop consistent covariance estimators using policy perturbations and score matching. These results can yield asymptotically valid confidence regions for population iterates and the performative stable point. We further extend prediction-powered inference to this dynamic setting, combining labeled observations with predictions on additional unlabeled data to improve statistical efficiency. We evaluate our framework in synthetic, semi-synthetic educational, and self-evolving large language model routing settings. Our experiments demonstrate that ignoring propagated uncertainty can lead to substantial undercoverage even after repeated retraining. Together, these results provide statistical tools for learning systems that help determine the distributions from which they learn. Our work establish a complete statistical inference under performativity, introducing new challenges and inference settings that we believe will provide substantial value to policy-making, statistics, and machine learning.
Sources
- PPI++: Efficient Prediction-Powered Inference
- Bias-Aware Inference in Regularized Regression Models
- Microfoundation Inference for Strategic Prediction
- Training Verifiers to Solve Math Word Problems
- FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference
- Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation
- Performative Prediction: Past and Future
- Bayesian Prediction-Powered Inference
- Tight Lower Bounds and Improved Convergence in Performative Prediction
- Making Decisions under Outcome Performativity
- Plug-in Performative Optimization
- Stochastic Optimization for Performative Prediction
- Treatment Effects in Market Equilibrium
- Bias-Aware Inference in Fuzzy Regression Discontinuity Designs
- Incentivizing honest performative predictions with proper scoring rules
- Revisiting the Predictability of Performative, Social Events
- Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
- Policy design in experiments with unknown interference
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via Past-Future
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis