Post-Selection Distributional Model Evaluation
The authors introduce Post-Selection Distributional Model Evaluation (PS-DME), a framework for reliably assessing model performance distributions after arbitrarily selecting a subset of candidates based on data. PS-DME controls the false coverage rate, ensuring that the estimated performance distributions are statistically valid despite post-selection bias. This method enables users to compare models across reliability levels with statistical guarantees.