Behavior of prediction performance metrics with rare events
The episode discusses a paper analyzing how prediction performance metrics like AUC behave when event rates are low. Authors Emily Minus1, R. Yates Coley2, Susan M. Shortreed2, and Brian D. Williamson2 suggest AUC is reliable with at least one thousand events in large healthcare databases. The discussion concludes that reporting multiple metrics and their variability is necessary for a complete picture.