On the estimation and validity of AI time horizons---a statistical look at the METR plot
The study reanalyzed METR's time horizon plot using two new statistical models: a shared monotone spline and an explanatory Item-Response Theory model. These models provide better estimates by relaxing assumptions about how AI difficulty relates to human time. New diagnostic plots help assess the validity of these time horizons, suggesting that interpretation should change around the 2-30 minute mark.