Prospective Prediction of OOD Degradation from Source-Side Training Dynamics
The study investigated whether looking at how a model's internal states change over time (training dynamics) can predict future performance drops when tested on new data (OOD degradation). Researchers found that using summaries of past training history, rather than just current values, significantly improves the ability to forecast these failures. This suggests that the temporal structure of source-side quantities holds crucial predictive information.