Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification
This research tested whether using very large pretraining datasets guarantees better robustness when fine-tuning models for specific image tasks. Findings show that larger foundational models can actually suffer greater robustness loss after specialization. The study concludes that tailored fine-tuning strategies, especially those involving continual learning and specific regularization techniques, are necessary to preserve model reliability.