Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?
The study compared large pre-trained time series models (LSTSMs) against smaller, non-pretrained transformers for forecasting. The results show that pre-trained LSTSMs, like Timer_XL, significantly outperform small models in both long-term and short-term forecasting tasks by minimizing prediction errors (MSE and MAE). This suggests that transfer learning from large datasets provides a major advantage over training smaller models from scratch.