Series-wise Cross-validation for Global Time Series Forecasting
stat.ML, cs.LG
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/vcerqueira/experiments-tscv
License: http://creativecommons.org/licenses/by/4.0/
The gist: We consider the problem of cross-validation (CV) in time series forecasting, with a focus on global forecasting models based on deep learning.
Terminology
Abstract
We consider the problem of cross-validation (CV) in time series forecasting, with a focus on global forecasting models based on deep learning. The prevailing CV strategy in forecasting is to partition data across time with the last part of each series used for testing. With global models trained on collections of time series, however, a fundamentally different approach becomes possible: partitioning across series and time. We formalize series-wise CV for global forecasting models and benchmark several instantiations, including K-fold CV, bootstrapping, and Monte Carlo CV, on 11 datasets with 10 neural network architectures. We evaluate CV procedures along five dimensions: performance estimation, overoptimism rate, model selection accuracy, regret, and downstream forecasting performance. We withhold a random fraction of series from training so that they appear only in the test set. This protocol simulates the emergence of new series not seen during training, a common scenario in global forecasting. Under this protocol, our results reveal a nuanced trade-off: series-wise CV yields better downstream forecasting accuracy by guiding model selection toward configurations that generalize across both time and series, while time-wise CV retains a slight advantage for estimating out-of-sample error.
Sources
- GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
- Chronos: Learning the Language of Time Series
- Long-term Forecasting with TiDE: Time-series Dense Encoder
- KAN: Kolmogorov-Arnold Networks
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
- N-BEATS: Neural basis expansion analysis for interpretable time series forecasting
- Deep Non-Parametric Time Series Forecaster
Related papers
- Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
- Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
- Quantifying Error Propagation and Model Collapse in Diffusion Models
- Behavior of prediction performance metrics with rare events
- Statistical Inference and Learning for Shapley Additive Explanations (SHAP)
- Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference