Ground-Truth Neighborhood Regularization for Reinforcement Learning Post-Training of Time Series Foundation Models
Jianqi Zhang, Xingyu Zhang, Zeen Song, Changwen Zheng, Fanjiang Xu, Wenwen Qiang
cs.LG, cs.AI
Submitted: 2026-08-08
Updated: 2026-08-11
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Time series forecasting (TSF) plays an important role in a wide range of real-world applications.
Terminology
Abstract
Time series forecasting (TSF) plays an important role in a wide range of real-world applications. Recently, time series foundation models (TSFMs), pretrained on large-scale datasets, have demonstrated strong generalization capabilities and emerged as an important paradigm for TSF. Reinforcement learning (RL) post-training has consequently attracted growing attention as a means of further improving their performance on downstream tasks. However, we find that, in certain forecast regions, RL post-training may gradually shift the output distributions of TSFMs away from the ground truth, thereby limiting their performance. We refer to this phenomenon as suboptimal collapse. Our analysis suggests that difficulty in initially sampling high-quality trajectories near the ground truth is an important contributing factor to suboptimal collapse. To address this issue, we propose Ground-Truth Neighborhood Regularization (GTN-R) for RL post-training of TSFMs. GTN-R uses the ground truth as a reference for locating high-quality regions and guides the model's probability mass toward the ground-truth neighborhood. This increases the probability of sampling high-quality trajectories, mitigates suboptimal collapse, and improves performance. Moreover, GTN-R can be flexibly integrated into various RL methods for TSFMs. Extensive experiments show its effectiveness.
Sources
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
- Toto: Time Series Optimized Transformer for Observability
- MOMENT: A Family of Open Time-series Foundation Models
- Adam: A Method for Stochastic Optimization
- Foundation Models for Time Series: A Survey
- TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning
- Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models
- Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
- Timer: Generative Pre-trained Transformers Are Large Time Series Models
- A Survey of Deep Learning and Foundation Models for Time Series Forecasting
- TimeHF: Billion-Scale Time Series Models Guided by Human Feedback
- Multi-Scale Finetuning for Encoder-based Time Series Foundation Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Benefits and Pitfalls of Reinforcement Learning for Language Model Planning: A Theoretical Perspective
- TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting
- Explore-Go: Leveraging Exploration for Generalisation in Deep Reinforcement Learning
- S-GRPO: Unified Post-Training for Large Vision-Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks