Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting
stat.ML, cs.LG
Submitted: 2026-07-24
Updated: 2026-10-13
License: http://creativecommons.org/licenses/by/4.0/
The gist: Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information.
Terminology
Abstract
Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information. We introduce Hopformer (Homogeneity-Pursuit Transformer), a two-stage framework that addresses this challenge. In the first stage, we perform a Sparsity Pattern Aggregation (SPA) scheme extracting a common low-variance trend that incorporates the covariates. This acts as a homogenization layer. In the second stage, a LoRA-fine-tuned Transformer models the remaining complex dependencies in the residual. Our method is theoretically grounded. We prove that SPA achieves a near-optimal bias-variance trade-off via an oracle inequality. We also provide generalization bounds for the second stage under dependent time series data. Hopformer sets a new state of the art, improving MASE by an average of 6.56% across synthetic and real-world forecasting benchmarks.
Sources
- GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
- Chronos: Learning the Language of Time Series
- TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
- STD: A Seasonal-Trend-Dispersion Decomposition of Time Series
- Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series
- UniTS: A Unified Multi-Task Time Series Model
- Monash Time Series Forecasting Archive
- Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- The Power of Scale for Parameter-Efficient Prompt Tuning
- Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
- Timer: Generative Pre-trained Transformers Are Large Time Series Models
- N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
- Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
- TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables
- TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
- FlexTSF: A Flexible Forecasting Model for Time Series with Variable Regularities
- PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches