A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks
cs.LG
Submitted: 2026-03-23
Updated: 2026-09-22
License: http://creativecommons.org/licenses/by/4.0/
The gist: In-context learning (ICL) enables task adaptation at inference time by conditioning on demonstrations rather than updating model parameters.
Terminology
Abstract
In-context learning (ICL) enables task adaptation at inference time by conditioning on demonstrations rather than updating model parameters. Although recent time-series foundation models incorporate contextual conditioning, retrieval, or example-based prompting, they typically rely on implicit positional structure or task-specific objectives rather than explicit instruction-conditioned input-output demonstrations. We introduce iAmTime, a time-series foundation model trained with instruction-conditioned amortized meta-learning to infer tasks directly from example demonstrations. iAmTime represents each episode as a structured prompt over historical context and future-known variables using specialized semantic tokens that attend to designated time-series regions, exchange information across demonstrations, and inject task information into the query representation. The model combines a Hierarchical Multi-Scope Transformer Encoder, which captures temporal and covariate dynamics while inferring latent task structure from demonstrated input-output mappings, with a Task-Conditioned Patch Decoder, which adapts decoding through expert-based routing. We train iAmTime on large-scale real and synthetic corpora using supervised and self-supervised instruction-conditioned tasks, including forecasting, imputation, reconstruction, classification, anomaly detection, and source de-mixing. Across diverse domains, frequencies, and horizons, iAmTime improves zero-shot adaptation over strong time-series foundation baselines on probabilistic and point forecasting benchmarks, while achieving competitive or superior performance on four non-forecasting tasks.
Sources
- GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation
- Understanding intermediate layers using linear classifier probes
- Chronos: Learning the Language of Time Series
- Chronos-2: From Univariate to Universal Forecasting
- TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
- The UEA multivariate time series classification archive, 2018
- This Time is Different: An Observability Perspective on Time Series Foundation Models
- In-Context Fine-Tuning for Time-Series Foundation Models
- ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels
- Deep learning for time series classification
- Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
- From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting
- In-context Time Series Predictor
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
- N-BEATS: Neural basis expansion analysis for interpretable time series forecasting
- An Overview of Multi-Task Learning in Deep Neural Networks
- fev-bench: A Realistic Benchmark for Time Series Forecasting
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Finetuned Language Models Are Zero-Shot Learners
- When Do Curricula Work?
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks