LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting
cs.LG, cs.AI
Submitted: 2025-10-23
Updated: 2025-10-23
Comments: 15 pages, 8 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Forecasting in the real world requires integrating structured time-series data with unstructured textual information…
Terminology
Abstract
Forecasting in the real world requires integrating structured time-series data with unstructured textual information, but existing methods are architecturally limited by fixed input/output horizons and are unable to model or quantify uncertainty. We address this challenge by introducing LLM-integrated Bayesian State space models (LBS), a novel probabilistic framework for multimodal temporal forecasting. At a high level, LBS consists of two components: (1) a state space model (SSM) backbone that captures the temporal dynamics of latent states from which both numerical and textual observations are generated and (2) a pretrained large language model (LLM) that is adapted to encode textual inputs for posterior state estimation and decode textual forecasts consistent with the latent trajectory. This design enables flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization thanks to the well-suited inductive bias of SSMs toward modeling dynamical systems. Experiments on the TextTimeCorpus benchmark demonstrate that LBS improves the previous state-of-the-art by 13.20% while providing human-readable summaries of each forecast. Our work is the first to unify LLMs and SSMs for joint numerical and textual prediction, offering a novel foundation for multimodal temporal reasoning.
Sources
- TimeSeriesExam: A time series understanding exam
- Adapting Language Models to Compress Contexts
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- In-context Autoencoder for Context Compression in a Large Language Model
- The Llama 3 Herd of Models
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Mastering Atari with Discrete World Models
- Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data
- Deep Kalman Filters
- Prefix-Tuning: Optimizing Continuous Prompts for Generation
- Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative
- Language Models Still Struggle to Zero-shot Reason about Time Series
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
- Compute Optimal Scaling of Skills: Knowledge vs Reasoning
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data
- Intervention-Aware Forecasting: Breaking Historical Limits from a System Perspective
- Qwen2.5 Technical Report
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis