Lotus: Learning to Understand Time-series by Learning to Forecast

arXiv:2610.18877 · cs.LG, cs.AI, cs.CL · Submitted 2026-10-01 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Lotus: Learning to Understand Time-series by Learning to Forecast".

Jane: The gist The introduction introduces forecast grounding, a pretraining principle requiring language-aligned time-series representations to predict future evolution before adaptation,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper called "Lotus: Learning to Understand Time-series by Learning to Forecast". Basically, it tackles the problem that when we use time-series language models, they usually just link numbers to words using things like captions or question-answer pairs.

Jane: Right, so those methods teach the model how to produce the right language output, but they don't really force the model to actually understand *how* those numbers change over time.

Lu: That's exactly what this paper is addressing with forecast grounding. They propose a pretraining principle where a language-aligned representation has to be able to predict the future evolution of the signal before it even gets used for any specific task.

Meng: So, the core idea is that this forecasting objective acts like an extra constraint during pretraining, forcing the learned representation to keep track of temporal information instead of just memorizing labels.

Tom: They do this by building a model called LOTUS which divides each time-series channel into non-overlapping patches and maps those patches into the same embedding space as a language model.

Jane: And then what's the mechanism they use to make sure that representation stays predictive of the future?

Lu: They introduce something called a temporal-spectral patch representation. This means for every raw time-series patch, they combine it with its spectral representation before mapping it into the LLM embedding space.

Meng: So, you're saying they're feeding the model two views of each chunk of data—the raw temporal view and the spectral view—to capture both how things change over time and any periodic patterns too.

Tom: Exactly. They project these two views using separate learnable projections, Wt for temporal and Wf for spectral, and then fuse them with Wfuse to get a shared representation, hc,n.

Jane: That sounds clever because it’s trying to capture both the local dynamics of the patch and its underlying frequency characteristics simultaneously within that single embedding.

Lu: The pretraining objective they use is a mix: language generation objectives and next-patch prediction objectives, which they call LPG. This means the model is trained to produce answers while simultaneously predicting what the next patch in the time-series will look like from its hidden states.

Paper summary: Meng: That sounds like a dual training signal, constraining it both linguistically and temporally at the same time, which should help stabilize that representation learning.

Tom: The results they got are pretty solid: across three different language model backbones and six downstream evaluation datasets, LOTUS achieved the best average performance with very few trainable parameters.

Jane: They specifically mention reaching an accuracy of fifty-nine point nine one percent on a Qwen3-0 point 6B backbone, which is four point five eight percent better than TS-Reasoner-7B-FT while being fourteen times smaller in size.

Lu: And their analysis shows that when they look at the learned patch representations, they confirm that LOTUS actually uses the numerical input and preserves the temporal structure in those representations.

Meng: That's good for deployment because it means you don't need a massive model just to get decent time-series performance. If I had to deploy this, I’d be really happy with those parameter counts.

Tom: It does sound promising, and the paper frames forecast grounding as a way to make compact time-series language models perform much better than models that are twenty-five to fifty times their size on TSRBench.

Jane: So, the title of this work is "Lotus: Learning to Understand Time-series by Learning to Forecast", and it seems like the authors are really pushing the idea that forecasting is a necessary constraint for learning good time-series representations.

Lu: They argue that existing methods focusing only on language supervision don't explicitly require preserving how the underlying signal evolves, which is why they introduce this forecast grounding principle.

Meng: I wonder what happens when you try to adapt these models to something completely different, like a new type of time-series task?

Tom: The paper shows they can do that by removing the next-patch decoder during adaptation and continuing to train just with language supervision, which suggests the representation learned in phase one is quite flexible.

Jane: That makes sense. If the representation is already grounded in temporal evolution, it should transfer better when you switch to a different downstream task.

Lu: They also showed ablation results where removing the frequency branch—the spectral view—lowered the average accuracy by about three point eight percent in phase two alone and four point seven percent in both phases.

Paper summary: Meng: Hmm, so it’s important to have both views, even if one seems less useful at first glance, because losing that part hurts the final performance a bit.

Tom: And they also showed that the model's predictions really depend on the numerical input; when they corrupt or zero out the numerical input on something like HAR data, accuracy drops significantly from fifty-eight point four percent down to about twenty-six point nine to thirty-two point two percent.

Jane: That’s a big caveat—it shows that while the representation learns temporal structure, it still fundamentally needs the actual numerical signal to make any useful prediction; it doesn't just become a pure language model divorced from the data.

Lu: So, they are saying this pretraining principle is a strong direction because it enables these smaller models to achieve performance that was previously only possible with much larger ones.

Meng: For us in production, the main question would be how much compute is needed to run this whole pretraining process compared to the complexity of the final language model itself.

Tom: That’s a fair point about the cost, but if you look at what they are saying about performance gains relative to model size, it suggests a huge efficiency win for those who can afford that initial training.

Jane: So, to wrap up this discussion on "Lotus: Learning to Understand Time-series by Learning to Forecast", the main point is that forcing a language-aligned time-series representation to predict future signal evolution through forecast grounding provides a strong additional signal during pretraining.

Lu: This allows the model to learn representations that actually retain information about future signal evolution, which is what they call forecast grounding.

Meng: And it seems like this approach makes compact models competitive by giving them a richer understanding of the time-series data upfront.

Tom: It's an interesting constraint to add—using forecasting as a pretraining objective even when the final task only asks for language output, which is what they are demonstrating.

Jane: So, while it’s not trying to solve every single reasoning problem directly, it builds a much stronger foundation for time-series understanding before you even start fine-tuning.

Conclusion: Tom: So we’ve been looking at LOTUS, which is titled "Lotus: Learning to Understand Time-series by Learning to Forecast". It’s a paper from researchers at Tsinghua University, and they did something pretty specific with time-series language models.

Jane: Right, so what was the big idea behind that title? What are they actually trying to do that isn't already being done?

Tom: The core concept is forecast grounding. It’s this pretraining principle where the language representation has to predict how the signal evolves before it gets used for any specific task. It’s not just about getting right answers, it’s about forcing the model to keep track of time.

Lu: Exactly. They argue that you can have a language-aligned time-series representation, but if it can't predict the future evolution, it isn't truly grounded in the data structure. That constraint is what they call forecast grounding.

Meng: So during pretraining, you’re training the model on two things simultaneously: generating text and predicting what comes next in the numbers. It’s a dual objective for representation learning.

Jane: And how does that dual training actually work inside the LOTUS architecture? What are they building with those language models?

Tom: They divide every single time-series channel into non-overlapping patches, and they map those patches into the LLM embedding space. The key thing is that each patch gets a temporal and a spectral view combined before it hits the embedding layer.

Lu: That temporal-spectral patch representation is crucial because it captures both how fast things change over time and any repeating patterns in the data, using both raw time and its Fourier transform.

Jane: That’s interesting—combining those two views into one representation to keep things grounded in reality. But what are they saying about the actual performance numbers?

Tom: Across three different language model backbones and six evaluation datasets, LOTUS got the best average performance with a surprisingly small number of trainable parameters.

Jane: And what kind of numbers are we talking about for that performance? What’s on those charts?

Tom: They hit fifty-nine point nine one percent accuracy on a Qwen3-zero point 6B model, and they say that’s four point five eight percent better than a much bigger model while being fourteen times smaller in backbone size. That's what caught my eye as an engineer, because of the efficiency gain there.

Meng: But what about the limitations? Does this work on any kind of data? What happens if you try to use it on something completely different, say a new type of sensor signal?

Jane: That’s a fair question. The paper shows they can adapt the model for different downstream tasks by just removing that next-patch decoder and continuing to train it with language supervision alone.

Tom: That suggests the representation learned in that first phase is pretty flexible, which is what we want when we're building general models.

Lu: Yeah, but they also showed that their predictions are really dependent on the numerical input. If you corrupt or zero out the raw numbers on something like HAR data, accuracy drops significantly from fifty-eight point four percent down to about twenty-six to thirty-two percent.

Jane: So, it’s not just a clever language trick; the model still fundamentally needs that actual numerical information to be useful. That’s an important caveat for anyone thinking about deploying this on real-world time-series data.

Tom: Exactly. So, while they didn't claim a total revolution yet, they established forecast grounding as a solid pretraining principle that makes compact models way more competitive on benchmarks like TSRBench.

Jane: It’s definitely a strong direction to consider for building smarter time-series AI, especially if you’re looking to keep your model size small but still want it to understand the underlying physics of the data.

Lu: That's right. The next thing we need to look at is how they handle those spectral representations when dealing with very complex, non-stationary signals.

Van Dai Do, Huu Hiep Nguyen, Minh Hoang Nguyen, Manh Nguyen, Hung Le Deakin

Deakin's Applied Artificial Intelligence Initiative, Australia

cs.LG, cs.AI, cs.CL

Submitted: 2026-10-01

Updated: 2026-10-01

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 92/100

The gist: The gist The introduction introduces forecast grounding, a pretraining principle requiring language-aligned time-series representations to predict future evolution before adaptation, which is

Key concepts

Forecast Grounding: This is a pretraining principle requiring language-aligned time-series representations to be predictive of the future evolution of the underlying signal. It acts as a constraint during training, ensuring the learned representation retains crucial temporal information necessary for forecasting, rather than just being good at generating text.
Temporal–Spectral Patch Representation: LOTUS combines each raw time-series patch with its spectral representation before embedding it into the LLM space. This dual view—raw temporal coordinates and Fourier-based spectral coordinates—allows the model to capture both how signals change over time and their underlying periodic patterns simultaneously.
LOTUS Architecture: The architecture divides time-series channels into non-overlapping patches mapped to LLM embeddings. Training involves a joint objective: language generation (Llang) and next-patch prediction (Lpred). After pretraining, the model adapts to downstream tasks by focusing only on language supervision for encoder and LLM fine-tuning.

Summary

The gist The introduction introduces forecast grounding, a pretraining principle requiring language-aligned time-series representations to predict future evolution before adaptation, which is instantiated in LOTUS to encourage temporal information preservation

Forecast Grounding Principle

The paper introduces forecast grounding as a pretraining principle that mandates language-aligned timeseries representations must remain predictive of the future evolution before they are adapted to downstream tasks This objective serves as a pretraining constraint that encourages the representation supplied to the LLM to preserve future-relevant temporal information rather than being an additional downstream task This principle is implemented in LOTUS by having language examples train the model to generate answers while forecasting examples train it to predict the next patch from the LLM states of the signal tokens

Temporal–Spectral Patch Representation

LOTUS proposes a temporal–spectral patch representation that combines each raw time-series patch with its spectral representation before mapping it into the LLM embedding space This approach captures both temporal changes and periodic patterns by representing each patch using both temporal and Fourier coordinates Specifically, for a patch xc,n, the raw patch is used as the temporal view while its real FFT is used for the spectral view These views are embedded separately by learnable projections Wt and Wf and then fused by Wfuse to create a shared representation hc,n

LOTUS Architecture and Training Phases

The LOTUS architecture divides each time-series channel into non-overlapping patches which are then mapped into soft tokens in the LLM embedding space The process involves three phases: Phase 1 (Forecast Grounding Pretraining), Phase 2 (Downstream Adaptation), and the downstream task adaptation During pretraining, language supervision trains textual outputs while next-patch prediction constrains LLM’s hidden states to retain future-relevant information After pretraining, the model is jointly adapted to different downstream time-series tasks by removing the next-patch decoder and continuing to adapt the encoder and LLM using language supervision alone

Evaluation and Performance

Across three language model backbones on six downstream evaluation datasets, LOTUS achieves the best average performance with minimal trainable parameters For instance, on Qwen3-0.6B it achieves an overall accuracy of 59.91%, which is 4.58% above TS-Reasoner-7B-FT while being 14 times smaller in backbone size The representation analyses further show that LOTUS uses the numerical input and preserves temporal structure in its learned patch representations

Ablation Results

Ablation studies confirm the benefits of the components, showing that removing the frequency branch lowers six-task average accuracy by 3.8% in Phase 2 alone and 4.7% in both phases Furthermore, input diagnostics show that accuracy on HAR falls from 58.4% to 26.9–32.2% when the numerical input is corrupted with noise or zeroed out This demonstrates that the model’s predictions depend materially on the numerical input

Conclusion

The paper establishes forecast grounding as a promising pretraining principle enabling compact time-series language models to outperform models 25–50× their size on TSRBench The findings support forecasting as a pretraining objective for time-series language models even when downstream tasks require only language outputs

How it works

Forecast grounding requires that during pretraining, a language-aligned time-series representation is also required to predict the future evolution of its source signal
LOTUS instantiates this by dividing each channel into non-overlapping patches and mapping them into the LLM embedding space
A temporal–spectral patch representation is proposed to combine raw temporal patches with their spectral representations before mapping them into the LLM embedding space
The pretraining objective is a mixture of language generation objectives (Llang) and forecasting objectives (Lpred), optimized jointly as LPG = Eb∼Dlang [Llang(b)] + Eb∼Dpred [Lpred(b)] This is achieved by projecting language-model hidden states back to the time-series representation space and applying a causally masked Transformer decoder followed by a linear prediction head
Downstream adaptation involves removing the next-patch decoder and training only with language supervision to adapt the encoder and LLM using LoRA

Improvements for AI systems

Forecast grounding will enable compact time-series language models to outperform models 25–50× their size on TSRBench, by requiring representations to remain predictive of the future evolution before they are adapted to downstream tasks.
The temporal–spectral patch representation will allow the model to capture both local temporal changes and frequency patterns by combining the raw patch with its Fourier transform before mapping it into the LLM embedding space.
The system can achieve superior performance across diverse backbones, as LOTUS achieves the best average accuracy in every matched backbone group while using only a small number of trainable parameters.

Abstract

Time-series language models typically align numerical signals with language using labels, captions, or question-answer pairs. Although such supervision encourages linguistic representations, it does not explicitly require them to preserve information about how the underlying time-series signal evolves. We introduce forecast grounding, a pretraining principle that requires language-aligned time-series representations to remain predictive of the future evolution before they are adapted to downstream tasks. We instantiate this principle in LOTUS, which divides each time-series channel into non-overlapping patches and maps them into the LLM embedding space. It then predicts the next patch from the LLM hidden states of the time-series tokens, encouraging the learned representation to retain temporal information. To capture both temporal changes and periodic patterns, we also propose a temporal-spectral patch representation that combines each raw time-series patch with its spectral representation before mapping it into the LLM embedding space. Across three language model backbones and six downstream evaluation datasets, LOTUS achieves the best average performance with minimal trainable parameters, reaching 59.91% accuracy and outperforming the strongest matched baseline by 3.36%. Our representation analyses further show that LOTUS uses the numerical input and preserves temporal structure in its learned patch representations. These results establish forecast grounding as a promising pretraining principle, enabling compact time-series language models to outperform models 25 to 50 times their size on TSRBench.