RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting

arXiv:2607.08234 · cs.LG, cs.AI · Submitted 2026-07-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "RhyMix: A Lightweight Adaptive Multi-Rhythm Network for Long-Term Time Series Forecasting".

Jane: The gist:

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We’ve just looked at the RhyMix paper, and it really comes down to how they managed to balance explicit seasonal knowledge with data driven multi scale convolutions using adaptive gating.

Jane: Right. It’s about taking the messy dynamics of real time series—those short term fluctuations, those long term trends—and fitting them into a structure that respects both the predictable cycles and the complex context.

Lu: The authors are proposing this dual path approach as a way to address the limitations of single path temporal modeling, where transformers might smooth out local variations or convolutions have limited receptive fields.

Meng: So they’re suggesting that by having two complementary encoding branches, one for cycles and one for multi scale convolutions, you get better coverage of the temporal dynamics.

Tom: Right. And the key mechanism is that adaptive gating dynamically fuses these two branches based on input characteristics to ensure the model adapts its behavior to the specific temporal patterns present in each sample.

Jane: It’s not just about having two inputs; it’s about how much weight you give each input based on what kind of time series you are feeding it right now.

Lu: They integrate a Cyclic Path that incorporates explicit seasonal inductive bias through learnable cyclic embeddings, capturing predictable rhythmic patterns, and the MSTCN-CA for contextual temporal representation learning.

Meng: I like the idea of explicit seasonal inductive bias because it’s something you can tune manually if you know your data has strong seasonality.

Tom: Exactly. They use four learnable cyclic embedding tables with periods Pk in twelve twenty-four forty-eight and one hundred sixty-eight time steps corresponding to half day daily two day and weekly cycles.

Jane: And then the MSTCN-CA path uses two stacked TemporalMSBlock with four parallel depthwise dilated convolutions with dilation rates d in one two four and eight to capture short term fluctuations at d=one medium term trends at d=two and long term dependencies at d=eight <ref:2607.08234#pg1>.

Lu: That multi scale aspect is designed to capture those different scales of temporal patterns effectively through these varying dilation rates.

Meng: So it’s like they are explicitly telling the model, "Hey, this data might have a daily pattern here, or maybe a weekly pattern there."

Tom: Right. And the adaptive gating mechanism at multiple levels modulates these contributions based on input characteristics to ensure the model adapts its behavior to the specific temporal patterns present in each sample.

Jane: That dynamic balancing is what really makes it flexible for different kinds of time series, not just one fixed structure.

Lu: They also use gate feature extraction which computes statistical descriptors like mean, standard deviation, last value, slope between the first and last step of a sequence. These compact features are the basis for all subsequent adaptive gating mechanisms they mentioned earlier.

Meng: So it’s taking those simple stats and turning them into complex control signals for the model flow? I need to understand how that translates to actual training stability.

Tom: It’s about using those compact features as a foundation for every subsequent adaptive gating mechanism, ensuring computational efficiency through linear complexity O(L·C) with respect to sequence length and channels.

Jane: And the idea is that this design keeps the model computationally efficient while still allowing it to be smart enough to handle complex temporal structures.

The paper's summary: Tom: Now, let’s look at the RhyMix summary again, focusing on what they actually achieved in terms of results and why it matters for long term forecasting.

Jane: Basically, they’re saying that real world time series have these complex dynamics—short term fluctuations, seasonal cycles, long term trends—and existing architectures struggle because they rely on single path modeling that either smooths local variations or has limited receptive fields.

Lu: RhyMix directly addresses this by introducing a hybrid architecture designed around a parallel dual path modeling paradigm with adaptive gating mechanisms to capture diverse temporal patterns.

Meng: So the summary is that it integrates two complementary encoding branches: one cyclic path for explicit seasonal inductive bias and another lightweight MSTCN-CA path for contextual temporal representation learning.

Tom: Right. And they also have four specialized forecasting heads—linear, trend-seasonal decomposition, local fluctuations, and periodic cycles—applied to both paths.

Jane: So the model isn't just making one prediction; it’s trying to capture different aspects of the signal at the same time using these different heads.

Lu: The path weights combine these heads, and a path gate averages the gate features across channels to produce a single set of head weights shared across all channels.

Meng: So this fusion step is crucial because it’s where they synthesize what both paths learned into something useful for the final forecast.

Tom: And finally, the hybrid gate adaptively balances those two paths based on input characteristics, producing the final forecast Y hat equals h0 times Y hat cyclic plus h1 times Y hat tcn in R×H×C.

Jane: That means the final prediction is a weighted combination of the two specialized predictions, where that weighting is learned dynamically based on what’s happening in that specific sample.

Lu: This allows for a high capacity design through its design, satisfying criteria like asymmetry, lagginess, polarity, group wise structure and dynamism with adaptive gates lambda k w h.

Meng: The authors are very proud of showing that this architecture satisfies those six key architectural criteria—it shows it’s not just some random stacking.

Tom: And they put the performance down on ten out of twelve datasets across diverse long term forecasting benchmarks, achieving state-of-the art performance on these benchmarks <ref:2607.08234#pg2,of 12 datasets across diverse long term forecasting benchmarks>.

Jane: So, in short, RhyMix offers a very compact and fast way to get high quality long term forecasts without needing massive parameter counts.

The paper's improvements: Tom: Let’s talk about the specific improvements they suggest for the RhyMix architecture that make it stand out from what came before.

Jane: They are highlighting the explicit multi period cyclic prior integration through the Cyclic Path, which incorporates learnable cyclic embeddings with periods like twelve twenty-four forty-eight and one hundred sixty-eight time steps corresponding to half day daily two day and weekly cycles.

Lu: That gives an explicit seasonal inductive bias in a very structured way. It’s a structured way to model predictable rhythms without having the model have to learn those cycles purely from scratch.

Meng: So it’s like they are giving the AI a head start on what we already know about time series structure.

Tom: Exactly. And then they have this Multi Scale Temporal Convolution with Channel Attention MSTCN-CA path which uses four parallel depthwise dilated convolutions with dilation rates d in one two four and eight to capture short term fluctuations at d=one medium term trends at d=two and long term dependencies at d=eight <ref:2607.08234#pg1,with Channel Attention MSTCN-CA>.

Jane: That’s a clear mapping of dilation rates to different temporal scales. It’s not just a random set of convolutions; it’s tailored for specific temporal dependencies.

Lu: This combination, alongside the adaptive gating mechanisms, is what allows RhyMix to dynamically fuse the outputs from both pathways based on input characteristics.

Meng: So the dynamic balancing between the two paths is really key to making sure that when one path isn't working as well as another, it can compensate for it.

Tom: The improvements they propose include using gate feature extraction which computes statistical descriptors like mean, standard deviation, last value, slope between the first and last step of a sequence. These compact features serve as the basis for all subsequent adaptive gating mechanisms.

Jane: So they are turning simple stats into powerful control signals that drive the model’s internal decision making based on those inputs.

Lu: The paper also points out that their overall design satisfies criteria like asymmetry, lagginess, polarity, group wise structure and dynamism with adaptive gates lambda k w h.

Meng: I see how they are systematically checking every architectural requirement to show it’s not just some random stacking of components; it’s a carefully engineered system.

Tom: And they also provide that the total asymptotic complexity is dominated by the four forecasting heads, each applying a linear transformation from L to H, resulting in O(L·C · (K + D + r + 4H)), which scales linearly with both L and C <ref:2607.08234#pg1>.

Jane: That linear scaling with respect to sequence length and channels makes it very attractive for practical use because it means the computational cost grows predictably.

Lu: The ultimate goal is to achieve SOTA performance on ten of twelve datasets across diverse long term forecasting benchmarks while maintaining this efficiency <ref:2607.08234#pg2,performance on 10 of 12 datasets across diverse long term forecasting benchmarks>.

Conclusion: Tom: So, to wrap up RhyMix, we’ve seen how this lightweight adaptive multi rhythm network integrates explicit cyclic priors with data driven multi scale convolutions to get high quality forecasts.

Jane: It boils down to a dual path design where the Cyclic Path handles the predictable rhythms and the MSTCN-CA path handles the context.

Lu: The whole system is controlled by adaptive gating mechanisms that dynamically fuse four specialized forecasting heads based on input characteristics, which gives it a really flexible structure.

Meng: For me, what I see is a model that’s very efficient because of its linear complexity and low latency inference, which is something I can actually deploy without too much worry about hardware constraints.

Tom: It’s definitely compact at 40K parameters compared to models like Time o1 with 290K parameters, so it offers a significant reduction in size for similar performance.

Jane: It’s a solid piece of research because it shows that you can achieve state of the art forecasting results with substantially fewer parameters and still maintain real world efficiency.

Lu: The work introduces RhyMix as a lightweight adaptive multi rhythm network, which is fully self-contained with linear complexity, making it suitable for real time edge deployment.

Meng: It’s definitely a promising direction for future work because the modular architecture allows practitioners to adapt it by adjusting cyclic periods for different frequencies or incorporating domain specific preprocessing.

Tom: I agree. It’s a model that does what it does well, and its efficiency is hard to beat, so if you want SOTA performance on time series forecasting this RhyMix paper is worth checking out.

Sumit Satishrao Shevtekar, Chandresh Kumar Maurya

Department of Computer Science and Engineering, Indian Institute of Technology Indore

cs.LG, cs.AI

Submitted: 2026-07-09

Updated: 2026-10-13

Comments: 28 Pages

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist: RhyMix introduces a lightweight hybrid architecture combining explicit multi-period cyclic priors with data-driven multi-scale convolutions to achieve state-of-the-art long-term time series

Key concepts

Cyclic Path: This path incorporates explicit seasonal knowledge by using learnable cyclic embeddings. It includes four distinct embedding tables corresponding to different periods (half-day, daily, two-day, and weekly cycles), providing the model with a built-in understanding of recurring temporal patterns.
MSTCN-CA Path: This path uses a lightweight Multi-Scale Temporal Convolutional Network with Channel Attention. It employs stacked depthwise dilated convolutions with dilation rates of 1, 2, 4, and 8 to effectively capture temporal dependencies across various time scales, from short-term fluctuations to long-term trends.
Adaptive Gating: RhyMix utilizes two levels of adaptive gating: a per-channel path gate and a per-sample hybrid gate. These mechanisms dynamically modulate the contributions of the cyclic and MSTCN-CA paths based on the input data, allowing the model to adapt its forecasting strategy to specific temporal characteristics.
Linear Complexity: The architecture is designed to maintain linear complexity, meaning its computational cost scales linearly with both sequence length (L) and number of channels (C). This ensures that the model remains computationally efficient for real-time deployment on resource-constrained hardware.

Terminology

Benchmark — 1× in this paper · explained in 2 episodes
Edge Devices — 1× in this paper · explained in 2 episodes
Embeddings — 1× in this paper · explained in 2 episodes
Time Series Forecasting — 1× in this paper · explained in 3 episodes
Transformer — 1× in this paper · explained in 2 episodes

Summary

The gist: RhyMix introduces a lightweight hybrid architecture combining explicit multi-period cyclic priors with data-driven multi-scale convolutions to achieve state-of-the-art long-term time series forecasting performance while maintaining linear complexity and low latency.

How it works

RhyMix is designed around a parallel dual-path modeling paradigm with adaptive gating mechanisms to capture diverse temporal patterns, as described on Page 3. The architecture integrates two complementary encoding branches: (i) a Cyclic Path that incorporates explicit seasonal inductive bias through learnable cyclic embeddings, and (ii) a lightweight Multi-Scale Temporal Convolutional Network with Channel Attention Path that employs multi-scale depthwise dilated convolutions to capture temporal dependencies across different receptive fields and. A key innovation is the use of adaptive gating mechanisms at multiple levels, which dynamically modulates the contributions of different components based on input characteristics, ensuring the model adapts its behavior to the specific temporal patterns present in each sample while maintaining computational efficiency through linear complexity O(L·C) with respect to sequence length and channels.

Key Architectural Components

The RhyMix architecture comprises six principal modules:

RevIN normalization is applied to the input, normalizing each channel independently using statistics computed from the look-back window, enabling the model to focus on relative temporal patterns rather than absolute values.
Gate feature extraction computes a set of statistical descriptors from the normalized input, including mean, standard deviation, last value, slope (last minus first), and mean absolute change between consecutive steps. These compact features serve as the basis for all subsequent adaptive gating mechanisms.
A channel mixer enables cross-channel interaction while maintaining computational efficiency through a lightweight bottleneck MLP with a residual connection, where the bottleneck dimension adapts to the number of input channels.
The Cyclic Forecasting Path incorporates explicit seasonal inductive bias through four learnable cyclic embedding tables, with periods Pk in 12, 24, 48, and 168 time steps corresponding to half-day, daily, two-day, and weekly cycles.
The MSTCN-CA Path captures multi-scale temporal dependencies using two stacked Temporal Multi-Scale (TMS) blocks with four parallel depthwise dilated convolutions with dilation rates d in 1, 2, 4, and 8 to capture short-term fluctuations (d=1), medium-term trends (d=2,4), and long-term dependencies (d=8).
The multi-scale fusion module integrates outputs from both pathways through two levels of adaptive combination: a path gate learns to combine four specialized forecasting heads for each sample and channel, and a hybrid gate adaptively balances the contributions of the Cyclic and MSTCN-CA Paths based on input characteristics.

Forecasting Heads and Fusion

The model employs four specialized forecasting heads to capture the dominant temporal patterns in time series—linear, trend-seasonal, local fluctuations, and periodic cycles—avoiding the limitations of a single head. Both paths apply these four heads to their respective representations. The path weights combine these heads, and the path gate averages the gate features across channels to produce a single set of head weights shared across all channels. The final forecast is determined by a hybrid gate that balances the contributions of the Cyclic and MSTCN-CA Paths, producing Yˆ = h0 ⊙ Yˆ cyclic + h1 ⊙ Yˆ tcn in R×H×C.

Performance and Efficiency

RhyMix achieves state-of-the-art performance on 10 of 12 benchmark datasets across diverse long-term forecasting benchmarks. The model remains lightweight (∼40K parameters) with linear complexity and low inference latency (<5 milliseconds), making it suitable for resource-constrained edge devices and real-time deployment. It achieves better performance compared to SOTA baselines including Time-o1, SEG-MOE, GCMNet, MSTN, SOFTS, TexFilter, iTransformer, and PatchTST while using 10× fewer parameters than Time-o1 and requiring no pre-computation.

Comparative Analysis

RhyMix is positioned as a hybrid design (CI+CD) via parallel Cyclic and MSTCN-CA Paths with adaptive gating to capture multi-scale dynamics, demonstrating high capacity through its design. Table 2 summarizes how RhyMix combines desirable characteristics from different design approaches through its hybrid architecture, achieving high capacity with linear complexity O(L · C); depthwise convolutions; bottleneck channel mixer. Specifically, RhyMix satisfies six key architectural criteria—Asymmetry (L vs H embeddings), Lagginess (dilated convs d = 1, 2, 4, 8), Polarity (trend-seasonal decomposition), Group-wise (channel mixer bottleneck C → r → C), Dynamism (adaptive gates λk, w, h), and Multi-scale (dilations + periods). The total asymptotic complexity is dominated by the four forecasting heads, each applying a linear transformation from L to H, resulting in O(L·C ·(K +D+r+4H)), which scales linearly with both L and C.

Conclusion

The work introduces RhyMix, a lightweight hybrid architecture combining explicit cyclic priors with data-driven multi-scale convolutions, which employs a dual-path design—a Cyclic Path for seasonal patterns and an MSTCN-CA Path for multi-scale dynamics—with adaptive gating that dynamically fuses four specialized forecasting heads per sample. This design captures diverse temporal patterns while maintaining linear complexity and delivering SOTA performance. RhyMix achieves this SOTA performance with substantially fewer parameters (40K) than comparable SOTA methods such as Time-o1 (290K) and TimeMixer++ (647K), offering three key advantages over Time-o1 and TimeMixer++. RhyMix is fully self-contained with linear complexity, making it suitable for real-time edge deployment. Future work will explore extending RhyMix to other time series tasks such as imputation and anomaly detection, as well as investigating adaptive period selection mechanisms. The modular architecture allows practitioners to adapt it—for instance, by adjusting cyclic periods for different frequencies or incorporating domain-specific preprocessing—which are promising directions for future work.

Data and Code Availability

All datasets used in this study are publicly available from their respective sources as cited throughout the paper. The source code for RhyMix will be released publicly upon acceptance of this manuscript.

Declaration of Competing Interest

The authors declare no conflict of interest.

References

Bai, S., Kolter, J.Z., Koltun, V., 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR abs/1803.01271.

Chang, C., Wang, W.Y., Peng, W.C., Chen, T.F., 2025. Llm4ts: Aligning pre-trained llms as data-efficient time-series forecasters.

Chen, S.A., Li, C.L., Arik, S.O., Yoder, N.C., Pfister, T., 2023. TSMixer: An all-MLP architecture for time series forecasting.

donghao, L., wang xue, 2024. ModernTCN: A modern pure convolution structure for general time series analysis, in: The Twelfth International Conference on Learning Representations.

Ekambaram, V., Jati, A., Dayama, P., Mukherjee, S., Nguyen, N.H., Gifford, W.M., Reddy, C., Kalagnanam, J., 2024. Tiny time mixers (ttms): Fast pre-trained models for enhanced zero/few-shot forecasting of multivariate time series.

Franceschi, J.Y., Dieuleveut, A., Jaggi, M., 2019. Unsupervised scalable representation learning for multivariate time series, in: Advances in Neural Information Processing Systems (NeurIPS), pp. 4652–4663.

Fristiana, A.H., Alfarozi, S.A.I., Permanasari, A.E., Pratama, M., Wibirama, S., 2024. A survey on hyperparameters optimization of deep learning for time series classification.

Gruca, A., Serva, F., Lliso, L., Rípodas, P., Calbet, X., Herruzo, P., Pihrt, J., Raevskyi, R., Šimánek, P., Choma, M., et al. 2022. Weather4cast at neurips 2022: Super-resolution rain movie prediction under spatio-temporal shifts, in: NeurIPS 2022 Competition Track.

Han, L., Chen, X.Y.

Improvements for AI systems

textbfExplicit Multi-Period Cyclic Prior Integration (Cyclic Path): The system can explicitly model multiple seasonal cycles by incorporating learnable cyclic embeddings with periods like 12, 24, 48, and 168 time steps, providing an explicit seasonal inductive bias.
textbfAdaptive Feature Fusion via Four Specialized Heads: The architecture can dynamically combine four specialized forecasting heads—Direct, Trend-Seasonal Decomposition, Local Convolution, and Periodic Fusion—using a path gate that learns weights based on input characteristics to balance the contributions of the two main paths (Cyclic Path and MSTCN-CA Path).
textbfMulti-Scale Temporal Dependency Capture (MSTCN-CA): The system can capture temporal dependencies across different scales by employing a lightweight Multi-Scale Temporal Convolutional Network (TCN) utilizing four parallel depthwise dilated convolutions with dilation rates d ∈ 1, 2, 4, and 8 to model short-, medium-, and long-term patterns.
textbfend-to-End Resource Efficiency: The resulting AI system can maintain linear complexity with respect to sequence length, channels, and prediction horizon, offering a compact model size (157 KB) and low inference latency ("<5 milliseconds), making it suitable for resource-constrained edge devices."
textbfDynamic Adaptability (Adaptive Gating): The hybrid gate learns to adapt its behavior to the specific temporal patterns present in each sample by dynamically balancing the contributions of the two encoding branches based on input characteristics, ensuring that features are weighted optimally for every individual time series.

Abstract

Real-world time series exhibit complex dynamics characterized by multiple simultaneous temporal patterns: short-term fluctuations, periodic seasonal cycles, long-term trends, and irregular abrupt changes. However, most forecasting architectures rely on single-path temporal modeling---transformers capture long-range dependencies but smooth local variations, convolutions capture local patterns but have limited receptive fields, and linear models are efficient but cannot capture nonlinear dynamics---so no single architectural paradigm captures all these pattern types simultaneously. To address this, we introduce RhyMix (RHYthm MIXture), a hybrid neural architecture designed around a parallel dual-path modeling paradigm with adaptive gating mechanisms. RhyMix integrates two complementary encoding branches: (i) a Cyclic Path that incorporates explicit seasonal inductive bias through learnable cyclic embeddings; and (ii) a lightweight Multi-Scale Temporal Convolutional Network with Channel Attention (MSTCN-CA) Path that captures temporal dependencies across different receptive fields. Unlike prior hybrid architectures that combine components with fixed weights, RhyMix computes its fusion weights from the input, adapting to each sample. Its key methodological contribution is a two-level adaptive gating framework: a per-channel path gate that fuses four specialized forecasting heads, and a per-sample hybrid gate that balances the two paths. This design maintains linear complexity. Across 12 real-world benchmarks, RhyMix achieves the best average MSE on 9 datasets and the best average MAE on 8 datasets. The model remains lightweight (40K parameters for C=7) with low-latency inference (5.2--7.5 ms), making it suitable for resource-constrained edge devices and real-time deployment.