A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments
summary
The gist
Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting This research, detailed in arXiv:2603.16886v1 (q-fin.ST), presents a rigorous…
In short
This research compared nine deep learning architectures across four model families for multi-horizon financial forecasting using five market features. ModernTCN and PatchTST emerged as top performers, showing stable superiority across short and long horizons. The study proves that specific architectural design (inductive bias) is more important than raw model size for accuracy.
Key concepts
- ModernTCN
- A specialized deep learning architecture that proved to be the best performer in the study. It excels at capturing temporal patterns in financial data, consistently ranking at the top across different testing scenarios, especially for long-term predictions.
- PatchTST
- Another highly effective Transformer variant that ranked second overall. It demonstrates strong performance and maintains its leading position when comparing short-term forecasts against longer-range predictions, indicating robustness in time series modeling.
- Inductive Bias
- The inherent assumptions or structural preferences built into a deep learning model's design. The study found that the specific architectural choices (like those in ModernTCN) are more critical for forecasting accuracy than simply increasing the total number of parameters in the model.
- Non-monotonic Complexity
- The finding that increasing a model's size (parameter count) does not always lead to better performance. Instead, there is an optimal point; models with specific structural biases outperform much larger models, suggesting architectural quality trumps sheer capacity.
Terminology used across episodes
This episode discusses
- A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments · Paper Radio
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
The paper
A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting: Evidence from 918 Experiments · Read on arXiv
Dr. A.P.J. Abdul Kalam Technical University (AKTU)
Reliable comparison of deep learning architectures for financial forecasting requires controlled evaluation that separates architectural effects from hyperparameter tuning, random initialization, forecast horizon, and asset characteristics. We present a five-stage benchmark using equal-budget Bayesian hyperparameter optimization, category-level configuration freezing, three-seed final training, held-out metric aggregation, and rank-based statistical validation. Nine architectures from Transformer, MLP/linear, convolutional, and recurrent families are evaluated with direct multi-step prediction from five OHLCV features across 12 assets (four cryptocurrency, four foreign exchange, four equity-index instruments), using chronological 70/15/15 splits of 30,000 hourly windows per asset at 4- and 24-hour horizons. The benchmark includes 918 runs (270 optimization trials, 648 final training runs). Across 24 asset-horizon points, ModernTCN has mean rank 1.333 and the lowest RMSE in 18 points (75.0%), followed by PatchTST (mean rank 2.000). Global ranking differences are strongly supported by an Iman-Davenport test (F(8,184)=101.36, p<1e-15), with 33 of 36 Holm-adjusted pairwise comparisons significant. Rankings are consistent across horizons for all 12 assets (rho>=0.683; Stouffer p=3.47e-10), while errors increase at the longer horizon. Architecture explains 99.90% of raw error variance and 68.33% after within-slot normalization excluding LSTM; seed variance contributes at most 0.04% (0.02% without LSTM). Mean directional accuracy is 50.08%. These findings provide a statistically validated comparison of nine architectures, identify ModernTCN and PatchTST as leading performers under the evaluated conditions, and offer a reproducible benchmark for multi-asset financial time-series forecasting.
Transcript
Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.
Ruth: We're looking at the paper: "A Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting".
Warren: Here's the gist of the paper. Comprehensive Research Summary: Controlled Comparison of Deep Learning Architectures for Multi-Horizon Financial Forecasting This research, detailed in arXiv:2603.16886v1 (q-fin.ST), presents a rigorous, protocol-controlled comparison of nine distinct deep learning architectures—Autoformer, DLinear, iTransformer, ModernTCN, N-HiTS, PatchTST, TimesNet, and TimeXer—across four fundamental model families (Transformer variants and MLP/Linear/CNN/RNN) applied to financial time-series forecasting.
Ruth: The study was designed to provide a robust evaluation framework for selecting the optimal architecture for multi-horizon tasks in portfolio allocation, risk management, and algorithmic trading.
Warren: Experimental Design and Methodology The core of the study lies in its highly structured experimental design, involving 918 independent runs.
Ruth: These runs adhere strictly to a five-stage protocol to ensure a fair comparison: 1.
Title and authors: Ruth: Now let’s talk about what this paper actually sets out to do, this controlled comparison of deep learning architectures for multi-horizon financial forecasting. It’s basically saying that existing benchmarks are too weak because they lack controlled hyperparameter budgets and single-seed evaluation, which is what they address >
Warren: So, the core idea is moving away from just looking at one number for performance and instead testing if the models actually perform consistently across different time scales and market types >
Iris: They are comparing nine distinct architectures—like Autoformer, DLinear, iTransformer, ModernTCN—across transformer variants and other families to see which one works best when you look at both short-term forecasts, four hours out, and long-term ones like twenty-four hours >
Ruth: And they test this across cryptocurrency, forex indices, and equity markets. They are trying to answer if the architectural choice matters more than just having a big model with lots of parameters >
Warren: The implication for us is that we can’t just pick whatever transformer sounds fanciest; we need a systematic way to pick one based on how it handles different market structures >
Iris: And they are also looking at the relationship between model complexity, like the number of parameters, and the actual forecasting error, which they suggest isn't a simple straight line >
The paper's summary: Ruth: So what does this study actually find when it runs all those nine hundred eighteen experiments? The main takeaway is that certain modern architectures are clearly superior to the older recurrent models we’ve been using for a long time >
Warren: They found that ModernTCN came out on top, achieving the best mean rank of one point three three three across all twenty-four evaluation points, and it had a seventy-five percent first-place rate over those points >
Iris: That’s interesting because they also found a clear three-tier structure to the global performance leaderboard; you have this top pair, then a middle cluster, and then some lower performers >
Ruth: And they confirmed that these top two models maintain their superiority whether you are looking at a short horizon of four hours or extending it out to twenty-four hours >
Warren: That cross-horizon stability is important because it means you don't have to re-tune your entire system just because you change the forecast window >
Iris: They also found that the choice of architecture explains almost all the performance variance, accounting for ninety-nine point nine zero percent of the raw root mean squared error variance, with seed randomness only contributing less than one percent >
The paper's improvements: Ruth: The authors suggest a few ways to improve this kind of comparison work. They point out that large-kernel temporal convolutions and patch-based Transformers are consistently better for this type of financial forecasting >
Warren: So, the practical improvement is leaning into those specific architectural choices rather than just throwing massive models at the problem hoping something works >
Iris: They also highlight that model selection should prioritize architectural inductive bias over raw parameter capacity when trying to minimize forecasting error, which supports what we’ve been seeing about complexity and performance being non-monotonic >
Ruth: That confirms what we suspected: having more parameters doesn't always mean better results if the architecture isn't right for the task >
Warren: And they also suggest that for cryptocurrency assets, specifically lower capitalisation ones like ETH USDT and ADA USDT, you should deploy N-HiTS because it achieved the lowest root mean squared error at a four-hour horizon there >
Iris: That’s a specific recommendation. It suggests tailoring the model to the asset class dynamics rather than using one universal architecture for everything >
Conclusion: Ruth: So, to wrap up this paper, the effort spent on selecting a sophisticated architecture yields significantly better returns than just spending time on initialisation-based ensembles or hyperparameter tuning alone >
Warren: The main recommendation seems to be adopting ModernTCN as your default architecture because it has those general-purpose temporal modeling capabilities across the board >
Iris: I think the most important thing is that the study confirms that three-seed replication is enough for robustness, and since top models maintain their ranking across all three representative assets, we can trust those relative rankings between four hours and twenty-four hours >
Ruth: And remember, directionally speaking, they found that MSE-trained architectures produce directional forecasts equivalent to a fair coin flip at the hourly resolution tested >
Warren: That means if you’re building a trading signal, you need to explicitly redesign your loss function or post-process the point forecasts into something that captures directional skill >
Iris: So, we’ve got some clear direction now: prioritize architectural bias, use ModernTCN as a starting point, and always be careful about what your loss function is actually measuring >
More episodes
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- The day in conversation for 2026-11-18
- Daily Summary for 2026-11-18
- 2601.05274-On the use of case estimate and transactional payment data in neural networks for individual loss reserving
- 2009.00868-Loss-Given-Default Modeling by Post-Last Passage Time Process
- 2606.30070-Financial Resilience Evaluation: From Conditional Expectations to Dynamic Convex Risk Measures
- 2212.12687-Measuring price impact and information content of trades in a time-varying setting
- Daily Summary for 2026-11-17
- 2401.08323-Dynamic portfolio selection under generalized disappointment aversion