Fine-Tuning an Order-Flow Foundation Model for Sub-Second Forecasting

arXiv:2611.14117 · q-fin.TR · Submitted 2026-11-10 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.

Ruth: We're looking at the paper: "Fine-Tuning an Order-Flow Foundation Model for Sub-Second Forecasting".

Warren: Here's the gist of the paper. Fine-tuned generative models pretrained on Level-3 order-flow data can be fine-tuned to predict future microprice and volatility, outperforming classical baselines in sub-second forecasting and demonstrating zero-shot asset transfer capabilities Model Pretraining and Tokenization The approach involves pretraining a 300M-parameter generative model on Level-3 order-flow data collected from two crypto exchanges, Bitstamp and Bitfinex The model is trained to predict the next token with a cross-entropy loss on 2.8 billion training tokens Tokenization converts one order into one token, such as "ten buy limit orders will be converted into ten tokens in the sequence" The tokenizer uses fields like action, side, δp, log v, and ∆t at radices (2, 2, 16, 16, 16) with a vocabulary of 8192 elements Crypto required three departures from the equities recipe due to bursty activity and trade modifications Fine-Tuning Methodology The pretrained model is fine-tuned via LoRA to predict the first and second moments of the future microprice on BTC/USD across horizons from 200 ms to 30 seconds The fine-tuning uses a decoupled loss Huber(µ, r) + Huber(log σ, log rv) instead of a naive NLL because the latter causes the mean to collapse The model is compared against classical benchmarks such as gradient boosting and GARCH for predicting microprice change direction and volatility Performance on Sub-Second Horizons The fine-tuned model outperforms classical baselines in predicting the microprice change direction at the 200 ms and 500 ms horizons Furthermore, it outperforms GARCH on volatility in four of five evaluation windows The model beats order-book imbalance (OBI), including a 10-level book and OBI combined with recent-return momentum, at every sub-second horizon The margin against the gradient-boosted baseline is steady, reaching +0.05–+0.08 in correlation across sub-second horizons Zero-Shot Transfer and Generalization The predictive ability of the model transfers zero-shot to assets it was never fine-tuned on, specifically beating basic baselines for SOL and ETH The backbone is pretrained on 73 asset streams across both venues, so the representation has seen their order flow, but the LoRA and head are fine-tuned on BTC alone The fine-tuned mapping is thus asset-general, with liquidity setting how short the horizon can be pushed Inference Latency and Practical Use The end-to-end inference latency for a streaming system is 4.0 ms on the H100, which is 2% of the shortest horizon forecasted and sustains 247 events/s against the median rate of this instrument The signal is a real, leakage-free, asset-general prediction, but not directional alpha The plausible uses are in execution (anticipating imminent adverse selection) rather than taking The fine-tuned model beats gradient boosting on order-flow features, and the edge is a short-horizon forecast that forecasts the imminent order-flow continuation rather than a persistent multi-horizon signal The barrier to tradeability is one of move magnitude, not forecast accuracy Limitations include fine-tuning on a single asset and a single two-day window, which limits regime coverage The volatility comparison shows the second-moment claim is regime-conditional The model’s edge forecasts the imminent order-flow continuation rather than a persistent multi-horizon signal Future Work Future work suggests pretraining on more data, per-asset fine-tunes, and larger models might improve accuracy The state of the order book is potentially useful information, hence encoding it and feeding it to the model might improve the accuracy A longer context window might lead to higher accuracy for pairs with a lot of transactions Modifications to the architecture like linear attention or sliding-window attention might be helpful once a longer context window is achieved References Fernando Acero What do foundation models for market microstructure learn?

Ruth: A controlled study with synthetic limit order books.

Warren: arXiv preprint arXiv:2610.20104, 2026 Jakob Albers Mihai Cucuringu Sam Howison, and Alexander Y Shestopaloff The good, the bad, and latency: exploratory trading on Bybit and Binance.

Ruth: Quantitative Finance, 25(6):919–947, 2025 Abdul Fatir Ansari et al.

Paper summary: Ruth: So looking at the whole thing, the paper "Fine-Tuning an Order-Flow Foundation Model for Sub-Second Forecasting" is essentially about taking a large generative model and tailoring it to predict very short-term price and volatility changes using order flow data.

Warren: The authors claim their fine-tuned model beats classical methods on prediction direction at two hundred millisecond and five hundred millisecond horizons, and it also shows zero-shot transfer to new assets, which is a pretty big part of the contribution <ref:2611.14117#pg2>.

Iris: It seems like the main point here is that they've managed to fine-tune this foundation model for predicting both the first and second moments of microprice using a specific loss function that handles uncertainty well.

Ruth: The implication is that instead of just looking at handcrafted features, we can use these large models to capture order flow dynamics in a way that outperforms things like gradient boosting on those features.

Warren: And while it's useful for sub-second execution, the authors are honest about the limitations, noting that they only fine-tuned on one asset and two days of data, which means the regime coverage is pretty limited.

Iris: And they also clarify that the claim about beating volatility is actually conditional on specific market regimes because it's not universally true across all conditions.

Ruth: So in simple terms, this paper shows how a generative foundation model can be fine-tuned to predict very short-term price moves based on order flow, offering a tool for execution and making things more efficient in the market.

Conclusion: Ruth: So we’ve seen how they took this huge generative model and fine-tuned it specifically for order flow predictions on BTC, looking at how they framed that in "Fine-Tuning an Order-Flow Foundation Model for Sub-Second Forecasting."

Warren: Yeah, I think the authors are saying that this isn't just another fancy trading tool; it’s about getting a model to actually learn the structure of the order book itself.

Iris: What they're doing is taking data from those two exchanges and using it to train a model that predicts both the next price and how much volatility there will be over very short periods, like two hundred milliseconds or five hundred milliseconds.

Ruth: Exactly, and what’s interesting is that they use this specific way of training—decoupling the loss function—to make sure the model doesn't just guess a number, but actually captures the uncertainty in those price changes correctly.

Warren: And on the results side, they show it beats classical stuff like gradient boosting and GARCH for predicting whether the price is going up or down at those sub-second levels.

Iris: They also managed to get this model to work on assets it hadn't seen before—like SOL and ETH—even though they only fine-tuned the part dealing with Bitcoin. That’s a pretty strong generalization claim there.

Ruth: It suggests that the way this model learns order flow patterns is actually kind of general, not just specific to one coin or one set of trading conditions.

Warren: But we gotta remember the caveats they laid out in their conclusion—the edge they find is really tied to that very short timeframe, which means it’s forecasting what's about to happen right now, not a long-term trend.

Iris: And they also flagged that the claim about beating volatility isn't solid across all market situations; it depends on the specific regime you are in.

Ruth: So the real implication here is that this AI approach can provide a very fast, real-time signal for execution, but we have to be careful about overinterpreting it as a persistent alpha generator.

Warren: It’s definitely a tool for anticipating immediate adverse selection during trades, but the barrier to actually using it is how much movement you are expecting.

Iris: If we look at what they suggest next—pretraining on more data and maybe incorporating the actual state of the order book—that could definitely push those accuracy numbers further.

Fedor Shabashev

q-fin.TR

Submitted: 2026-11-10

Updated: 2026-11-10

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 76/100

The gist: Fine-tuned generative models pretrained on Level-3 order-flow data can be fine-tuned to predict future microprice and volatility…

Key concepts

Level-3 Order Flow Data
This is the raw data used to train the model. It records every single order in the limit order book, including details like action (buy/sell), side, price difference ($\delta p$), log volume ($\log v$), and time difference ($\Delta t$). This detailed data captures the microstructure of how trades are executed on exchanges.
Tokenization
The process of converting complex order book events into a sequence of tokens that a generative model can understand. Each individual order is converted into one token, allowing the model to learn patterns across sequences of trades rather than just single points in time.
LoRA Fine-Tuning
A technique used to efficiently adapt a large, pre-trained generative model to a specific task. Instead of retraining the entire massive model, LoRA introduces small, trainable matrices that are added to the existing model layers. This allows for rapid and effective fine-tuning on specific prediction tasks like forecasting microprice moments.
Zero-Shot Asset Transfer
The ability of the fine-tuned model to successfully predict outcomes for assets it was never explicitly trained on during its final tuning phase. This demonstrates that the learned underlying representation of order flow is general enough to apply effectively across different cryptocurrencies like BTC, SOL, and ETH.

Terminology

Summary

Fine-tuned generative models pretrained on Level-3 order-flow data can be fine-tuned to predict future microprice and volatility, outperforming classical baselines in sub-second forecasting and demonstrating zero-shot asset transfer capabilities

Model Pretraining and Tokenization

The approach involves pretraining a 300M-parameter generative model on Level-3 order-flow data collected from two crypto exchanges, Bitstamp and Bitfinex The model is trained to predict the next token with a cross-entropy loss on 2.8 billion training tokens Tokenization converts one order into one token, such as ten buy limit orders will be converted into ten tokens in the sequence The tokenizer uses fields like action, side, δp, log v, and ∆t at radices (2, 2, 16, 16, 16) with a vocabulary of 8192 elements Crypto required three departures from the equities recipe due to bursty activity and trade modifications

Fine-Tuning Methodology

The pretrained model is fine-tuned via LoRA to predict the first and second moments of the future microprice on BTC/USD across horizons from 200 ms to 30 seconds The fine-tuning uses a decoupled loss Huber(µ, r) + Huber(log σ, log rv) instead of a naive NLL because the latter causes the mean to collapse The model is compared against classical benchmarks such as gradient boosting and GARCH for predicting microprice change direction and volatility

Performance on Sub-Second Horizons

The fine-tuned model outperforms classical baselines in predicting the microprice change direction at the 200 ms and 500 ms horizons Furthermore, it outperforms GARCH on volatility in four of five evaluation windows The model beats order-book imbalance (OBI), including a 10-level book and OBI combined with recent-return momentum, at every sub-second horizon The margin against the gradient-boosted baseline is steady, reaching +0.05–+0.08 in correlation across sub-second horizons

Zero-Shot Transfer and Generalization

The predictive ability of the model transfers zero-shot to assets it was never fine-tuned on, specifically beating basic baselines for SOL and ETH The backbone is pretrained on 73 asset streams across both venues, so the representation has seen their order flow, but the LoRA and head are fine-tuned on BTC alone The fine-tuned mapping is thus asset-general, with liquidity setting how short the horizon can be pushed

Inference Latency and Practical Use

The end-to-end inference latency for a streaming system is 4.0 ms on the H100, which is 2% of the shortest horizon forecasted and sustains 247 events/s against the median rate of this instrument The signal is a real, leakage-free, asset-general prediction, but not directional alpha The plausible uses are in execution (anticipating imminent adverse selection) rather than taking

Conclusion and Limitations

The fine-tuned model beats gradient boosting on order-flow features, and the edge is a short-horizon forecast that forecasts the imminent order-flow continuation rather than a persistent multi-horizon signal The barrier to tradeability is one of move magnitude, not forecast accuracy Limitations include fine-tuning on a single asset and a single two-day window, which limits regime coverage The volatility comparison shows the second-moment claim is regime-conditional The model’s edge forecasts the imminent order-flow continuation rather than a persistent multi-horizon signal

Future Work

Future work suggests pretraining on more data, per-asset fine-tunes, and larger models might improve accuracy The state of the order book is potentially useful information, hence encoding it and feeding it to the model might improve the accuracy A longer context window might lead to higher accuracy for pairs with a lot of transactions Modifications to the architecture like linear attention or sliding-window attention might be helpful once a longer context window is achieved

References

Fernando Acero What do foundation models for market microstructure learn? A controlled study with synthetic limit order books. p. 2 arXiv preprint arXiv:2610. p. 020104, 2026 Jakob Albers Mihai Cucuringu Sam Howison, and Alexander Y Shestopaloff The good, the bad, and latency: exploratory trading on Bybit and Binance. Quantitative Finance, 25(6):919–947, 2025 Abdul Fatir Ansari et al. p. 2 Chronos: Learning the language of time series. p. 2 Transactions on Machine Learning Research, 2024 Emmanuel Bacry Iacopo Mastromatteo, and Jean-Fran¸cois Muzy Hawkes processes in finance. p. 2 Market Microstructure and Liquidity, 1(1), 2015 Jonathan Baxter A model of inductive bias learning. Journal of Artificial Intelligence Research, 12:149–198, 2000 Tim Bollerslev Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31(3):307–327, 1986 Rich Caruana Multitask learning. p. 2 Machine Learning, 28(1):41–75, 1997 Rama Cont Empirical properties of asset returns: stylized facts and statistical issues. p. 2 Quantitative Finance, 1(2):223–236, 2001 Fulvio Corsi A simple approximate long-memory model of realized volatility. p. 2 Journal of Financial Econometrics, 7(2):174–196, 2009 Abhimanyu Das Weihao Kong Rajat Sen, and Yichen Zhou A decoder-only foundation model for time-series forecasting. In ICML, 2024 Francis X Diebold and Roberto S Mariano Comparing predictive accuracy. p. 7 Journal of Business & Economic Statistics, 13(3):253–263, 1995 Sean Foley William Krekel Vito Mollica, and Jiri Svec Not so fast: Identifying and remediating slow and imprecise cryptocurrency exchange data. Finance Research Letters, 51:103401, 2023 Edward J Hu et al LoRA: Low-rank adaptation of large language models. In ICLR, 2022 Rakshit Jha Mattijs De Paepe Samuel Holt James West, and Shaun Ng Deep learning for digital asset limit order books. arXiv preprint arXiv:2010. p. 001241, 2020 Maxime Kawawa-Beaudan Srijan Sood Kassiani Papasotiriou Daniel Borrajo, and Manuela Veloso TradeFM: A generative foundation model for trade-flow and market microstructure. p. 1 arXiv preprint arXiv:2602. p. 023784, 2026 Philipp Koehn Statistical significance tests for machine translation evaluation. p. 7 In EMNLP, pages 388–395, 2004 Petter N Kolm Jeremy Turiel, and Nicholas Westray Deep order flow imbalance: Extracting alpha at multiple horizons from the limit order book. p. 1 Mathematical Finance, 33(4):1044–1081, 2023 Junjie Li Yang Liu Weiqing Liu Shikai Fang Lewen Wang Chang Xu, and Jiang Bian MarS: A financial market simulation engine powered by generative foundation model. p. 1 In ICLR, 2025 Eljas Linna Kestutis Baltakys Alexandros Iosifidis, and Juho Kanniainen LOBERT: Generative AI foundation model for limit order book messages. p. 2 arXiv preprint arXiv:2511. p. 012563, 2025 Peer Nagy Sascha Frey Silvia Sapora Kang Li, Anisoara Calinescu, Stefan Zohren, and Jakob Foerster Generative AI for end-to-end limit order book modelling: A token-level autoregressive generative model of message flow using a deep state space network. p. 1 In ICAIF, 2023 Maximilian Seitzer Arash Tavakoli Dimitrije Antic, and Georg Martius On the pitfalls of heteroscedastic uncertainty estimation with probabilistic neural networks. p. 7 In ICLR, 2022 Zihao Zhang Stefan Zohren, and Stephen Roberts DeepLOB: Deep convolutional neural networks for limit order books. p. 1

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Pretrain a 300M-parameter generative model on Level-3 order-flow data collected from two crypto exchanges to create a foundation model capable of learning useful representations for price prediction. This foundation model can then be fine-tuned via LoRA to predict the first and second moments of future microprice and its realized volatility.

  2. Fine-tune the pretrained generative model to predict the microprice return and its realized volatility across multiple horizons, including 200 ms, 500 ms, 1, 2, 5, 10 seconds, demonstrating that our fine-tuned model outperforms all classical benchmarks in predicting the microprice change direction at the sub-second horizons.

  3. Enable zero-shot asset transfer by applying a model fine-tuned on BTC to predict future microprice to new assets like SOL and ETH, showing that a generative model fine-tuned to predict the future microprice for BTC outperforms basic baselines when it comes to predicting the future microprice for SOL and ETH.

  4. Improve directional prediction accuracy by using the foundation model as a baseline against classical methods like order-book imbalance (OBI), gradient boosting on handcrafted features, and GARCH models, achieving gains such as the fine-tuned model beats gradient boosting on order-flow features.

  5. Enhance volatility forecasting by using the model to predict realized volatility, showing that the fine-tuned model outperforms GARCH on volatility in four of five evaluation windows at sub-second horizons.

Sources

Related papers