Deep Learning of Robust Market Making under Regime-Switching Order Flow
summary
The gist
Deep Learning of Robust Market Making under Regime-Switching Order Flow develops a two-stage fine-tuning pipeline that turns a stationary-optimal RLMM into a controller that remains profitable under…
In short
The research develops a two-stage fine-tuning pipeline to make deep reinforcement learning market makers robust against changing order flow regimes. It starts with a stationary RLMM and refines it using Bayesian change-point filters and scenario bandits. This results in a controller that remains profitable under unpredictable flow shifts and has a tighter loss tail during stress tests.
Key concepts
- Regime-Switching Order Flow
- This refers to situations where the nature of incoming buy or sell orders changes suddenly, indicating different market conditions. Classical models assume order flow is constant, but real markets change regimes without warning, which breaks standard trading strategies.
- Deep Reinforcement Learning Market Maker (RLMM)
- This is an AI agent trained using a specific type of deep learning (distributional DQN) to decide the best quotes for a market maker. It learns optimal quoting rules by interacting with a simulated order book, aiming to maximize profit over time.
- Scenario-Bandit Robust Fine-Tuning
- This is the second stage of the pipeline where the RL agent is trained using 'scenario bandits.' This technique specifically focuses on upweighting low-return or stressful market scenarios during training. This forces the agent to learn how to handle rare, difficult conditions better.
- Bayesian Online Change-Point Filter
- This is a statistical tool used to detect when the underlying pattern of directional order flow has shifted. It continuously monitors incoming data and calculates a probability that a new regime has started, allowing the RL agent to adapt its strategy in real-time.
Terminology used across episodes
This episode discusses
- Deep Learning of Robust Market Making under Regime-Switching Order Flow · Paper Radio
- Deep Reinforcement Learning in Cryptocurrency Market Making
- Reinforcement Learning for Market Making in a Multi-agent Dealer Market
- Bayesian Online Changepoint Detection
The paper
Deep Learning of Robust Market Making under Regime-Switching Order Flow · Read on arXiv
Scuola Normale Superiore di Pisa
Classical market-making strategies based on stochastic control, such as the Avellaneda--Stoikov and the Guéant--Lehalle--Fernandez-Tapia (GLFT) extension, provide closed-form quoting approximations, but rest on assumptions that break down at realistic microstructure timescales. One of them is that order flow is stationary, while empirical evidence points to the existence of regimes, possibly associated with algorithmic execution of metaorders. In our experiments, persistent directional flow makes stationary controllers unprofitable. In this paper, we develop a deep reinforcement-learning market maker (RLMM)---a Rainbow-style distributional deep Q-network which is calibrated and tested in a zero-intelligence limit order book. We find that, in the stationary setting, RLMM outperforms GLFT at comparable levels of profit and loss (PnL) dispersion. The stationarily trained RLMM is more robust to flow asymmetry than GLFT, but still suffers inventory saturation and large drawdowns under persistent directional flow. Training a separate RLMM under regime-switching flow, with a Bayesian flow-belief signal and a queue-adjusted quote exposure signal, restores positive mean PnL in our experiments. A final scenario-bandit step prioritizes low-return scenarios and improves performance under varying regime persistence and directional correlation.
Transcript
Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.
Ruth: We're looking at the paper: "Deep Learning of Robust Market Making under Regime-Switching Order Flow".
Warren: Here's the gist of the paper. Deep Learning of Robust Market Making under Regime-Switching Order Flow develops a two-stage fine-tuning pipeline that turns a stationary-optimal RLMM into a controller that remains profitable under regime-switching flow and has a tighter stressed terminal PnL tail Classical Benchmarks and the Need for Robustness Classical market-making strategies based on stochastic control, such as the Avellaneda–Stoikov and the Guéant–Lehalle–Fernandez-Tapia (GLFT) extension, provide closed-form quoting rules but rest on assumptions that break down at realistic microstructure timescales One of them is that order flow is stationary while empirical evidence points to the existence of regimes possibly associated with algorithmic execution of metaorders Existing methods provide negative PnL in this case The problem is harder still because real order flow is neither stationary nor symmetric and the prevailing regime can shift without warning The Deep Reinforcement Learning Controller A deep reinforcement-learning market maker (RLMM)—a Rainbow-style distributional DQN (C51) which is calibrated and tested in a zero-intelligence limit order book—is developed In the stationary setting, RLMM outperforms GLFT across the entire observed risk–return frontier The RLMM is more robust to flow asymmetry than GLFT but still suffers large drawdowns from inventory saturation under persistent directional imbalance The state of RLMM is augmented with two auxiliary signals—a Bayesian online change-point filter over the directional flow bias and a queue-adjusted quote-exposure imbalance—to restore profitability Regime Adaptation and Fine-Tuning The paper introduces a fine-tuning pipeline to address regime fragility This pipeline involves two main steps: 1.
Ruth: Fine-tuning the stationary controller on regime-switching market-order flow using two belief-style auxiliary signals: a Bayesian online change-point filter for the latent directional order flow bias and a queue-adjusted quote-exposure imbalance for the agent’s own resting quotes 2.
Warren: Introducing a scenario-bandit robust fine-tuning step that upweights low-return regime scenarios to improve performance under random-persistence and correlated-direction stress tests Performance Under Stress Tests The robustness of the Algorithm B controller is tested against three stress diagnostics 1.
Ruth: Persistence sweep: The Algorithm A controller decays monotonically under persistent directional regimes, whereas Algorithm B stays profitable throughout 2.
Paper summary: Ruth: To summarize the paper "Deep Learning of Robust Market Making under Regime-Switching Order Flow," the core thesis is that classical market making strategies fail when order flow isn't stationary, which is a reality in modern markets with algorithmic metaorders.
Warren: The authors claim that they develop a deep reinforcement learning market maker called an RLMM, specifically using a Rainbow style distributional DQN, and they find it outperforms the GLFT benchmark in the stationary setting.
Iris: But the critical part isn't just the stationary performance; it’s how they tackle regime switching. They show that by augmenting the state with a Bayesian change point filter for flow bias and a queue adjusted quote exposure imbalance, profitability is substantially restored under regime switching order flow.
Ruth: Furthermore, they introduce a scenario bandit robust fine tuning step where low return regime scenarios are upweighted to improve performance under random persistence and correlated direction stress tests.
Warren: The paper uses the event driven zero intelligence Santa Fe LOB simulator to calibrate the GLFT policy through censore. This simulator is designed to model a realistic limit order book environment, though it has some limitations that they acknowledge later on.
Iris: They also identify inventory saturation under persistent directional order flow as the main failure point for stationarily trained market making agents and GLFT.
Ruth: So, it moves beyond just finding a static optimal strategy; it builds a controller that adapts its belief about the current market regime dynamically.
Conclusion: Ruth: In conclusion, this paper on "Deep Learning of Robust Market Making under Regime-Switching Order Flow" is about developing a method to make deep learning models resilient to changing market conditions.
Warren: The central contribution is that the two stage fine tuning pipeline takes a stationary optimal RLMM and makes it profitable even when order flow switches regimes, and it tightens the tail of potential profit and loss under stress tests.
Iris: It means that for someone who only listens to the show, this work shows how you can take a strategy optimized for a steady market and make it robust against the kind of non stationary behavior we actually see in real order flow.
Ruth: The paper addresses the problem of profitability under non stationary order flow by combining Bayesian regime belief with a queue adjusted exposure signal, followed by scenario bandit reweighting.
Warren: It’s about turning a stationarily trained agent into one that stays profitable under clustered directional pressure and random persistence.
More episodes
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- The day in conversation for 2026-11-18
- Daily Summary for 2026-11-18
- 2601.05274-On the use of case estimate and transactional payment data in neural networks for individual loss reserving
- 2009.00868-Loss-Given-Default Modeling by Post-Last Passage Time Process
- 2606.30070-Financial Resilience Evaluation: From Conditional Expectations to Dynamic Convex Risk Measures
- 2212.12687-Measuring price impact and information content of trades in a time-varying setting
- Daily Summary for 2026-11-17
- 2401.08323-Dynamic portfolio selection under generalized disappointment aversion