Amortized Neural Filtering for Calibrated Posteriors in Statistical Arbitrage

summary

The gist

Statistical arbitrage trades on an estimate of the state of a residual spread, and conventional practice discards the uncertainty of that estimate.

In short

Statistical arbitrage uses estimates of residual spread states but ignores uncertainty. The Neural Filter Estimator (NFE) is trained on simulated mean-reverting processes to return a calibrated posterior distribution for each spread. Results show this posterior accurately reflects its own uncertainty, significantly reducing forecast errors on Korean equities compared to conventional point estimates.

Key concepts

Neural Filter Estimator (NFE): A neural network trained on simulated paths of a mean-reverting process. It learns the latent state and uncertainty of spread movements by testing candidates against real residuals, allowing it to return a calibrated probability distribution for each spread's regime.
Posterior Distribution: The NFE provides a probability distribution over the unknown latent state of every spread in a cross-section. This moves beyond simple point estimates by quantifying the uncertainty associated with each estimate, which is crucial for robust trading decisions.
Innovation Law Selection: This process involves testing different mathematical models (candidates) against real market residuals to select the best one. Only stochastic volatility processes with asymmetric innovations pass this test, ensuring the model captures realistic market dynamics.
Amortized Methods: These methods reduce forecast errors by using a uniform shrinkage approach when sizing positions based on posterior estimates. This technique effectively cuts the gap between point forecasts and realized returns by a factor of two to eight.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 26 episodes
Flow Matching — explained in 23 episodes
Quantum Fisher Information (QFI) — explained in 22 episodes
Retrieval-Augmented Generation (RAG) — explained in 21 episodes
Sample Complexity — explained in 20 episodes
Federated Learning — explained in 19 episodes

This episode discusses

The paper

Amortized Neural Filtering for Calibrated Posteriors in Statistical Arbitrage · Read on arXiv

Minkey Chang

Seoul National University

Statistical arbitrage trades on an estimate of the state of a residual spread, and conventional practice discards the uncertainty of that estimate. We keep the uncertainty and use it in the trading decision. A Neural Filter Estimator (NFE) is trained on simulated paths of a mean-reverting process whose innovation law is selected by testing candidates against real residuals; once trained, it returns the posterior over the latent state of every spread in a cross-section in one forward pass. Three results follow. On simulated paths with known truth, every practical estimator, ours included, falls in one narrow band of accuracy well below an estimator given full information, so the posterior gains no accuracy over a point estimate; its contribution is a correct account of its own uncertainty. On Korean equities, net of costs, the conventional point estimate forecasts about a thousand basis points per holding period more than it realizes, and the amortized methods cut that gap by a factor of roughly two to eight. Sizing positions from the posterior on the same learned factors as the end-to-end Attention Factor Model matches the Sharpe ratio of those factors under the base rule within the period's noise, at no more risk.

DOI: 10.1145/3858063.3858894

Transcript

Introduction to the show: ident: Quantitative Finance. The day in quantitative finance: models of markets, risk and prices.

Ruth: We're looking at the paper: "Amortized Neural Filtering for Calibrated Posteriors in Statistical Arbitrage".

Warren: Here's the gist of the paper. Statistical arbitrage trades on an estimate of the state of a residual spread, and conventional practice discards the uncertainty of that estimate.

Ruth: The Neural Filter Estimator (NFE) is trained on simulated paths of a mean-reverting process whose innovation law is selected by testing candidates against real residuals; once trained, it returns the posterior over the latent state of every spread in a cross-section in one forward pass.

Warren: The Core Problem and Solution Conventional statistical arbitrage discards the uncertainty of its estimates, leading to potential misreadings of data.

Ruth: The paper introduces the Neural Filter Estimator (NFE) which adds a dynamic model of the latent state and its uncertainty to trading decisions.

Paper summary: Iris: To summarize this paper, "Amortized Neural Filtering for Calibrated Posteriors in Statistical Arbitrage," the core thesis is that traditional statistical arbitrage discards the uncertainty of its estimates when trading residual spreads >

Ruth: They propose using a Neural Filter Estimator, or NFE, which trains on simulated paths of a mean-reverting process where they select the innovation law by testing against real residuals >

Warren: Once trained, this NFE returns the posterior over every spread's latent state in a cross-section with just one forward pass >

Iris: The key is that this posterior carries a calibrated probability attached to each regime, allowing traders to use that probability instead of a simple point estimate for trading decisions >

Ruth: The paper shows three main results. First, on simulated paths with known truth, every estimator falls in one narrow band of accuracy well below what you get with full information >

Warren: So the posterior doesn't actually improve the accuracy over a point estimate; it just provides a correct account of its own uncertainty >

Iris: Second, they find that on Korean equities, net of costs, conventional point estimates forecast about a thousand basis points per holding period more than they realize >

Ruth: And their amortized methods cut that gap by a factor of roughly two to eight >

Warren: Third, sizing positions using the posterior on those same learned factors matches the Sharpe ratio of those factors under the base rule within the period's noise at no more risk >

Conclusion: Iris: Looking at "Amortized Neural Filtering for Calibrated Posteriors in Statistical Arbitrage," the authors are essentially moving away from just forecasting what happens next and instead nowcast what the state of a residual spread might be >

Ruth: The NFE is presented as an amortized filter that returns the marginal of the posterior in a single forward pass for every window and every asset >

Warren: So it’s not just one prediction; it’s a continuous update across time, giving you information on where you are now >

Iris: The central claim they make is that this posterior is calibrated, meaning its coverage tracks the nominal level throughout the process, and sizing based on this matches the Sharpe ratio of those factors within the period's noise at no more risk >

Ruth: The main methodological lesson here seems to be that a label-free diagnostic needs a null model and some positive control before you can actually interpret its output >

Warren: And they pinpoint where identification loss is highest, which is when you estimate the level from the same window as the reversion itself >

More episodes

← Home

Watch the video for this episode