Bayesian Reconciliation of Overlapping Multi Step Forecasts: A Post Processing Framework for Cross Origin Coherence

summary

The gist

Multi-step forecasting systems often issue overlapping predictions that can disagree even from the same model…

In short

The framework addresses cross-origin incoherence, where overlapping forecasts disagree even from the same model. It treats these forecasts as noisy measurements of a single underlying latent trajectory using a Bayesian state-space model. This post-processing method reduces disagreement by estimating horizon-specific biases and uncertainties, successfully enforcing coherence across different forecast origins.

Key concepts

Cross-Origin Incoherence
This is the disagreement among multiple forecasts that all aim to predict the same future outcome but come from different forecasting systems. A perfectly coherent system would have zero incoherence, meaning all overlapping predictions for a specific time point match exactly.
Bayesian State-Space Framework
The method models the entire forecast history as noisy observations of a hidden, smooth underlying process (a stationary AR(1) process). It uses Bayesian statistics to estimate this latent trajectory while accounting for different biases and uncertainties that occur at different forecast horizons.
Model Agnosticism
The framework is designed to work on any existing forecast, regardless of how it was generated. It does not require access to the forecasting model's internal structure or training data, making it a simple post-processing layer for improving forecast quality.

Terminology used across episodes

This episode discusses

The paper

Bayesian Reconciliation of Overlapping Multi Step Forecasts: A Post Processing Framework for Cross Origin Coherence · Read on arXiv

University of Sao Paulo, Brazil

Overlapping multi step forecasts often disagree, a form of vertical instability that we term cross origin incoherence. We propose a Bayesian reconciliation framework that treats overlapping forecasts as noisy, biased measurements of a common latent trajectory. Operating as a model agnostic post processing layer, it requires no access to model internals. Simulations demonstrate substantial incoherence reduction and robust latent trajectory recovery, remaining highly resilient (correlation 0.972) even under severe nonlinear stress tests. An empirical application to S&P 500 volatility evaluates AR(1), ETS, and LSTM models. The framework reduces incoherence by over 97% across all models while delivering fully calibrated probabilistic forecasts (93% to 94% coverage). Notably, it improves point accuracy for ETS and LSTM. Furthermore, the estimated horizon specific parameters act as a diagnostic X-ray, revealing hidden biases in neural networks without requiring access to their architecture

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Bayesian Reconciliation of Overlapping Multi Step Forecasts".

Jane: Here's the gist of the paper. Multi-step forecasting systems often issue overlapping predictions that can disagree even from the same model, a phenomenon termed cross-origin incoherence The proposed Bayesian reconciliation framework treats these overlapping forecasts as noisy and potentially biased measurements of a common latent trajectory This method operates as a model-agnostic post-processing layer, requiring no access to the forecasting model’s architecture, parameters, or training data It jointly reconciles forecasts targeting the same future time while accounting for horizon-specific bias and uncertainty using a Bayesian state-space model.

Tom: Controlled simulations demonstrate substantial reductions in cross-origin incoherence and accurate latent trajectory recovery under the assumed data generating process.

Jane: Defining Cross-Origin Incoherence Cross-origin incoherence is defined as disagreement among overlapping forecasts that target the same future outcome but are issued at different forecast origins The cross-origin incoherence metric is formally defined as the average squared disagreement among all forecasts targeting the same future time t A perfectly coherent system has I = 0, meaning all forecasts for the same future date agree exactly.

Tom: The metric is used specifically as a post-processing diagnostic and as a measure of the degree of disagreement before and after Bayesian reconciliation.

Paper summary: Jane: We’ve been looking at how the authors of Bayesian Reconciliation of Overlapping Multi Step Forecasts: A Post Processing Framework for Cross Origin Coherence tackled this issue with a Bayesian state-space model to treat overlapping forecasts as noisy measurements of a common latent trajectory >

Tom: They're proposing this framework as something model agnostic, meaning it doesn't care if you used an AR one or an LSTM, its only job is to correct the structural failure mode where forecasts for the same future event shouldn't contradict each other depending on when they were made >

Lu: The core of it is that the inner sum in Equation one computes that average squared disagreement among all forecasts targeting the same future time t, and then they take an outer sum over all target times to get a single scalar measure of cross-origin incoherence >

Meng: So from what I’ve seen, if you're running a production system where you need multiple overlapping predictions for the same output, this is a tool you could use to enforce coherence after the fact >

Lalam: It’s essentially treating forecasts targeting the same timestamp as noisy measurements of a common latent process and using that structure to separate horizon specific bias from the overall level of that latent trajectory >

Tom: The title itself says it's about reconciliation, which is important because you can have multiple good predictions but they might fight each other when they overlap >

Jane: So what this implies for the broader area of forecasting is that we can have a system where different parts disagree on the exact future point, but we can use this Bayesian approach to make them all agree on a single coherent path >

Conclusion: Tom: So we've been looking at this paper about Bayesian Reconciliation of Overlapping Multi Step Forecasts, and what they're doing is taking all those overlapping predictions that might contradict each other and trying to make them agree on one path using a Bayesian approach >

Jane: Right, so the title itself says it’s about reconciliation, which is important because you can have multiple good predictions but they might fight each other when they overlap >

Lu: The authors are proposing this framework as something model agnostic, meaning it doesn't care if you used an AR one or an LSTM; its only job is to correct the structural failure mode where forecasts for the same future event shouldn't contradict each other depending on when they were made >

Meng: From my side, I’m wondering what that means for deployment; if we can use this after the fact, does it just fix a calibration issue or something deeper about how these models interact >

Lalam: It’s essentially treating forecasts targeting the same timestamp as noisy measurements of a common latent process and using that structure to separate horizon specific bias from the overall level of that latent trajectory >

Tom: So, what this implies for the broader area of forecasting is that we can have a system where different parts disagree on the exact future point, but we can use this Bayesian approach to make them all agree on a single coherent path >

Jane: It’s about taking these messy predictions and forcing them onto a shared reality, which is actually really cool because it gives us some concrete numbers on how much they were fighting each other >

More episodes

← Home