Online conformal inference for multi-step time series forecasting

summary

The gist

Online conformal inference for multi-step time series forecasting establishes that optimal h-step-ahead forecast errors exhibit serial correlation up to lag (h−1) under a general non-stationary…

In short

The Autocorrelated Multi-step Conformal Prediction (AcMCP) method was developed to improve online conformal inference for multi-step time series forecasting. It accounts for serial correlation in forecast errors up to lag h-1 using an MA(h-1) model, leading to more statistically efficient and adaptive prediction intervals that guarantee long-run coverage without distribution shift assumptions.

Key concepts

Optimal h-step-ahead forecast errors
The research shows that the best possible errors for predicting h steps ahead in a non-stationary time series process are related to an MA(h-1) process. This means the error at one step is correlated with previous errors up to lag h-1, which is crucial for building accurate prediction intervals.
Autocorrelated Multi-step Conformal Prediction (AcMCP)
This method incorporates the known serial correlation structure of multi-step forecast errors directly into the online prediction update. By using this structure, AcMCP produces more statistically efficient and narrower prediction intervals compared to simpler methods like MPID.
Long-run coverage guarantees
AcMCP proves that its prediction intervals maintain correct coverage over the long term (as time approaches infinity). This guarantee is achieved without needing assumptions about how the underlying data distribution might change over time, making it robust for real-world forecasting.

Terminology used across episodes

This episode discusses

The paper

Online conformal inference for multi-step time series forecasting · Read on arXiv

Academy of Mathematics and Systems Science, Chinese Academy of Sciences · Monash University

Transcript

Introduction to the show: ident: Statistics. The day in statistics: methods, theory and the inference under them.

Tom: We're looking at the paper: "Online conformal inference for multi-step time series forecasting".

Jane: Here's the gist of the paper. Online conformal inference for multi-step time series forecasting establishes that optimal h-step-ahead forecast errors exhibit serial correlation up to lag (h−1) under a general non-stationary autoregressive data generating process <ref:2410.13115#pg7>.

Tom: To leverage these properties, the Autocorrelated Multi-step Conformal Prediction (AcMCP) method is proposed, which effectively incorporates autocorrelations in multi-step forecast errors, resulting in more statistically efficient prediction intervals <ref:2410.13115#pg7>.

Jane: This method guarantees asymptotic marginal coverage for multi-step prediction intervals, though the coverage error admits an upper bound that increases with the forecasting horizon <ref:2410.13115#pg7>.

Tom: The paper extends several easy-to-implement conformal prediction methods to accommodate multi-step scenarios <ref:2410.13115#pg7>.

Title and authors: Tom: Okay, so we're shifting focus slightly now to the setup of this work, looking at the title and who wrote it. The paper is "Online conformal inference for multi-step time series forecasting," written by Xiaoqian Wang from the Academy of Mathematics and Systems Science and Monash University <ref:2410.13115#pg1>.

Jane: That’s a solid setup. The title itself immediately tells you the two big challenges they are tackling here: online inference and multi-step time series forecasting. It’s not just one problem, it's a combination of both <ref:2410.13115#pg7>.

Lu: The authors bring in expertise from different areas, which is typical for these kinds of advanced methods. They're drawing on conformal prediction foundations and applying them specifically to the tricky domain of time series data <ref:2410.13115#pg3>.

Meng: It’s good to see that the work is rooted in established frameworks, but the focus is clearly on extending those frameworks where they usually break down. They aren't just applying a standard technique; they are changing the technique itself to handle this specific temporal dependency <ref:2410.13115#pg7>.

Lalam: The paper draws heavily on established work in conformal prediction, referencing things like Papadopoulos et al. and Vovk et al., which shows they’re building on a known foundation <ref:2410.13115#pg2>. It’s not starting from scratch, which is smart for establishing credibility <ref:2410.13115#pg7>.

Jane: So, when you look at the authors and the title, what should a listener immediately understand about what this paper is trying to achieve? What’s the big picture?

Tom: The big picture is moving conformal prediction beyond exchangeable data into a more realistic setting—the time series domain—and making it robust enough to handle those inherent temporal dependencies <ref:2410.13115#pg3>.

Lu: They are trying to show that even with complex, nonstationary autoregressive data generating processes, we can still construct valid prediction intervals using this method <ref:2410.13115#pg7>.

Meng: So they’re aiming for distribution-free uncertainty quantification in a setting where the data isn't nice and tidy, which is exactly where most of our production models live <ref:2410.13115#pg7>.

Lalam: It’s about making prediction intervals reliable when the underlying process has a history, not just independent points <ref:2410.13115#pg7>.

The paper's summary: Tom: Now let’s look at the actual summary they provide. It boils down to this: they identify that for an h-step-ahead forecast error sequence, it has a specific serial correlation property up to lag h minus one <ref:2410.13115#pg7>.

Jane: That’s the key finding again. So they’re not just saying "time series are hard"; they're giving us a mathematical reason why their prediction intervals need to be different for different forecast horizons <ref:2410.13115#pg7>.

Lu: They then leverage this property by proposing the Autocorrelated Multi-step Conformal Prediction, AcMCP, which effectively incorporates these autocorrelations into the nonconformity scores <ref:2410.13115#pg7>.

Meng: So they are essentially modifying the standard conformal prediction process to account for that dependency structure in the errors <ref:2410.13115#pg7>. It’s like adding a correction term based on what happened a few steps ago <ref:2410.13115#pg7>.

Lalam: Exactly, and this incorporation of the lag h minus one dependence is what leads to more statistically efficient prediction intervals <ref:2410.13115#pg7>. It’s a direct result of using that property they established <ref:2410.13115#pg7>.

Jane: So, if I boil it down for the listener who isn't reading the paper, what is the practical difference between this AcMCP method and a standard conformal prediction approach?

Tom: The practical difference is that AcMCP doesn't treat each forecast error as completely independent when constructing its interval; it uses the known serial correlation up to lag h minus one <ref:2410.13115#pg7>. It’s a structural adjustment based on the data generating process, not just an arbitrary assumption about independence <ref:2410.13115#pg7>.

Lu: That distinction is important because standard methods often rely on exchangeability, which time series data inherently violates <ref:2410.13115#pg3>. This paper shows a way around that violation by modeling the correlation itself <ref:2410.13115#pg7>.

Meng: I see. So they are essentially building a model of the error structure first, and then using that model to make our prediction intervals more accurate, rather than just assuming independence <ref:2410.13115#pg7>. That makes sense for deployment concerns <ref:2410.13115#pg7>.

Lalam: It’s about being more statistically rigorous about the dependence, which ultimately translates into tighter, more accurate prediction intervals when you actually use this paper's framework <ref:2410.13115#pg7>.

The paper's improvements: Tom: Moving on to the specific improvements they are proposing in the methodology. They mentioned several ways to extend conformal prediction for time series, including split conformal prediction and online multi-step split conformal prediction <ref:2410.13115#pg10>.

Jane: So they aren't just presenting one solution; they are showing a family of tools, like SCP or MSCP, that try different things when dealing with the time series problem <ref:2410.13115#pg7>.

Lu: They also introduce weighted conformal prediction, MWCP, which adapts the NexCP method by using a weighted quantile estimate for building the intervals instead of just one fixed rule <ref:2410.13115#pg7>. This contrasts with MSCP because it assigns weights to the nonconformity scores based on observed feature values <ref:two thousand four hundred ten point one three one one five#pg7.

Meng: Weighted quantile estimates sound like they might be more adaptive than just a fixed set of rules, but that brings us back to my deployment concern—how much extra work does this weighting add to the inference pipeline?

Tom: It adds complexity, Meng. The paper notes that MWCP assigns weights to those nonconformity scores based on observed feature values, which is a different approach than MSCP's recursive update structure <ref:two thousand four hundred ten point one three one one five#pg7.

Jane: So it’s a choice between relying on a recursive online update versus using these feature-dependent weights for the quantile estimation process <ref:2410.13115#pg7>. Which one is better depends entirely on the data's specific characteristics, right?

Lu: The paper also extends adaptive conformal inference, or MACP, by iteratively estimating an optimal miscoverage rate alpha star t plus h t via a sequential adjustment process <ref:two thousand four hundred ten point one three one one five#pg7. This lets the system adjust its confidence level based on what it's seeing sequentially <ref:2410.13115#pg7>.

Meng: Sequential adjustment sounds ambitious, but if it works, it could mean we don't have to manually tune a coverage parameter for every single time step <ref:2410.13115#pg7>. That’s definitely appealing for scale <ref:two thousand four hundred ten point one three one one five#pg7.

Tom: The overall message here is that they are providing a toolkit—split conformal prediction, online multi-step split, weighted methods, and adaptive methods—so you can choose the one that fits your specific operational constraints <ref:2410.13115#pg7>.

Jane: So we have a lot of options to choose from when dealing with time series uncertainty. It’s not just one single answer for every prediction problem <ref:2410.13115#pg7>.

Conclusion: Tom: Alright, let's wrap up this discussion on the paper "Online conformal inference for multi-step time series forecasting." The main conclusion is that AcMCP provides long-run coverage guarantees without needing any assumptions about distribution shifts <ref:2410.13115#pg7>.

Jane: That’s a huge statement, Tom. It means we can deploy this method and have confidence in our prediction intervals even if the underlying data distribution changes over time, as long as we stick to the theoretical framework <ref:2410.13115#pg7>.

Lu: The paper confirms that by accounting for the autocorrelation structure, AcMCP is more statistically efficient than methods like MPID when you look at h equals three <ref:2410.13115#pg21>. It’s a measurable gain in efficiency <ref:two thousand four hundred ten point one three one one five#pg7.

Meng: And for production, the empirical validation suggests it can be closer to the target coverage within local windows, and its prediction intervals are shorter or at least comparable in length when the coverage is similar <ref:2410.13115#pg31>. That’s tangible improvement <ref:two thousand four hundred ten point one three one one five#pg35.

Lalam: I just want to say that the work shows we can achieve long-run coverage guarantees for multi-step univariate time series forecasting under a general nonstationary autoregressive data generating process without making any assumptions on data distributional shifts <ref:2410.13115#pg7>. That’s the main contribution I see here <ref:two thousand four hundred ten point one three one one five#pg7.

Jane: So, to summarize, this paper is about creating a robust online tool that accounts for the serial correlation in multi-step errors to give us tighter, more efficient prediction intervals for time series data <ref:2410.13115#pg7>. It sets a high bar for what we expect from uncertainty quantification in sequential forecasting <ref:two thousand four hundred ten point one three one one five#pg7.

Tom: That’s the essence of the Autocorrelated Multi-step Conformal Prediction method, or AcMCP. It’s about leveraging data structure to get better accuracy <ref:2410.13115#pg7>. Moving on, next week we'll look at some papers dealing with distribution-free inference for individual treatment effects.

More episodes

← Home