Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control

arXiv:2610.10440 · cs.LG · Submitted 2026-10-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control".

Jane: The gist Seq-Flow introduces a flow matching approach that tracks online forecast distributions by transporting the previous forecast to the updated one conditioned on new observations,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So wrapping up, Seq-Flow is a conditional flow model that transports samples from the previous forecast distribution to the updated one using an ODE, aiming for accuracy with few evaluations.

Jane: The authors introduce this flow matching approach and show that starting from an informative distribution allows for accurate updates because successive forecasts are often very similar.

Lu: They solve the issue of recursive reuse by training via self-rollouts and renoising the source distributions to prevent errors from compounding across updates, even though they admit that self-rollout training adds some extra cost.

Meng: And they showed stability over four hundred consecutive updates on Beam Spill forecasting, which is a solid performance metric for long-running systems.

Tom: The implication here is that this method offers a way to do online probabilistic forecasting where you don't have to rely on generating entirely new samples from noise at every single step.

Jane: It changes how we think about sequential prediction; instead of starting over, you are continuously refining your prediction based on what you already know.

Conclusion: Jane: So, to wrap up, Seq-Flow is basically this flow matching approach that lets you track online forecasts by moving from the previous forecast to the new one using an ODE, which means you can get accurate predictions with just a few flow evaluations.

Tom: Yeah, it’s about this conditional flow model that starts from an informative distribution and transports samples, which is smart because successive forecasts usually only differ a little bit.

Lu: The authors tackle the problem where reusing past forecasts messes up the source distribution, and they fix that by training with self-rollouts generated by an exponential moving average copy of the model.

Meng: But you have to be careful, I mean, self-rollout training adds extra time to get it running, so we need to know if that cost is worth it for a real deployment.

Lalam: Based on my literature scan, this paper is positioning itself as a way to handle sequential prediction where the distributions drift over time.

Jane: So, what does this mean for someone who just watches the show? It means we can keep refining our predictions step-by-step without having to start completely fresh every single time.

Tom: Exactly, it’s about continuous improvement instead of restarting the whole process from scratch on every new piece of data.

Lu: The core idea is that this recursive transport, when trained correctly with those self-rollouts and renoising, can stay stable over many updates.

Meng: Stability is key for production, because if the error starts compounding fast after just a few steps, that whole system breaks down on real traffic.

Lalam: The results they show—reducing the Continuous Ranked Probability Score by sixty-five percent relative to a standard diffusion baseline—that's a big number when you're talking about forecasting accuracy.

Jane: Sixty-five percent reduction in error sounds significant, but we need to look at the caveats they mention on page six about how much training time that self-rollout method actually takes.

Tom: The title itself, "Seq-Flow: Efficient Probabilistic Forecasting with Self-Rollout Error Control," tells you exactly what this paper is trying to achieve and how it does it.

Lu: It’s about getting the efficiency of flow matching while specifically controlling the error that builds up when you reuse your own past predictions as a starting point.

Meng: So, if we take this idea into a production setting, I'm wondering how robust this training strategy is when you introduce truly noisy or unpredictable new observations in real-time.

Jane: That’s the next big question—how well does it handle the messy reality of real-world data streams compared to their simulated tasks like Beam Spill?

Tom: It seems they are confident because they showed stability over four hundred consecutive updates on Beam Spill, which is a pretty long run for this kind of system.

Lalam: They’re suggesting that by training on these self-rollouts and adding that source perturbation, you get a model that won't immediately drift off course with each new observation.

Jane: And what does this mean for the future of AI forecasting? It suggests we can move toward systems that are inherently recursive and adaptive rather than just static models trained once.

Yinan Huang, Shitij Govil, Bo Dai, Pan Li

Georgia Institute of Technology

cs.LG

Submitted: 2026-10-07

Updated: 2026-10-07

Code: https://github.com/Graph-COM/Seq-Flow

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: The gist Seq-Flow introduces a flow matching approach that tracks online forecast distributions by transporting the previous forecast to the updated one conditioned on new observations, enabling

Key concepts

Flow Matching: A technique where a continuous Ordinary Differential Equation (ODE) is used to transform one probability distribution into another. In Seq-Flow, this ODE transports samples from an older forecast distribution to a newer one based on new data, mimicking how beliefs are updated in classical filtering.
Self-Rollout Training: A training strategy where the model generates its own training data by iteratively using its own previous outputs as input for the next step. This exposes the model to errors that compound over time, teaching it how to correct these recursive mistakes during forecasting.
Source Distribution Error: The problem where reusing a forecast distribution causes a mismatch between what the model was trained on (the source) and what it is currently predicting (the target). Seq-Flow uses self-rollouts and renoising to actively correct this error, ensuring stability across many successive forecast updates.
Renoising: A method used to improve robustness by adding controlled noise or perturbation to the source distributions. By slightly altering the starting points of the flow transport, Seq-Flow becomes less sensitive to errors introduced by reusing previous forecasts.

Terminology

Probabilistic Forecasting — 1× in this paper · explained in 3 episodes

Summary

The gist Seq-Flow introduces a flow matching approach that tracks online forecast distributions by transporting the previous forecast to the updated one conditioned on new observations, enabling accurate forecasting with few flow evaluations.

Seq-Flow Overview

Seq-Flow is a conditional flow model whose ODE transports samples from the previous forecast distribution to the updated one Because successive forecasts often differ only modestly, this transport starts from an informative distribution and can produce accurate updates with few flow evaluations. It uses a renoised sample from the previous forecast as the source distribution for the next flow, enabling efficient few-step updates. This recursive transport parallels classical filtering, which updates predictive beliefs as observations arrive.

Addressing Recursive Source-Distribution Error

The paper addresses the challenge that recursively reusing forecasts creates a train–test shift in the flow’s source distribution, which is a failure mode absent when each generation starts from a fixed Gaussian prior. To mitigate this mismatch, Seq-Flow is trained through self-rollouts generated by an exponential moving average (EMA) copy of the update model. This training strategy exposes the model to its own forecast sources, teaching it to correct errors that would otherwise compound across updates. Furthermore, renoising is used to improve robustness by perturbing the source distributions. The training objective involves regressing to the conditional displacement between a renoised previous forecast and the ground-truth future trajectory using a flow-matching objective.

Training Strategies for Stability

Several strategies are employed to make self-rollout training stable and efficient. These include:

Applying stop gradient to all model rollouts Xˆt, so that gradients are not propagated through earlier flow ODE and rollout steps.
Maintaining an exponential moving average of the model parameters, denoted by θEMA, and using the EMA model to generate rollout sources. This stabilizes the training and makes optimization easier.
Randomly slicing a contiguous trajectory chunk of length T0 ≪ T and performing self-rollout only within this T0-step chunk to reduce training cost while still exposing the model to multi-step error propagation.

Performance and Comparison

Seq-Flow is evaluated on particle-accelerator Beam Spill forecasting, simulated Burgers’ dynamics, and real-world fluid–structure interaction data from RealPDEBench. Experiments show that Seq-Flow reduces CRPS by 65% relative to a standard diffusion baseline under a matched 3-NFE sampling budget on Beam Spill. The model remains stable over long forecast runs, maintaining accuracy through more than 400 consecutive updates despite training on self-rollouts of at most four updates. Seq-Flow achieves the lowest CRPS and RMSE on Beam Spill forecasting (0.015 and 0.080, respectively) compared to other methods under three-NFE sampling.

Key Findings on Robustness

The ablation study indicates that both self-rollout exposure and source perturbation are important for robust recursive flow transports. Removing either component leads to rapidly accumulating error over successive forecast updates, consistent with the recursive sourcedistribution mismatch described in Section 4.3. Furthermore, using a short self-rollout horizon T0 = 1 and T0 = 4 is sufficient to control error over episodes T = 100 on synthetic and T = 430 on Beam Spill respectively. The paper concludes that Seq-Flow achieves strong few-step performance and remains stable over long recursive deployments.

Experimental Details

The experiments utilize Transformer backbones for synthetic and Beam Spill tasks, and spatiotemporal U-Nets for RealPDE-FSI. The forecast horizon H is set to 5 for synthetic tasks and 10 for Beam Spill, Burgers’, and FSI tasks. The evaluation metrics reported include the Continuous Ranked Probability Score (CRPS) and Root Mean Squared Error (RMSE). The model's training combines Gaussian-source and self-rollout losses with equal weights of 0.5. The inference process involves solving the ODE starting from a renoised previous forecast as the source distribution for the next flow. The model's parameters are updated using AdamW with weight decay 1e-4 and (β1, β2) = (0.9, 0.999). The renoising level τr is treated as a hyperparameter to tune in practice. The context lengths C0 and Cmax specify the initial and maximum observed history for the model.

Limitations

A limitation of our approach is that self-rollout training introduces extra training cost. This work is primarily supported by National Science Foundation (NSF) under awards PHY2117997, IIS-2239565, IIS-2428777, and CCF-2402816; the Google Cloud Research Credit, 2026; the Nvidia Academic Award, 2026. The complete training and inference procedures are described in Algorithms 1 and 2. The authors would like to thank Hans Hao-Hsun Hsu for his insightful discussions on this work. This work is primarily supported by National Science Foundation (NSF) under awards PHY2117997, IIS-2239565, IIS-2428777, and CCF-2402816; the Google Cloud Research Credit, 2026; the Nvidia Academic Award, 2026. Prof. Bo Dai would like to acknowledge support from NSF ECCS-2401391, NSF IIS-2403240, NSF IIS-2552007 and ONR N000142512173. The authors are available at https://github.com/Graph-COM/Seq-Flow. The references section lists numerous works on diffusion models, flow matching, and related temporal data modeling techniques.

--- Page 6 ---

Seq-Flow Overview

Seq-Flow is a conditional flow model whose ODE transports samples from the previous forecast distribution to the updated one. Because successive forecasts often differ only modestly, this transport starts from an informative distribution and can produce accurate updates with few flow evaluations. It uses a renoised sample from the previous forecast as the source distribution for the next flow, enabling efficient few-step updates. This recursive transport parallels classical filtering, which updates predictive beliefs as observations arrive.

Addressing Recursive Source-Distribution Error

Improvements for AI systems

Bold header: Sequence-based distribution tracking

Seq-Flow learns an update operator between successive forecast distributions by incorporating new observed history, which allows it to track the evolution of the forecasting distribution directly, rather than relying on transporting samples from a fixed Gaussian source to pt at every physical time step.

Bold header: Robust recursive error control

The system implements self-rollout training with robust source distributions and renoising, which addresses the train–test shift in the flow’s source distribution by exposing the model to source-distribution errors that arise during recursive deployment.

Bold header: Efficient few-step forecasting

Seq-Flow enables accurate updates with few flow evaluations by using samples from the previous predictive distribution as a source, which is particularly effective for tasks where recursively reuse creates a challenge and allows it to maintain accuracy over more than 400 consecutive updates.

Abstract

Many scientific forecasting tasks require updating a distribution over future trajectories as new observations arrive. Conventional diffusion and flow models generate each forecast from Gaussian noise, often at the cost of many sampling steps. Warm-start methods reuse earlier predictions to reduce this cost, but their models are not trained to perform the forecast update itself, which can compromise quality under few-step sampling. In this work, we introduce Seq-Flow, a conditional flow model whose ODE transports samples from the previous forecast distribution to the updated one. Because successive forecasts often differ only modestly, this transport starts from an informative distribution and can produce accurate updates with few flow evaluations. Recursive reuse also creates a challenge: errors in one forecast become errors in the initial states of subsequent flows. We address this with self-rollout training, in which a moving average copy of the model generates forecasts that initialize later training updates. Unlike self-forcing methods, which reuse generated outputs as conditioning context, Seq-Flow reuses them as the source of the next flow. Experiments On particle-accelerator beam spill forecasting show Seq-Flow reduces CRPS by 65% under a few-NFE sampling budget, while remaining competitive with strong baselines on fluid-dynamics forecasting tasks. Although trained on self-rollouts of at most four updates, Seq-Flow remains accurate over more than 400 consecutive updates. Our code is available at https://github.com/Graph-COM/Seq-Flow.