Skillful Thunderstorm Forecasts Learned Directly from Observations

arXiv:2601.17268 · physics.ao-ph · Submitted 2026-01-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Skillful Thunderstorm Forecasts Learned Directly from Observations".

Mira: Here's the gist of the paper. Accurate short-term prediction of clouds and precipitation is critical for severe weather warnings, aviation safety, and renewable energy operations The paper introduces Stormscope, a family of transformer-based generative diffusion models trained on high-resolution, multi-band geostationary satellite imagery and ground-based weather radar over the continental United States It achieves leading performance on standard deterministic and probabilistic verification metrics across forecast horizons from 1 to 6 hours when evaluated against extrapolation methods and operational mesoscale NWP models By operating in observation space, Stormscope establishes a new paradigm for multi-modal AI-driven nowcasting with direct applicability to operational forecasting workflows.

Kai: Model Architecture and Training Signal The backbone of the forecasting system is a Diffusion Transformer (DiT) architecture that operates on a 512 × 896 grid with a spatial resolution of 6 km derived from the HRRR grid The model learns the conditional probability distribution of the future state xt+ given a sequence of past observations xt− This is achieved by utilizing the Elucidated Diffusion Models (EDM) framework which provides a robust discretization of the Diffusion SDE The network Dθ is trained to minimize the weighted denoising loss L(θ) = Ex,σ,n λ(σ)∥Dθ(xt+ + nσ; xt−; c; σ) − xt+∥ 2 2 The training noise range

σmin, σmax: is partitioned into two regimes (coarse and fine) at an intermediate point σint with σmin < σint < σmax to improve fidelity of both large-scale structure and fine-scale convective detail.

Mira: Multi-Modal Input Integration The model employs a dense input conditioning scheme where exogenous variables, namely the GOES Input, MRMS Input, and GFS (z500) Input are used The nowcasting models are conditioned on only observational data and do not require NWP assimilation systems to initialize forecasts For nearcasting, the satellite radiance model is conditioned on the 500 hPa geopotential height forecast, while the weather radar model is conditioned on the simulated composite reflectivity forecast The high-level inference workflow involves a Diffusion Transformer (DiT) architecture that functions as a pure data-driven model in nowcasting and acts as a learned fusion mechanism in nearcasting.

Kai: Performance Gains and Evaluation Metrics The evaluation demonstrates that Stormscope outperforms the HRRR model in predicting explicit storm evolution, even without ensembling, for single-member deterministic FSS metrics up to 3 hr In ensemble mode, its probabilistic performance for radar CRPS systematically outperforms a 5-member lagged ensemble HRRR baseline Furthermore, the model shows superior nowcasting performance relative to both the pySTEPS and HRRR baselines when measured by Fractions Skill Score across various lead times.

Paper summary: Kai: So we're wrapping up on this paper, "Skillful Thunderstorm Forecasts Learned Directly from Observations," which shows how a diffusion transformer model trained purely on satellite and radar data can make short term forecasts of clouds and rain that are competitive with established numerical weather models <ref:2601.17268#pg1>.

Mira: The paper moves the conversation away from just using probability distributions alone to actually generating skillful forecasts of storm evolution itself, which is what matters for operational safety <ref:2601.17268#pg3>.

Lev: And when you consider the real world, this means that if we build the hardware and handle those initial shocks in the model correctly, we could see a way to get very fast initialization for weather warnings <ref:2601.17268#pg5>.

Kai: It's about using what's already measured—the satellite bands and the radar reflectivity—to predict the next few hours of atmospheric change without needing a full numerical model run first <ref:2601.17268#pg4>.

Mira: The implication is that data driven approaches have a real capacity to surpass strong physics baselines when dealing with mesoscale storm evolution, provided we address the limitations they pointed out about resolution and conditioning <ref:2601.17268#pg5>.

Conclusion: Kai: So we're looking at "Skillful Thunderstorm Forecasts Learned Directly from Observations," and the authors are showing how you can predict storms by just looking at what's already been observed, like satellite pictures and radar scans.

Mira: It really boils down to using AI, specifically diffusion models, to learn the physical process of storm evolution directly from that data stream. They're not relying on a massive computer simulation running constantly in the background.

Lev: From an error correction standpoint, it’s interesting because this model operates in observation space—it doesn't need those huge computational resources to run a full weather model initialization first.

Kai: Exactly. The title is quite direct, focusing on learning those forecasts directly from observations instead of relying on traditional physics-based models to do the heavy lifting upfront.

Mira: And what this means for us in the long term is that we might be able to get much faster, more up-to-date initial conditions for severe weather alerts than we currently can with standard numerical weather prediction systems.

Lev: But they did admit some constraints, like using only sparse data conditioning for the nearcasting part and working at a six kilometer resolution instead of the fine resolution you see in some radar products.

Kai: Right, so it's a powerful prototype that shows machine learning can handle storm evolution at this scale, but it’s not yet ready to replace everything because of those limitations.

Mira: It proves that these types of generative models have the capacity to outperform some physics-based methods when you look specifically at how storms actually evolve over a short time frame.

Lev: So next, we need to figure out if we can push this framework further, especially since they mentioned scalability for larger domains and higher resolutions in their training setup.

Jaideep Pathak, Mohammad Shoaib Abbas, Peter Harrington, Zeyuan Hu, Noah Brenowitz, Suman Ravuri, Alberto Carpentieri, Jussi Leinonen, Corey Adams, Oliver Hennigh, Nicholas Geneva, Dale Durran, and Mike Pritchard

NVIDIA Corporation

physics.ao-ph

Submitted: 2026-01-24

Updated: 2026-10-30

Comments: 38 pages, 13 figures. Substantially revised manuscript introducing a higher-resolution model with improved forecast skill, expanded comparisons with state-of-the-art numerical and AI forecasting models, and evaluation by operational meteorologists. Updated title and author list

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: Accurate short-term prediction of clouds and precipitation is critical for severe weather warnings, aviation safety, and renewable energy operations The paper introduces Stormscope…

Key concepts

Diffusion Transformer (DiT)
This is the core neural network architecture used in Stormscope. It learns the probability distribution of future weather states by gradually denoising noisy data, much like how diffusion models work in image generation. It processes spatial grids to predict what the next state will be.
Elucidated Diffusion Models (EDM)
The framework used for training Stormscope provides a robust way to discretize the mathematical process of diffusion. This allows the model to learn complex weather patterns by minimizing a denoising loss function, effectively teaching it how to reverse the noise and predict future conditions.
Multi-Modal Input Integration
Stormscope takes in several different types of data simultaneously, including satellite radiance, radar reflectivity forecasts, and atmospheric pressure forecasts. It uses these varied inputs to make highly informed predictions for both general nowcasting and specific nearcasting tasks.
Observation Space Operation
Instead of relying on traditional Numerical Weather Prediction (NWP) models to initialize the forecast, Stormscope is designed to work directly with observed data. This allows it to bypass slow assimilation steps, enabling very fast, high-frequency predictions crucial for real-time warnings.

Terminology

Summary

Accurate short-term prediction of clouds and precipitation is critical for severe weather warnings, aviation safety, and renewable energy operations The paper introduces Stormscope, a family of transformer-based generative diffusion models trained on high-resolution, multi-band geostationary satellite imagery and ground-based weather radar over the continental United States It achieves leading performance on standard deterministic and probabilistic verification metrics across forecast horizons from 1 to 6 hours when evaluated against extrapolation methods and operational mesoscale NWP models By operating in observation space, Stormscope establishes a new paradigm for multi-modal AI-driven nowcasting with direct applicability to operational forecasting workflows. p. 0

Where the paper says it

Evaluated against extrapolation methods and operational mesoscale NWP models, Stormscope achieves leading performance on standard deterministic and probabilistic verification metrics across forecast horizons from 1 to 6 hours. By operating in observation space, Stormscope establishes a new paradigm for multi-modal AI-driven nowcasting with direct applicability to operational forecasting workflows.

Model Architecture and Training Signal

The backbone of the forecasting system is a Diffusion Transformer (DiT) architecture that operates on a 512 × 896 grid with a spatial resolution of 6 km derived from the HRRR grid The model learns the conditional probability distribution of the future state xt+ given a sequence of past observations xt− This is achieved by utilizing the Elucidated Diffusion Models (EDM) framework which provides a robust discretization of the Diffusion SDE The network Dθ is trained to minimize the weighted denoising loss L(θ) = Ex,σ,n λ(σ)∥Dθ(xt+ + nσ; xt−; c; σ) − xt+∥ 2 2 The training noise range [σmin, σmax] is partitioned into two regimes (coarse and fine) at an intermediate point σint with σmin < σint < σmax to improve fidelity of both large-scale structure and fine-scale convective detail.

Multi-Modal Input Integration

The model employs a dense input conditioning scheme where exogenous variables, namely the GOES Input, MRMS Input, and GFS (z500) Input are used The nowcasting models are conditioned on only observational data and do not require NWP assimilation systems to initialize forecasts For nearcasting, the satellite radiance model is conditioned on the 500 hPa geopotential height forecast, while the weather radar model is conditioned on the simulated composite reflectivity forecast The high-level inference workflow involves a Diffusion Transformer (DiT) architecture that functions as a pure data-driven model in nowcasting and acts as a learned fusion mechanism in nearcasting.

Performance Gains and Evaluation Metrics

The evaluation demonstrates that Stormscope outperforms the HRRR model in predicting explicit storm evolution, even without ensembling, for single-member deterministic FSS metrics up to 3 hr In ensemble mode, its probabilistic performance for radar CRPS systematically outperforms a 5-member lagged ensemble HRRR baseline Furthermore, the model shows superior nowcasting performance relative to both the pySTEPS and HRRR baselines when measured by Fractions Skill Score across various lead times. Object-based verification metrics like POD, SR, CSI, and FB show that Stormscope generally validates more favorably than in pySTEPS for discrete precipitating cloud systems.

Scalability and Operational Utility

The framework is designed to be scalable to larger domains and higher resolutions through domain parallel training using the ShardTensor API from NVIDIA PhysicsNeMo This approach allows training a 3 km satellite nowcast model over the CONUS domain spanning 1.8 million pixels The system can generate forecasts every few minutes for maximum update frequency in near-real-time applications This capability points toward a future of high-frequency initialization that bypasses the latency of traditional data assimilation workflows.

Limitations and Future Directions

The work admits several important limitations, including using only sparse synoptic conditioning for the nearcasting model, suggesting that denser conditioning could uncover additional skill gains The current method operates at a 6 km resolution, whereas pySTEPS can operate at the native resolution of the gridded radar product Future work should involve scoring against full physics-based ensembles to better contextualize performance against non-ML methods. The authors hope this prototype demonstrates that ML methods have the capacity to surpass strong physics baselines of storm evolution at mesoscale resolution. p. 8

Where the paper says it

[17, 18] to surpass the strong physics-based HRRR baseline. This confirms our working hypothesis that explicit storm-resolving dynamical trajectories can be skillfully learned from direct observations, consistent with the qualitative view.

The success of this prototype over the US emboldens similar attempts over other countries. p. 0
Where the paper says it

The approach is extensible, with demonstrated computational scaling to larger domains and higher resolutions. As Stormscope relies on globally available satellite observations (and radar where available), it offers a pathway to extend skillful mesoscale forecasting to oceanic regions and countries without strong operational mesoscale modeling programs.

Variable Description Modality Use

The input and forecast data streams include Blue Visible band at 0.47 µm Satellite Input, Forecast, Red Visible band at 0. p. 4

Where the paper says it

The color visualization is composed of a linear combination of the visible radiance forecasts (a, c, e) and observations (b, d, f) using the Blue (0. 47 µm), Red (0.

64 µm Satellite Input, Forecast, Veggie Near-IR band at 0. p. 2
Where the paper says it

The model exclusively uses observations for nowcasting (0-2 hours) and incorporates synoptic-scale information (500 hPa geopotential height) for near-term mesoscale forecasts. Integrates radar observations where available as an additional prognostic variable to forecast composite radar reflectivity conditioned on the satellite imagery forecast, achieving performance competitive with the best numerical operational models for up to 6 hours.

86 µm Satellite Input, Forecast The Clean IR Longwave Window IR band at 10. p. 7
Where the paper says it

We visualize the Clean IR window channel (10. 35 µm) for our GOES forecast model and also show the simulated brightness temperature (SBT) forecast for the HRRR model, which synthetically models forward radiative transfer through physical state variables.

35 µm Satellite Input, Forecast is also utilized for the forecast. p. 2
Where the paper says it

86 µm)—to produce a natural-looking composite without the interference of overlaid terrain or textures. This specific forecast was initialized on June 25, 2024, at 12:00 UTC.

Radar MRMS Max. p. 3
Where the paper says it

35 µm IR channel, which is critical for identifying deep convection having cold cloud temperatures and associated vertical development of intense storm systems. Complementing the satellite radiance, Figure 3 shows the Stormscope radar reflectivity forecasts compared against Multi-Radar Multi-Sensor (MRMS) verification data.

Comp. Refl. MRMS Radar Input, Forecast serves as an additional prognostic variable conditioned on the backbone. p. 2
Where the paper says it

The model exclusively uses observations for nowcasting (0-2 hours) and incorporates synoptic-scale information (500 hPa geopotential height) for near-term mesoscale forecasts. Integrates radar observations where available as an additional prognostic variable to forecast composite radar reflectivity conditioned on the satellite imagery forecast, achieving performance competitive with the best numerical operational models for up to 6 hours.

The nearcasting model is conditioned on Z500 Geopotential Height at 500hPa GFS Model Input, and the radar model is conditioned on the simulated composite reflectivity forecast. p. 2
Where the paper says it

The model exclusively uses observations for nowcasting (0-2 hours) and incorporates synoptic-scale information (500 hPa geopotential height) for near-term mesoscale forecasts. Integrates radar observations where available as an additional prognostic variable to forecast composite radar reflectivity conditioned on the satellite imagery forecast, achieving performance competitive with the best numerical operational models for up to 6 hours.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Multi-modal Nowcasting Capability: The Stormscope model can perform nowcasting from 0 to 2 hours at a high temporal resolution of 10 minutes by leveraging only geostationary and radar observations without conditioning on Numerical Weather Prediction (NWP) states.

  2. Nearcasting with Synoptic Context: The nearcasting mode extends the forecast window to up to 12 hours, incorporating synoptic-scale information such as the 500 hPa geopotential height from the Global Forecast System (GFS) for a more constrained prediction at longer lead times.

  3. Explicit Dynamical Trajectory Generation: The system generates ensembles of spatiotemporally coherent dynamical weather trajectories, which allows for representing the shape, structure and timing of storm-scale weather as well as the uncertainty therein, rather than just probability distributions alone.

  4. Enhanced Object Tracking Performance: The model achieves superior nowcasting performance relative to both pySTEPS and HRRR baselines when using object-based verification metrics like Probability of Detection (POD), Success Ratio (SR), Critical Success Index (CSI), and Frequency Bias (FB) for storms above a 40 dBZ reflectivity threshold.

  5. Scalable Domain Parallelism for High Resolution: The framework is capable of scaling to higher resolutions, such as 3 km, over the entire Continental United States using domain parallelization on a two-dimensional device mesh to handle large spatial domains efficiently during training and inference.

Abstract

Nowcasting the evolution of severe convective storms over the first few hours after observations are collected is critical for issuing timely, accurate warnings. Forecast accuracy is hindered because the variables that matter most, such as rain, hail, and lightning, are imperfectly represented by numerical model physics and are difficult to initialize correctly. Here we introduce StormScope, a computationally efficient generative forecasting system that learns storm-scale evolution directly from satellite, radar, and lightning observations over the continental United States with no numerical model used for training or initialization. StormScope produces coherent, physically plausible storm trajectories with well-calibrated uncertainty. Across more than 900 forecasts, it greatly exceeds the National Oceanic and Atmospheric Administration's state-of-the-art forthcoming operational numerical modeling system in both deterministic and probabilistic skill through the 0-2-h nowcasting window. StormScope's skill for strong radar echoes exceeds the operational state of the art by approximately 75% and 45% at lead times of 30 min and 2 h, a relative improvement several times larger than machine learning has delivered for global medium-range forecasting. Crucially, it also anticipates storms that develop after forecast initialization, establishing its utility beyond extrapolation of existing radar echoes. It likewise surpasses a leading AI nowcasting model for intense precipitation. Together, these results establish learning directly from observations as a route toward skillful, well-calibrated storm-scale forecasts that is unconstrained by the initialization and microphysics bottlenecks limiting numerical models and their emulators.

Sources

Related papers