Skillful Thunderstorm Forecasts Learned Directly from Observations
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.
Kai: We're looking at the paper: "Skillful Thunderstorm Forecasts Learned Directly from Observations".
Mira: Here's the gist of the paper. Accurate short-term prediction of clouds and precipitation is critical for severe weather warnings, aviation safety, and renewable energy operations The paper introduces Stormscope, a family of transformer-based generative diffusion models trained on high-resolution, multi-band geostationary satellite imagery and ground-based weather radar over the continental United States It achieves leading performance on standard deterministic and probabilistic verification metrics across forecast horizons from 1 to 6 hours when evaluated against extrapolation methods and operational mesoscale NWP models By operating in observation space, Stormscope establishes a new paradigm for multi-modal AI-driven nowcasting with direct applicability to operational forecasting workflows.
Kai: Model Architecture and Training Signal The backbone of the forecasting system is a Diffusion Transformer (DiT) architecture that operates on a 512 × 896 grid with a spatial resolution of 6 km derived from the HRRR grid The model learns the conditional probability distribution of the future state xt+ given a sequence of past observations xt− This is achieved by utilizing the Elucidated Diffusion Models (EDM) framework which provides a robust discretization of the Diffusion SDE The network Dθ is trained to minimize the weighted denoising loss L(θ) = Ex,σ,n λ(σ)∥Dθ(xt+ + nσ; xt−; c; σ) − xt+∥ 2 2 The training noise range
σmin, σmax: is partitioned into two regimes (coarse and fine) at an intermediate point σint with σmin < σint < σmax to improve fidelity of both large-scale structure and fine-scale convective detail.
Mira: Multi-Modal Input Integration The model employs a dense input conditioning scheme where exogenous variables, namely the GOES Input, MRMS Input, and GFS (z500) Input are used The nowcasting models are conditioned on only observational data and do not require NWP assimilation systems to initialize forecasts For nearcasting, the satellite radiance model is conditioned on the 500 hPa geopotential height forecast, while the weather radar model is conditioned on the simulated composite reflectivity forecast The high-level inference workflow involves a Diffusion Transformer (DiT) architecture that functions as a pure data-driven model in nowcasting and acts as a learned fusion mechanism in nearcasting.
Kai: Performance Gains and Evaluation Metrics The evaluation demonstrates that Stormscope outperforms the HRRR model in predicting explicit storm evolution, even without ensembling, for single-member deterministic FSS metrics up to 3 hr In ensemble mode, its probabilistic performance for radar CRPS systematically outperforms a 5-member lagged ensemble HRRR baseline Furthermore, the model shows superior nowcasting performance relative to both the pySTEPS and HRRR baselines when measured by Fractions Skill Score across various lead times.
Paper summary: Kai: So we're wrapping up on this paper, "Skillful Thunderstorm Forecasts Learned Directly from Observations," which shows how a diffusion transformer model trained purely on satellite and radar data can make short term forecasts of clouds and rain that are competitive with established numerical weather models <ref:2601.17268#pg1>.
Mira: The paper moves the conversation away from just using probability distributions alone to actually generating skillful forecasts of storm evolution itself, which is what matters for operational safety <ref:2601.17268#pg3>.
Lev: And when you consider the real world, this means that if we build the hardware and handle those initial shocks in the model correctly, we could see a way to get very fast initialization for weather warnings <ref:2601.17268#pg5>.
Kai: It's about using what's already measured—the satellite bands and the radar reflectivity—to predict the next few hours of atmospheric change without needing a full numerical model run first <ref:2601.17268#pg4>.
Mira: The implication is that data driven approaches have a real capacity to surpass strong physics baselines when dealing with mesoscale storm evolution, provided we address the limitations they pointed out about resolution and conditioning <ref:2601.17268#pg5>.
Conclusion: Kai: So we're looking at "Skillful Thunderstorm Forecasts Learned Directly from Observations," and the authors are showing how you can predict storms by just looking at what's already been observed, like satellite pictures and radar scans.
Mira: It really boils down to using AI, specifically diffusion models, to learn the physical process of storm evolution directly from that data stream. They're not relying on a massive computer simulation running constantly in the background.
Lev: From an error correction standpoint, it’s interesting because this model operates in observation space—it doesn't need those huge computational resources to run a full weather model initialization first.
Kai: Exactly. The title is quite direct, focusing on learning those forecasts directly from observations instead of relying on traditional physics-based models to do the heavy lifting upfront.
Mira: And what this means for us in the long term is that we might be able to get much faster, more up-to-date initial conditions for severe weather alerts than we currently can with standard numerical weather prediction systems.
Lev: But they did admit some constraints, like using only sparse data conditioning for the nearcasting part and working at a six kilometer resolution instead of the fine resolution you see in some radar products.
Kai: Right, so it's a powerful prototype that shows machine learning can handle storm evolution at this scale, but it’s not yet ready to replace everything because of those limitations.
Mira: It proves that these types of generative models have the capacity to outperform some physics-based methods when you look specifically at how storms actually evolve over a short time frame.
Lev: So next, we need to figure out if we can push this framework further, especially since they mentioned scalability for larger domains and higher resolutions in their training setup.
Jaideep Pathak, Mohammad Shoaib Abbas, Peter Harrington, Zeyuan Hu, Noah Brenowitz, Suman Ravuri, Alberto Carpentieri, Jussi Leinonen, Corey Adams, Oliver Hennigh, Nicholas Geneva, Dale Durran, and Mike Pritchard
NVIDIA Corporation
physics.ao-ph
Submitted: 2026-01-24
Updated: 2026-10-30
Comments: 38 pages, 13 figures. Substantially revised manuscript introducing a higher-resolution model with improved forecast skill, expanded comparisons with state-of-the-art numerical and AI forecasting models, and evaluation by operational meteorologists. Updated title and author list
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 92/100
The gist: Accurate short-term prediction of clouds and precipitation is critical for severe weather warnings, aviation safety, and renewable energy operations The paper introduces Stormscope…
Key concepts
- Diffusion Transformer (DiT)
- This is the core neural network architecture used in Stormscope. It learns the probability distribution of future weather states by gradually denoising noisy data, much like how diffusion models work in image generation. It processes spatial grids to predict what the next state will be.
- Elucidated Diffusion Models (EDM)
- The framework used for training Stormscope provides a robust way to discretize the mathematical process of diffusion. This allows the model to learn complex weather patterns by minimizing a denoising loss function, effectively teaching it how to reverse the noise and predict future conditions.
- Multi-Modal Input Integration
- Stormscope takes in several different types of data simultaneously, including satellite radiance, radar reflectivity forecasts, and atmospheric pressure forecasts. It uses these varied inputs to make highly informed predictions for both general nowcasting and specific nearcasting tasks.
- Observation Space Operation
- Instead of relying on traditional Numerical Weather Prediction (NWP) models to initialize the forecast, Stormscope is designed to work directly with observed data. This allows it to bypass slow assimilation steps, enabling very fast, high-frequency predictions crucial for real-time warnings.
Terminology
Summary
Accurate short-term prediction of clouds and precipitation is critical for severe weather warnings, aviation safety, and renewable energy operations The paper introduces Stormscope, a family of transformer-based generative diffusion models trained on high-resolution, multi-band geostationary satellite imagery and ground-based weather radar over the continental United States It achieves leading performance on standard deterministic and probabilistic verification metrics across forecast horizons from 1 to 6 hours when evaluated against extrapolation methods and operational mesoscale NWP models By operating in observation space, Stormscope establishes a new paradigm for multi-modal AI-driven nowcasting with direct applicability to operational forecasting workflows. p. 0 Evaluated against extrapolation methods and operational mesoscale
NWP models, Stormscope achieves leading performance on standard deterministic and probabilistic verification metrics across forecast horizons from 1 to 6 hours. By operating in observation
space, Stormscope establishes a new paradigm for multi-modal AI-driven nowcasting with direct
applicability to operational forecasting workflows.Where the paper says it
Model Architecture and Training Signal
The backbone of the forecasting system is a Diffusion Transformer (DiT) architecture that operates on a 512 × 896 grid with a spatial resolution of 6 km derived from the HRRR grid The model learns the conditional probability distribution of the future state xt+ given a sequence of past observations xt− This is achieved by utilizing the Elucidated Diffusion Models (EDM) framework which provides a robust discretization of the Diffusion SDE The network Dθ is trained to minimize the weighted denoising loss L(θ) = Ex,σ,n λ(σ)∥Dθ(xt+ + nσ; xt−; c; σ) − xt+∥ 2 2 The training noise range [σmin, σmax] is partitioned into two regimes (coarse and fine) at an intermediate point σint with σmin < σint < σmax to improve fidelity of both large-scale structure and fine-scale convective detail.
Multi-Modal Input Integration
The model employs a dense input conditioning scheme where exogenous variables, namely the GOES Input, MRMS Input, and GFS (z500) Input are used The nowcasting models are conditioned on only observational data and do not require NWP assimilation systems to initialize forecasts For nearcasting, the satellite radiance model is conditioned on the 500 hPa geopotential height forecast, while the weather radar model is conditioned on the simulated composite reflectivity forecast The high-level inference workflow involves a Diffusion Transformer (DiT) architecture that functions as a pure data-driven model in nowcasting and acts as a learned fusion mechanism in nearcasting.
Performance Gains and Evaluation Metrics
The evaluation demonstrates that Stormscope outperforms the HRRR model in predicting explicit storm evolution, even without ensembling, for single-member deterministic FSS metrics up to 3 hr In ensemble mode, its probabilistic performance for radar CRPS systematically outperforms a 5-member lagged ensemble HRRR baseline Furthermore, the model shows superior nowcasting performance relative to both the pySTEPS and HRRR baselines when measured by Fractions Skill Score across various lead times. Object-based verification metrics like POD, SR, CSI, and FB show that Stormscope generally validates more favorably than in pySTEPS for discrete precipitating cloud systems.
Scalability and Operational Utility
The framework is designed to be scalable to larger domains and higher resolutions through domain parallel training using the ShardTensor API from NVIDIA PhysicsNeMo This approach allows training a 3 km satellite nowcast model over the CONUS domain spanning 1.8 million pixels The system can generate forecasts every few minutes for maximum update frequency in near-real-time applications This capability points toward a future of high-frequency initialization that bypasses the latency of traditional data assimilation workflows.
Limitations and Future Directions
The work admits several important limitations, including using only sparse synoptic conditioning for the nearcasting model, suggesting that denser conditioning could uncover additional skill gains The current method operates at a 6 km resolution, whereas pySTEPS can operate at the native resolution of the gridded radar product Future work should involve scoring against full physics-based ensembles to better contextualize performance against non-ML methods. The authors hope this prototype demonstrates that ML methods have the capacity to surpass strong physics baselines of storm evolution at mesoscale resolution. p. 8 [17, 18] to surpass the strong physics-based HRRR baseline. This confirms our working hypothesis
that explicit storm-resolving dynamical trajectories can be skillfully learned from direct observations,
consistent with the qualitative view. The approach is extensible, with demonstrated
computational scaling to larger domains and higher resolutions. As Stormscope relies on globally
available satellite observations (and radar where available), it offers a pathway to extend skillful mesoscale forecasting to oceanic regions and countries without strong operational mesoscale
modeling programs.Where the paper says it
Where the paper says it
Variable Description Modality Use
The input and forecast data streams include Blue Visible band at 0.47 µm Satellite Input, Forecast, Red Visible band at 0. p. 4 The
color visualization is composed of a linear combination of the visible radiance forecasts (a, c, e) and
observations (b, d, f) using the Blue (0. 47 µm), Red (0. The model exclusively uses observations for nowcasting (0-2 hours) and incorporates synoptic-scale information
(500 hPa geopotential height) for near-term mesoscale forecasts. Integrates radar observations where available as an additional prognostic variable to forecast
composite radar reflectivity conditioned on the satellite imagery forecast, achieving performance
competitive with the best numerical operational models for up to 6 hours. We visualize the Clean IR window channel (10. 35 µm) for our GOES forecast model and also show the
simulated brightness temperature (SBT) forecast for the HRRR model, which synthetically models
forward radiative transfer through physical state variables. 86
µm)—to produce a natural-looking composite without the interference of overlaid terrain or textures. This specific forecast was initialized on June 25, 2024, at 12:00 UTC. 35
µm IR channel, which is critical for identifying deep convection having cold cloud temperatures and
associated vertical development of intense storm systems. Complementing the satellite radiance, Figure 3 shows the Stormscope radar reflectivity forecasts compared against Multi-Radar Multi-Sensor
(MRMS) verification data. The model exclusively uses observations for nowcasting (0-2 hours) and incorporates synoptic-scale information
(500 hPa geopotential height) for near-term mesoscale forecasts. Integrates radar observations where available as an additional prognostic variable to forecast
composite radar reflectivity conditioned on the satellite imagery forecast, achieving performance
competitive with the best numerical operational models for up to 6 hours. The model exclusively uses observations for nowcasting (0-2 hours) and incorporates synoptic-scale information
(500 hPa geopotential height) for near-term mesoscale forecasts. Integrates radar observations where available as an additional prognostic variable to forecast
composite radar reflectivity conditioned on the satellite imagery forecast, achieving performance
competitive with the best numerical operational models for up to 6 hours.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Multi-modal Nowcasting Capability: The Stormscope model can perform nowcasting from 0 to 2 hours at a high temporal resolution of 10 minutes by leveraging only geostationary and radar observations without conditioning on Numerical Weather Prediction (NWP) states.
-
Nearcasting with Synoptic Context: The nearcasting mode extends the forecast window to up to 12 hours, incorporating synoptic-scale information such as the 500 hPa geopotential height from the Global Forecast System (GFS) for a more constrained prediction at longer lead times.
-
Explicit Dynamical Trajectory Generation: The system generates ensembles of spatiotemporally coherent dynamical weather trajectories, which allows for representing
the shape, structure and timing of storm-scale weather as well as the uncertainty therein,
rather than just probability distributions alone. -
Enhanced Object Tracking Performance: The model achieves superior nowcasting performance relative to both pySTEPS and HRRR baselines when using object-based verification metrics like Probability of Detection (POD), Success Ratio (SR), Critical Success Index (CSI), and Frequency Bias (FB) for storms above a 40 dBZ reflectivity threshold.
-
Scalable Domain Parallelism for High Resolution: The framework is capable of scaling to higher resolutions, such as 3 km, over the entire Continental United States using domain parallelization on a two-dimensional device mesh to handle large spatial domains efficiently during training and inference.
Abstract
Nowcasting the evolution of severe convective storms over the first few hours after observations are collected is critical for issuing timely, accurate warnings. Forecast accuracy is hindered because the variables that matter most, such as rain, hail, and lightning, are imperfectly represented by numerical model physics and are difficult to initialize correctly. Here we introduce StormScope, a computationally efficient generative forecasting system that learns storm-scale evolution directly from satellite, radar, and lightning observations over the continental United States with no numerical model used for training or initialization. StormScope produces coherent, physically plausible storm trajectories with well-calibrated uncertainty. Across more than 900 forecasts, it greatly exceeds the National Oceanic and Atmospheric Administration's state-of-the-art forthcoming operational numerical modeling system in both deterministic and probabilistic skill through the 0-2-h nowcasting window. StormScope's skill for strong radar echoes exceeds the operational state of the art by approximately 75% and 45% at lead times of 30 min and 2 h, a relative improvement several times larger than machine learning has delivered for global medium-range forecasting. Crucially, it also anticipates storms that develop after forecast initialization, establishing its utility beyond extrapolation of existing radar echoes. It likewise surpasses a leading AI nowcasting model for intense precipitation. Together, these results establish learning directly from observations as a route toward skillful, well-calibrated storm-scale forecasts that is unconstrained by the initialization and microphysics bottlenecks limiting numerical models and their emulators.
Sources
- AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score
- Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling
- HRRRCast: a data-driven emulator for regional weather forecasting at convection allowing scales
- How to use score-based diffusion in earth system science: A satellite nowcasting example
- Four-hour thunderstorm nowcasting using a deep diffusion model of satellite data
- A review of radar-based nowcasting of precipitation and applicable machine learning techniques
- Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification
- An Operational Deep Learning System for Satellite-Based High-Resolution Global Nowcasting
- Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light
- eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
- Climate in a Bottle: Towards a Generative Foundation Model for the Kilometer-Scale Global Atmosphere
- FourCastNet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale
Related papers
- Anthropogenic Heat in Urban Climate Systems: Forcing, Sensitivity, and Feedback
- NORi: An ML-Augmented Ocean Boundary Layer Parameterization
- A Checklist to assess the energy and carbon impacts of ML/AI applications in Earth System Modeling
- Quantifying AI data center nitrogen oxide (NO x) emissions from space
- Wind vector retrieval from miniaturized wave-enabled sea-surface drifters
- Stochastic coupling of climate variables and ice volume over the Late Pleistocene glacial cycles