Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning

arXiv:2603.03351 · physics.geo-ph, physics.comp-ph, physics.flu-dyn · Submitted 2026-02-26 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning".

Mira: Here's the gist of the paper. Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical.

Kai: This paper introduces scientific multi-agent reinforcement learning (SMARL) to develop closures for canonical prototypes of atmospheric/oceanic turbulence using only the enstrophy spectrum as a reward.

Mira: These online-learned closures enable stable simulations with up to five orders of magnitude fewer degrees of freedom that reproduce high-fidelity simulation statistics and capturing in particular extremes.

Kai: Pruning redundant reasoning tokens The research addresses the challenges in predicting extreme weather events by overcoming the limitations of traditional closures and existing AI approaches.

Title and authors: Kai: We're moving into the title and authors of this paper, "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning". I want to explain what that actually means for us.

Mira: The title tells us they are using reinforcement learning on multiscale simulations to predict extreme events in geophysical turbulence <ref:2603.03351#pg1>.

Kai: It's about taking these complex atmospheric and oceanic flows and building something that can handle the really rare, intense weather events that we need to model accurately <ref:2603.03351#pg1>.

Lev: From an engineering standpoint, what they're doing is developing a closure—that’s essentially a mathematical approximation for the physics happening at scales smaller than what you can resolve directly in a simulation <ref:2603.03351#pg2>.

Kai: They are training this closure using reinforcement learning, which is an online learning method where the agent learns by interacting directly with the simulation environment <ref:2603.03351#pg5>.

Mira: The authors are Guan, Amoudruz, Litvinov, Jakhar, Mojgani, Koumoutsakos and Hassanzadeh <ref:2603.03351#pg1>.

Lev: It's notable that they are focusing on canonical prototypes of atmospheric and oceanic turbulence to test this method <ref:2603.03351#pg3>.

The paper's summary: Kai: Now, let’s look at the core summary of this paper, "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning". They explain how this SMARL framework works in simpler terms.

Mira: They train a DNN to learn the flow-dependent coefficient cl for a physics-based closure, specifically the Leith model <ref:2603.03351#pg3>.

Lev: The training signal is defined by matching the enstrophy spectrum of their LES plus SMARL solver with that of a very short direct numerical simulation or DNS <ref:2603.03351#pg2>.

Kai: The instantaneous reward is chosen to be large when that enstrophy spectrum is close to the reference, which helps guide the learning process <ref:2603.03351#pg1>.

Mira: They state that this setup allows them to test performance by comparing the kinetic energy spectrum and vorticity probability density function, or PDF, of the DNS against an LES solver with a resolution that is one hundred sixty to one hundred sixty-three times coarser <ref:2603.03351#pg3>.

Lev: They show that this LES plus SMARL spectrum and PDF match the DNS for simulations that are about two thousand times the length of their training dataset <ref:2603.03351#pg2>.

The paper's improvements: Kai: So, what are the actual improvements this paper suggests over existing methods? They’re focusing on how this SMARL-based closure actually performs better.

Mira: The main improvement is that the RL-Leith model outperforms traditional physics-based dynamic models when matching the DNS PDF, even at those tails corresponding to rare extreme weather events <ref:2603.03351#pg1>.

Lev: It seems they capture both diffusion and backscattering, which is shown by the distribution of Leith coefficients cl spreading across a wider range than DLeith <ref:2603.03351#pg1>, which suggests less excessive diffusion compared to other models.

Kai: They also show generalization capability, meaning the RL-Leith closure trained on one flow can be used directly for a flow with fifteen times higher Reynolds number without needing new data <ref:2603.03351#pg1>.

Mira: And their interpretability analysis using Sobol indices shows that large-scale structures containing most of the kinetic energy and enstrophy have the most contribution to predicting the closure coefficient cl <ref:2603.03351#pg1>.

Conclusion: Kai: To wrap up, we're looking at the conclusion of "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning". They summarize what this means for climate modeling and weather forecasts.

Mira: Essentially, they demonstrate a data-efficient framework that learns LES closures for geophysical turbulence <ref:2603.03351#pg1>.

Lev: It’s superior to existing AI-based closure modeling approaches because it works with sparse training data and low LES resolutions that are usually inaccessible to equation discovery methods <ref:2603.03351#pg4>.

Kai: The results show that simulations with SMARL-based closures significantly outperform those with classical physics-based closures in capturing extreme events, suggesting both diffusion and backscattering are accurately represented <ref:2603.03351#pg1>.

Mira: This framework is shown to be stable and can run thoughts of times longer than the lengths of the training set and training horizon <ref:2603.03351#pg1>.

Lev: For my work in error correction, it’s important that this stability holds, because running real hardware simulations requires a very reliable closure to manage the overhead effectively <ref:2603.03351#pg4>.

Kai: It really shows promise for scaling up these methods to global climate models and Earth system models in the future <ref:2603.03351#pg1>.

Mira: It’s a way forward for handling those hard, rare events that current models struggle with <ref:2603.03351#pg1>.

Yifei Guan, Lucas Amoudruz, Sergey Litvinov, Karan Jakhar, Rambod Mojgani, ‡Petros Koumoutsakos‡and Pedram Hassanzadeh

Department of Geophysical Sciences, University of Chicago · Department of Mechanical Engineering, Union College · Computational Science and Engineering Laboratory, Harvard University · Department of Mechanical Engineering, Rice University · Committee on Computational and Applied Mathematics, University of Chicago

physics.geo-ph, physics.comp-ph, physics.flu-dyn

Submitted: 2026-02-26

Updated: 2026-02-26

Comments: 21 pages; 8 figures

DOI: 10.1016/j.jocs.2026.103022

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 87/100

The gist: Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical.

Key concepts

Subgrid-scale (SGS) closures
These are mathematical approximations used in large-eddy simulations (LES) to model physical processes occurring at scales smaller than the grid resolution. Traditional physics-based closures often fail to accurately capture complex energy transfers between resolved and subgrid scales.
Enstrophy Spectrum
This is a measure of the kinetic energy distribution across different spatial scales in a turbulent flow. The SMARL framework uses this spectrum as the primary reward signal, training agents to learn how to correctly model these scale-dependent interactions for accurate simulation results.
Reinforcement Learning (RL) for Closures
SMARL is an online learning approach where agents iteratively refine a closure model by interacting with a numerical solver. The system learns the flow-dependent coefficient of a physics model by matching its output spectrum to that of high-fidelity simulations, enabling it to capture extremes without needing massive training datasets.

Terminology

Summary

Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical. p. 0 This paper introduces scientific multi-agent reinforcement learning (SMARL) to develop closures for canonical prototypes of atmospheric/oceanic turbulence using only the enstrophy spectrum as a reward. p. 0 These online-learned closures enable stable simulations with up to five orders of magnitude fewer degrees of freedom that reproduce high-fidelity simulation statistics and capturing in particular extremes. p. 0

Pruning redundant reasoning tokens

The research addresses the challenges in predicting extreme weather events by overcoming the limitations of traditional closures and existing AI approaches. Traditional physics-based closures suffer from structural errors, such as producing excessive diffusion that dampens extremes. p. 0 Supervised offline learning is hindered by the need for abundant high-fidelity training data and can lead to instabilities. p. 0 Online learning algorithms are an alternative, but reliance on differentiable numerical solvers or scalable optimizers hinders broad use. p. 0 SMARL is introduced as a potent tool for developing closures capable of capturing extremes in atmospheric/oceanic flows. p. 0

How the training signal is built

The SMARL framework utilizes a specific reward function to guide the learning process. The agents are trained to learn a closure by matching the enstrophy spectrum of the LES solver with that of a very short DNS. p. 2 Specifically, the instantaneous reward is chosen to be large when the enstrophy spectrum is close to the reference. p. 0 The state of the system s′(t) is defined by the enstrophy spectrum Zˆ(k, t) up to the Nyquist wavenumber at LES (or cut-off wavenumber) kc. p. 2

Where the gains come from

The advantages of SMARL are demonstrated through several key findings in the experiments. The RL-Leith model outperforms traditional physics-based dynamic models in terms of matching the DNS PDF, even at the tails corresponding to rare extreme weather events. p. 4 Furthermore, the model captures both diffusion and backscattering, which is evidenced by the distribution of Leith coefficients cl spreading across a wider range than DLeith. p. 4 The generalization capability is shown as the RL-Leith closure trained on one flow can be directly used for a flow with 15× higher Reynolds number without new data. p. 6

Interpretability of the SMARL model

The interpretability of the SMARL-based closures is analyzed using Sobol indices to indicate the significance of contributions from states to actions. The analysis shows that large-scale structures containing most kinetic energy and enstrophy have the most contribution to the prediction of the closure’s coefficient cl. p. 8 The next significant contribution stems from the high-wavenumber region near the cut-off wavenumber, kc, which is interpreted as directly responsible for interscale energy and enstrophy transfer. p. 8

Conclusion

The paper demonstrates a data-efficient, SMARL-based framework to learn LES closures for geophysical turbulence with broad potential applications to climate modeling. This framework is superior to existing AI-based closure modeling approaches as it works with sparse amounts of training data and low LES resolutions inaccessible to equation discovery. p. 8 The results show that LES with SMARL-based closures significantly outperform LES with classical physics-based closures in capturing extreme events, suggesting that both diffusion and backscattering are accurately represented. p. 8 This framework is shown to be stable and can easily run thoughts of times longer than the lengths of the training set and training horizon. p. 8 The SMARL-based framework shows promise for scaling up to GCMs and Earth system models in the future. p. 9

--- Page 1 ---

Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning Yifei Guan, Lucas Amoudruz, Sergey Litvinov, Karan Jakhar, Rambod Mojgani, Petros Koumoutsakos and Pedram Hassanzadeh Department of Geophysical Sciences, University of Chicago

--- Page 2 ---

Extreme weather events have enormous socioeconomic impacts [1, 2], increasing the demand for the accurate estimates of their statistics, e.g., frequency, especially in a changing climate [3, 4, 5] However, due to their rarity and complex physics, high-fidelity simulations of these extreme events needs global climate models (GCMs) that are fast, thus of moderate to low resolution, while adequately representing the underlying multiscale physics of the atmosphere, and more broadly, the Earth system Practically fast-enough GCMs require sub-grid scale (SGS) closures for many physical processes [6, 7, 8] However, traditional physics-based closures have various parametric and structural errors, making them the main source of epistemic uncertainty in climate change predictions and weather forecasts

--- Page 3 ---

Artificial intelligence (AI) applications for enhancing efficiency, e.g, through direct emulation of high-resolution simulations from GCMs or other models [13, 14, 15] or rare-event sampling with AI + GCM [16], are emerging However, these applications are in their early stages, and AI models have been shown to have challenges with the most extreme weather events, particularly those rarer and stronger than events in the training set, the so-called gray swans [17, 16] Furthermore, even with these approaches, the need for high-fidelity closures for GCMs remains Recently, there has been great interest in using AI, particularly deep neural networks (DNNs), to develop better closures by learning directly from data These AI-based approaches can be broadly categorized into two groups: supervised (offline) learning and online learning [12] In supervised learning, snapshots of high-fidelity data (e. p. 1g., from direct numerical simulation (DNS)) are used as reference, then filtered and coarse-grained to extract the SGS terms [19, 20, 21] The filtered data are then used to train a DNN to learn the mapping between the filtered flow and the SGS terms such as fluxes Once trained, the DNN is coupled to a low-resolution numerical solver, the large-eddy simulation (LES) hereafter, which represents a practical GCM Studies with a variety of canonical prototypes of atmospheric and oceanic turbulence have shown the possibility of outperforming classical physics-based closures such as Smagorinsky and Leith models [22, 23, 20, 24, 25] In particular, it was found that these DNN-based closures not only reproduce the right amount of diffusion, but also capture backscattering

--- Page 4 ---

As a result, supervised-learning with DNNs faces several major shortcomings: it is data hungry (a significant bottleneck for realistic systems), requires exact SGS terms extracted from high-fidelity data (a non-trivial task), and can lead to unstable LES with the root cause(s) remaining unclear [27, 28, 12] As a result, supervised-learning with DNNs faces significant challenges in scaling up beyond prototypes and to GCMs More recently, supervised-learning with physics-constrained equation-discovery has shown promises as a data-efficient tool for developing accurate, generalizable, and interpretable closures [29] However, the LES with these closures is not stable for very low resolutions including those that could be of practical interest in atmosphere and ocean modeling

--- Page 5 ---

Online learning is emerging as a potent alternative to supervised learning, with the closure being learned by a DNN while it is directly coupled to the low-resolution numerical (LES) solver Here, the goal may not be to match spatially local flow quantities (such as the state’s trajectory or the SGS field), but rather, to match the low-order statistics of the high-fidelity simulations (e.g., DNS) or even (sparse) observations In climate modeling, these statistics could be key properties of climate variability [30] or kinetic energy spectra of turbulent flows [31, 32] Finally, online learning can account for the interactions between several closures, a common problem in climate modeling [11] and unaddressed by supervised learning We distinguish three related approaches to online learning: 1) using a differentiable LES solver/GCM that allows using backpropagation for optimization [33, 34, 35], 2) using derivative-free ensemble-based optimization methods; e. p. 1g., ensemble Kalman inversion (EKI) [36, 37, 38, 30, 39, 40], and 3) using reinforcement learning [41, 42, 43] SMARL has improved the parametric and structural uncertainties of closures for LES of homogeneous and wall-bounded turbulent flows [41, 42, 44] However, its potential for climate modeling applications has remained unexplored

--- Page 6 ---

In this paper, we introduce the first application of reinforcement learning to closure modeling of geophysical turbulence As summarized in Fig. 1, we apply SMARL to a canonical prototype: several setups of 2D turbulence that contain a broad range of scales, structures (jets and vortices), and dynamics, well representing some of the key characteristics of atmospheric and oceanic flows (see Table 1 for details) Briefly, i We train a SMARL-based closure using low-order statistics We use global states and targets to train a DNN and learn the flow-dependent coefficient cl of a physics-based closure (Leith model) by matching the enstrophy spectrum of the LES+SMARL solver with that of a very short DNS These few training samples are far too few for DNN-based supervised learning The LES solver does not need to be differentiable here

--- Page 7 ---

We test the performance of this closure by comparing the kinetic energy spectrum and vorticity probability density function (PDF) of the DNS with LES having 160 to 163, 840× spatio-temporally coarser resolution The LES+SMARL spectrum and PDF match that of the DNS for simulations that are ∼ 2000× the length of the training dataset In particular, the PDFs match at the tails, which represent the rare extreme weather events in these prototypes The LES+SMARL work at resolutions much lower than those required by the recently AI-discovered closure of [46]

--- Page 8 ---

Together, (i)-(iv) demonstrate the promises of SMARL over supervised learning and other online learning approaches We interpret the closures by analyzing the SMARL policy and demonstrate generalization to other flows. The results highlight SMARL as a potent tool for developing closures capable of capturing extremes in atmospheric/oceanic flows, opening new capabilities for effective climate modeling

--- Page 9 ---

We evaluated the proposed framework in five test cases, described in detail in Section 5.5 These benchmarks are widely used for atmospheric, oceanic, and planetary flows [57, 58, 24, 20, 20] We quantify extreme events by estimating the probability density function P(ω) of the vorticity field ω over time and space The tails of these PDFs represent rare, extreme events, i. p. 4e., significantly large ω with a small probability of occurrence [60]

--- Page 10 ---

In all cases, the RL-Leith model outperforms traditional physics-based dynamic models in terms of matching the DNS PDF Except for case 4, the RL-Leith PDF matches the DNS within uncertainties, even up at the tails of the distribution, corresponding to low probability or extreme events The lack of extreme events in the DSmag and DLeith predictions is attributed to the excessive diffusion from the positive clipping essential for stability [20] This is consistent with Fig. 3 showing the excessive interscale enstrophy transfer predicted by DSmag and DLeith for all cases

--- Page 11 ---

A main criterion for a functional eddy-viscosity LES model is the correct prediction of the total energy/enstrophy transfer between resolved and subgrid scales [49] The RL-Leith model outperforms DSmag and DLeith in predicting the true enstrophy interscale transfer, likely due to the ability of RL-Leith to capture backscattering resulting in less diffusion than DSmag and DLeith The advantage of DLeith over DSmag in terms of capturing the interscale enstrophy transfer is also reflected on a-posteriori analysis

--- Page 12 ---

To understand the superior performance of RL-Leith over classical SGS models, we compare the distribution of Leith coefficients cl obtained from RL-Leith and DLeith The first column of Fig. 4 shows the PDF of the Leith model coefficients cl for the four cases In all cases, the coefficients obtained with the DLeith model are concentrated around some positive values (rare negative values have been removed with positive clipping, which is essential for LES stability with DLeith and DSmag [22, 20]) In contrast, the Leith coefficients learned from RL-Leith spread across a much wider range of values than DLeith, and include negative values (LES with Leith-RL remains stable despite these negative viscosities)

--- Page 13 ---

This shows that the RL-Leith model captures both the forward/diffusive and backscatter inter-scale energy/enstrophy transfer Similarly to DLeith, the majority of the coefficients predicted by RL-Leith is positive, showing that the RL-Leith model is more likely to be diffusive in general However, the highest probability density of the coefficients obtained with RL-Leith occurs closer to 0, especially for Cases 3 and 4, indicating that the RL-Leith is less diffusive than DLeith

--- Page 14 ---

The generalization capability of data-driven SGS models across different flow regimes (e.g.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. SMARL-based closures can produce stable Large Eddy Simulations (LES) at resolutions up to 840 times coarser than DNS, allowing for more efficient climate modeling that captures extreme events while requiring fewer degrees of freedom.

  2. The improved system can capture both forward/diffusive and backscattering inter-scale energy/enstrophy transfer, which is crucial because the RL-Leith model learn[s] both diffusion and backscattering, unlike traditional models which are too diffusive.

  3. The framework demonstrates successful generalization to higher Reynolds numbers, meaning a closure trained on one flow regime can be directly used for simulations with 15 times higher Re without new training data.

  4. The system can accurately predict the tails of vorticity probability density functions (PDFs), which represent rare, extreme events, outperforming classical physics-based closures that underestimate extremes.

  5. The interpretability analysis using Sobol indices shows that large-scale structures containing most of the kinetic energy and enstrophy have the most contribution to predicting the closure coefficient cl.

Sources

Related papers