Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning

summary

The gist

Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical.

In short

This research uses Scientific Multi-Agent Reinforcement Learning (SMARL) to create data-efficient closures for atmospheric and oceanic turbulence models. By rewarding agents based on matching the enstrophy spectrum of high-fidelity simulations, the learned RL-Leith model outperforms traditional physics models in capturing rare extreme weather events and accurately representing both diffusion and backscattering.

Key concepts

Subgrid-scale (SGS) closures
These are mathematical approximations used in large-eddy simulations (LES) to model physical processes occurring at scales smaller than the grid resolution. Traditional physics-based closures often fail to accurately capture complex energy transfers between resolved and subgrid scales.
Enstrophy Spectrum
This is a measure of the kinetic energy distribution across different spatial scales in a turbulent flow. The SMARL framework uses this spectrum as the primary reward signal, training agents to learn how to correctly model these scale-dependent interactions for accurate simulation results.
Reinforcement Learning (RL) for Closures
SMARL is an online learning approach where agents iteratively refine a closure model by interacting with a numerical solver. The system learns the flow-dependent coefficient of a physics model by matching its output spectrum to that of high-fidelity simulations, enabling it to capture extremes without needing massive training datasets.

Terminology used across episodes

This episode discusses

The paper

Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning · Read on arXiv

Yifei Guan, Lucas Amoudruz, Sergey Litvinov, Karan Jakhar, Rambod Mojgani, ‡Petros Koumoutsakos‡and Pedram Hassanzadeh

Department of Geophysical Sciences, University of Chicago · Department of Mechanical Engineering, Union College · Computational Science and Engineering Laboratory, Harvard University · Department of Mechanical Engineering, Rice University · Committee on Computational and Applied Mathematics, University of Chicago

DOI: 10.1016/j.jocs.2026.103022

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning".

Mira: Here's the gist of the paper. Accurate subgrid-scale closures are essential for weather/climate models, where predicting extreme events is critical.

Kai: This paper introduces scientific multi-agent reinforcement learning (SMARL) to develop closures for canonical prototypes of atmospheric/oceanic turbulence using only the enstrophy spectrum as a reward.

Mira: These online-learned closures enable stable simulations with up to five orders of magnitude fewer degrees of freedom that reproduce high-fidelity simulation statistics and capturing in particular extremes.

Kai: Pruning redundant reasoning tokens The research addresses the challenges in predicting extreme weather events by overcoming the limitations of traditional closures and existing AI approaches.

Title and authors: Kai: We're moving into the title and authors of this paper, "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning". I want to explain what that actually means for us.

Mira: The title tells us they are using reinforcement learning on multiscale simulations to predict extreme events in geophysical turbulence <ref:2603.03351#pg1>.

Kai: It's about taking these complex atmospheric and oceanic flows and building something that can handle the really rare, intense weather events that we need to model accurately <ref:2603.03351#pg1>.

Lev: From an engineering standpoint, what they're doing is developing a closure—that’s essentially a mathematical approximation for the physics happening at scales smaller than what you can resolve directly in a simulation <ref:2603.03351#pg2>.

Kai: They are training this closure using reinforcement learning, which is an online learning method where the agent learns by interacting directly with the simulation environment <ref:2603.03351#pg5>.

Mira: The authors are Guan, Amoudruz, Litvinov, Jakhar, Mojgani, Koumoutsakos and Hassanzadeh <ref:2603.03351#pg1>.

Lev: It's notable that they are focusing on canonical prototypes of atmospheric and oceanic turbulence to test this method <ref:2603.03351#pg3>.

The paper's summary: Kai: Now, let’s look at the core summary of this paper, "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning". They explain how this SMARL framework works in simpler terms.

Mira: They train a DNN to learn the flow-dependent coefficient cl for a physics-based closure, specifically the Leith model <ref:2603.03351#pg3>.

Lev: The training signal is defined by matching the enstrophy spectrum of their LES plus SMARL solver with that of a very short direct numerical simulation or DNS <ref:2603.03351#pg2>.

Kai: The instantaneous reward is chosen to be large when that enstrophy spectrum is close to the reference, which helps guide the learning process <ref:2603.03351#pg1>.

Mira: They state that this setup allows them to test performance by comparing the kinetic energy spectrum and vorticity probability density function, or PDF, of the DNS against an LES solver with a resolution that is one hundred sixty to one hundred sixty-three times coarser <ref:2603.03351#pg3>.

Lev: They show that this LES plus SMARL spectrum and PDF match the DNS for simulations that are about two thousand times the length of their training dataset <ref:2603.03351#pg2>.

The paper's improvements: Kai: So, what are the actual improvements this paper suggests over existing methods? They’re focusing on how this SMARL-based closure actually performs better.

Mira: The main improvement is that the RL-Leith model outperforms traditional physics-based dynamic models when matching the DNS PDF, even at those tails corresponding to rare extreme weather events <ref:2603.03351#pg1>.

Lev: It seems they capture both diffusion and backscattering, which is shown by the distribution of Leith coefficients cl spreading across a wider range than DLeith <ref:2603.03351#pg1>, which suggests less excessive diffusion compared to other models.

Kai: They also show generalization capability, meaning the RL-Leith closure trained on one flow can be used directly for a flow with fifteen times higher Reynolds number without needing new data <ref:2603.03351#pg1>.

Mira: And their interpretability analysis using Sobol indices shows that large-scale structures containing most of the kinetic energy and enstrophy have the most contribution to predicting the closure coefficient cl <ref:2603.03351#pg1>.

Conclusion: Kai: To wrap up, we're looking at the conclusion of "Prediction of Extreme Events in Multiscale Simulations of Geophysical Turbulence using Reinforcement Learning". They summarize what this means for climate modeling and weather forecasts.

Mira: Essentially, they demonstrate a data-efficient framework that learns LES closures for geophysical turbulence <ref:2603.03351#pg1>.

Lev: It’s superior to existing AI-based closure modeling approaches because it works with sparse training data and low LES resolutions that are usually inaccessible to equation discovery methods <ref:2603.03351#pg4>.

Kai: The results show that simulations with SMARL-based closures significantly outperform those with classical physics-based closures in capturing extreme events, suggesting both diffusion and backscattering are accurately represented <ref:2603.03351#pg1>.

Mira: This framework is shown to be stable and can run thoughts of times longer than the lengths of the training set and training horizon <ref:2603.03351#pg1>.

Lev: For my work in error correction, it’s important that this stability holds, because running real hardware simulations requires a very reliable closure to manage the overhead effectively <ref:2603.03351#pg4>.

Kai: It really shows promise for scaling up these methods to global climate models and Earth system models in the future <ref:2603.03351#pg1>.

Mira: It’s a way forward for handling those hard, rare events that current models struggle with <ref:2603.03351#pg1>.

More episodes

← Home