Long-Range Distillation: Distilling 10,000 Years of Simulated Climate into Long Timestep AI Weather Models

The gist: We introduce long-range distillation, a method that trains a long-timestep probabilistic “student” model to forecast directly at long-range using a huge synthetic training dataset generated by a short-timestep autoregressive “teacher” model.

Long-Range Distillation

Long-range distillation assumes access to a short time-step autoregressive model trained on reanalysis data capable of stable long rollouts, which here will be DLESyM (Section 2.1). This autoregressive ‘teacher’ model is used to train a distilled ‘student’ model to do long-range forecasting in a single model timestep (Figure 1). First, a large simulation is created using the teacher model, generating a diverse dataset of synthetic input-output pairs for forecasting at long time-scales (Section 2.1). This dataset contains orders of magnitude more long-range training examples than the original reanalysis dataset on which the autoregressive teacher was trained (Section 2.1).

DLESyM Climate Simulation

We use DLESyM (Cresswell-Clay et al., 2025) to generate O(10,000 years) of synthetic climate for training our long-range distilled models (Section 2.2). To parallelize data generation, we choose to generate an ensemble of simulations initialized on different dates rather than running a single long-running simulation (Section 2.2). We initialized DLESyM simulations on 200 dates equally spaced between 2008-01-01 and 2016-12-31, running each simulation for 90 years, yielding 18,000 years of simulated climate (Section 2.2). We used the version of DLESyM distributed through the NVIDIA Earth2Studio package (v0.8.0), with each ensemble member using a randomly selected checkpoint from the four checkpoints provided through the NGC catalog (NVIDIA, 2025c) to boost ensemble variability (Section 2.2).

Distilled Student Model Architecture: HEALPix Conditional Diffusion

We implement our distilled student model as a conditional diffusion model, enabling probabilistic forecasting (Section 2.3). As input conditioning, c, we provide a short history of daily average weather states for the 4 preceding days (Section 2.3). We then seek to model the conditional distribution, p(xc), of future global weather states, x, at some long-range target lead time (Section 2.3). The neural network architecture we use to perform global conditional diffusion on the HEALPix64 grid follows that used in the recently-developed global foundation model, cBottle (Brenowitz, Ge, et al., 2025) (Section 2.3).

Calibrating Distilled Model Forecast with Classifier-Free Guidance

We exploit classifier-free guidance (Ho & Salimans, 2022) to control the ensemble spread of our long timestep conditional diffusion models (Section 2.4). Concretely, the conditional score, ∇xlog(p(x(t)c)), used during sampling, is replaced with a linear combination of the conditional and unconditional scores, ∇xlog (p(x(t)) + w [∇xlog(p(x(t)c)) − ∇xlog(p(x)(t))], (3) (Section 2.4). When w = 0, the model generates samples unconditionally (Section 2.4). When w = 1, the model generates samples conditionally with no guidance (Section 2.4).

Perfect-Model Experiment

We evaluate the ability of our long-range distilled models to produce skillful ensemble forecasts across a range of lead times in the controlled setting of a perfectmodel experiment (Section 3.1). We apply the following experiment protocol: 1. Perform a single, multi-year deterministic simulation with DLESyM referred to hereafter as the Nature Run (Section 3.1). 2. Select a series of starting-point dates from the Nature Run (Section 3.1). Apply a different set of initial condition perturbations to each of these starting points to create the suite of imperfect initial conditions for each ensemble forecast (Section 3.1).

Distilled Models Are Skillful Across Lead Times

We first assess medium-range (7-day) forecast skill under a realistic setting where both models are initialized from an imperfect estimate of the atmospheric state that contains non-negligible error (Figure 6a) (Section 3.2.3). In contrast, DLESyM10K shows much greater robustness to this initialization error: its day 7 Z500 ensemble RMSE increases by only 25 m2/s2 due to the addition of initial condition error (comparing panels a & b) (Section 3.2.3).

Real-World S2S Forecasting

Ultimately, our goal is to apply DLESyM10K to real-world long-range forecasting taking ERA5 reanalysis as initial condition (Section 4.1.1). To benchmark DLESyM10K against an operational baseline, we compare the week4 2 m temperature forecast skill to that of the ECMWF ensemble forecast system from 2018-2022 (Figure 9) (Section 4.1.3). DLESyM10K outperforms the long-timestep model trained from scratch on ERA5, further highlighting the value of pre-training on a large DLESyM simulation (Section 4.2).

Conclusions

In this study, we proposed a novel approach to long-range weather modeling (Section 5). Rather than relying on ensemble autoregressive rollouts, we repurpose a reanalysis-trained autoregressive model as a generator of large volumes of synthetic data to train a separate distilled model that predicts long-range forecasts in a single model timestep (Section 5). We further demonstrate that classifier-free guidance, commonly used in image generation, offers a simple recipe for controlling ensemble spread in conditional diffusion models (Section 5). Training with such long timesteps (equivalent to up to 392 autoregressive steps) without overfitting required a massive synthetic dataset, and we observed that S2S forecast skill scaled with the number of synthetic years, even when the dataset far exceeded the size of ERA5 (Section 5). Finally, finetuning on ERA5 reanalysis allowed our distilled models to achieve real-world forecast skill comparable to the operational ECMWF S2S ensemble forecast system (Section 5).

Table B1

Parameter Value

Model timestep 6 hours

Output frequency saved Daily averages

Data volume 7 TB

Number of stable ensemble members 162

Initialization date range 2008-01-01 to 2016-12-31

Table B3

Parameter Value

σmin 0.002

σmax 200

Training σ distribution Log uniform

Appendix A Difference in Scores Significance Testing

Throughout this manuscript where we plot maps of differences in scores between one forecasting model and another, we indicate statistical significance using a paired test on the difference in similar to that used in Mardani et al. (2025) (Appendix A). The purpose of this test is to test whether observed differences between the time mean scores of two models are statistically significant given the large day-to-day variability of the atmosphere, and hence of the scores (Appendix A).

1. This system can produce long-range forecasts in a single timestep by distilling knowledge from a massive synthetic dataset, as described by long-range distillation, replacing hundreds of autoregressive steps with a single timestep.

2. The distilled model achieves S2S forecast skill comparable to the ECMWF ensemble forecast after ERA5 fine-tuning, demonstrating that AI-generated synthetic training data can be used to scale long-range forecast skill.

3. The system can achieve well-calibrated ensemble forecasts by using classifier-free guidance (CFG) during sampling, which is described as a simple recipe for controlling ensemble spread in conditional diffusion models, allowing the model to explore spread-skill space.

4. The model can be fine-tuned on ERA5 reanalysis data to bridge the domain shift between DLESyM simulations and real-world conditions, ensuring that the large-scale mean and seasonal cycle of our target distribution matches that seen during training.

5. The system can outperform deterministic climatology for S2S and seasonal lead times, achieving global mean skill within 3-7% of an ensemble forecast using the autoregressive teacher model.

Accurate long-range weather forecasting remains a major challenge for AI models, both because errors accumulate over autoregressive rollouts and because reanalysis training datasets provide limited sampling of the slow modes of variability underpinning predictability. Most AI and all numerical weather prediction models are autoregressive, repeatedly applying short-lead forecasts to reach subseasonal-to-seasonal (S2S) or seasonal lead times, often causing instability and miscalibration. Long-timestep probabilistic models that generate long-range forecasts in a single step offer an alternative, but training on the 40-year reanalysis record leads to overfitting, suggesting orders of magnitude more data are required. We introduce long-range distillation, whereby a long-timestep probabilistic ``student'' model is trained to forecast directly at long-range using a huge synthetic dataset generated by an autoregressive ``teacher'' model. Using the Deep Learning Earth System Model (DLESyM) as teacher, we generate over 10,000 years of simulated climate to train student models across a range of timescales. In perfect-model experiments, the distilled models outperform climatology and approach their teacher's skill while replacing hundreds of autoregressive steps with a single timestep. In the real world, both a diffusion model student fine-tuned on ERA5 and a linear inverse model student with mixed DLESyM-ERA5 training achieve S2S forecast skill competitive with the ECMWF IFS ensemble, with synthetic pre-training providing improvements beyond ERA5-only training. This demonstrates the benefit of synthetic pre-training for both deep learning and classical data-driven methods, and represents the first demonstration that real-world skill can be improved by training on AI-generated synthetic data in volumes far exceeding the observational record.