Unmasking Compensating Errors in Numerical Weather Models: A Process-Additive Neural Network Approach
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.
Kai: We're looking at the paper: "Unmasking Compensating Errors in Numerical Weather Models".
Mira: Here's the gist of the paper. This work introduces a Process Additive Neural Network (PANN) framework to disentangle compensating biases in atmospheric models where errors in different physical processes can offset each other and obscure their origins.
Kai: The aim is not to replace human interpretation of physical processes, but to extend it to regimes where model complexity, process interaction, and compensating errors make direct diagnosis difficult.
Mira: Pruning redundant reasoning tokens The paper applies a branched neural network architecture to identify and analyse compensating errors in atmospheric models.
Kai: In many atmospheric systems, the total observed behaviour results from multiple distinct physical processes acting in superposition.
Title and authors: Kai: So we're looking at a paper called "Unmasking Compensating Errors in Numerical Weather Models: A Process-Additive Neural Network Approach". It sounds like they are tackling a problem where different physical processes in these models can cancel each other out, making it really hard to figure out where the actual errors are coming from.
Mira: Exactly. It’s about those compensating biases in atmospheric models, where an error in one process just offsets the error from another process, obscuring what the real issue is >
Lev: It's interesting because standard methods often struggle when you have this high autocorrelation between different errors >
Kai: Right. So what they propose is a new framework called the Process Additive Neural Network, or PANN. They are trying to disentangle those biases by treating each physical process as its own separate branch in the network >
Mira: That's the core idea. They use this architecture to attribute errors in specific humidity tendency, which we call dQ, to individual physical processes >
Lev: And they train it by having each branch only see the tendency of one specific process, and then they sum up those branches to try and match a known total error that we already know how it breaks down into cloud and convection contributions >
Kai: It’s like trying to take a complicated total measurement, but instead of just looking at the final number, you break it down into the individual ingredients before you even look at the whole thing >
Mira: That's right. They do this in two stages. First, they use a synthetic error where they know exactly how much cloud and convection are contributing to the total error, and they check if PANN can recover those known contributions separately >
Lev: And then in the second stage, which is harder because we don't know what the per-process truth is for real data, they apply this same structure to a real discrepancy between two models called OpenIFS and C18 >
Kai: So when they run it on those real discrepancies, they aren't just getting a single total error number anymore; they’re getting process-level error attributions >
Mira: They say these branch outputs give them process-level error attributions, and when you average those branch outputs, you get estimates of the process-level bias >
Lev: That’s important because it means they can diagnose parameterization deficiencies during tuning and calibration cycles, which is where things usually get tricky >
Kai: And they suggest that this could guide tuning towards physically consistent fixes instead of just trying to reduce some net bias number >
Mira: They also point out that the architecture itself removes connections that physics doesn't allow, so you can’t mix convection with cloud in the hidden layers, which helps keep things structured >
Title and authors: Lev: That parallel structure is key to keeping the model constrained and interpretable, even when dealing with complex interactions >
Kai: So they use leave-one-covariate-out ordering during training to estimate the fixed weights for each branch, which helps them get those individual process contributions right >
Mira: The results on the synthetic error were pretty strong; they said attribution of error variance was nearly exact, but the attribution of the mean error, or bias, was less identifiable >
Lev: That makes sense. You can figure out how much noise is coming from a specific component very well, but figuring out the actual systematic offset between processes is harder >
Kai: And they found that the error variance was largest in the lower troposphere and got enhanced in the equatorial convection belt when looking at real SCM discrepancies >
Mira: That’s a spatial pattern. It suggests where we should focus our attention when we look at model errors, which is valuable information >
Lev: But they also noted a limitation; the method is only demonstrated for two branches and for specific humidity, but they say that a humidity attribution can give useful constraints if you try to extend it to temperature >
Kai: That means it’s not a complete solution yet, but it gives us hypotheses on what's happening in the system >
Mira: And they also suggest that this approach can help reveal patterns arising from interacting processes without needing to simplify the whole system first >
Lev: They conclude that better-targeted diagnostics could make the iterative calibration loop more efficient, which is a practical thing for modelers >
Kai: So, in summary, the Process Additive Neural Network paper introduces this framework to break down compensating errors in atmospheric models by enforcing a parallel structure where each branch handles one physical process >
Mira: It gives us process-level error attributions and estimates of process-level bias when averaged, which helps diagnose tuning issues during calibration cycles >
Lev: The method's strength is turning an additive physical constraint into a mechanism for attribution, while removing physically impossible connections a priori >
Kai: They show it works on synthetic errors with known contributions and then test it on real data discrepancies like the OpenIFS to C18 difference >
Mira: The results show that error variance is largest in the lower troposphere and enhanced in the equatorial convection belt, which points to specific areas of interest >
Lev: While they show promise for humidity, they acknowledge that extending this approach to temperature will provide more constraints on interactions between processes >
Kai: So it’s a tool for getting better-targeted diagnostics when dealing with model complexity and interacting physical processes that obscure the error origins >
The paper's summary: Kai: So, to recap, this paper is introducing something called a Process Additive Neural Network, or PANN, which is designed to untangle errors in atmospheric models when different physical processes are canceling each other out.
Mira: Right. They build this network so that every physical process gets its own separate branch in the structure >
Lev: The training setup is really clever because they force those branches to learn only their own process tendency, and then they sum them up to match a known total error constraint >
Kai: It’s like taking a big total measurement, but instead of just looking at the final number, you break it down into the individual ingredients before you even look at the whole thing >
Mira: Exactly. They call this process-level error attribution because when you sum those branches up, you get estimates of how much each physical process is contributing to the total discrepancy >
Lev: The architecture itself has a structural constraint built in: it prevents convection from mixing with cloud contributions in the hidden layers, which keeps things physically separated >
Kai: And they use leave-one-covariate-out ordering during training, which is how they figure out the weights for each branch, making sure those individual process contributions are estimated correctly >
Mira: The big takeaway is that this shifts how we look at model tuning; instead of just chasing a lower total error number, you can guide the tuning towards fixes that are physically consistent across those different processes >
Lev: And they show it works on synthetic errors where they know the true cloud and convection contributions, which proves the mechanism is sound mathematically >
Kai: But when you test it on real discrepancies, like comparing two different weather models, these branch contributions become a diagnosis rather than just a verified truth from first principles >
Mira: They found that this error variance is really high in the lower atmosphere and gets amplified in the equatorial convection belt when they look at real data differences >
Lev: That spatial pattern suggests where model errors are concentrated, which helps pinpoint exactly where we need to focus our effort for revisions >
Kai: It’s a process-aware way to diagnose those compensating errors, and while they only demonstrated it for specific humidity right now, they suggest extending it to temperature could give us better constraints on interactions between processes >
Mira: The authors say this approach lets you reveal patterns from interacting processes without having to simplify the whole system first, which is a big deal for understanding complexity >
Lev: They conclude that using these targeted diagnostics could make the whole iterative calibration loop much more efficient, which is what we need when we're trying to fix these models >
Kai: So it’s about moving past just seeing a big error number and instead getting process-level details on where that error actually originates >
Mira: It gives us a clearer hypothesis for tuning, even if the method isn't ready for every single atmospheric variable yet >
Lev: The real constraint is that this method only works well when you have this clear additive physical constraint to start with, and it relies on you having some idea of what those processes are >
The paper's improvements: Tom: So, to recap, this paper suggests several ways to make the PANN framework even more useful for modelers >
Kai: They’re talking about turning that additive physical constraint into a better diagnostic tool by removing physically impossible connections right at the start >
Mira: That means they prune away any hidden layers where convection and cloud can mix, which keeps the interpretation much cleaner than what you'd get from a standard neural network >
Lev: The next thing is how they use training data to get those weights; they mentioned using leave-one-covariate-out ordering to estimate the fixed branch weights for each process >
Kai: That helps them nail down those individual process contributions better, which is key when you're trying to figure out what a specific physical term is actually doing >
Mira: They also point out that this design keeps the model simple enough that standard tools like SHAP or LOCO can still work with little change, which means it’s practical for real work >
Lev: They also explored using the error variance and mean-error patterns to quantify uncertainty in atmospheric components, which helps you see spatial and temporal mismatches in where active cloud points are occurring >
Kai: That connects back to my work on qubits; you need that kind of fidelity to trust the measurements, right? You can't get a good picture if the underlying physical assumption is messy >
Mira: And they mentioned that even though it only tested humidity right now, extending this method to temperature could provide useful constraints and hypotheses when you try to analyze those other variables >
Lev: So for someone running this on real hardware, the overhead lands in how much you need to calibrate against these multiple physical processes, not just one bulk metric >
Kai: The authors are also suggesting that PANN can diagnose model errors against actual observations rather than just comparing two different models to each other >
Mira: That’s a practical point; it could help guide the iterative calibration loop towards physically consistent fixes instead of just trying to reduce a net bias number >
Lev: Ultimately, they think better-targeted diagnostics could make the entire calibration process much more efficient for modelers >
Conclusion: Kai: So, to wrap up, this paper introduces the Process Additive Neural Network framework for disentangling compensating errors in numerical weather models >
Mira: It’s about using a parallel structure so each physical process gets its own branch and you can attribute those total model errors to specific processes >
Lev: The main thing is turning that additive physical constraint into an actual attribution mechanism, which lets us diagnose tuning issues during calibration cycles >
Kai: And they showed that this approach works on real discrepancies between models, even if it’s only for specific humidity right now >
Mira: It gives us a process-aware way to look at model errors, and it suggests that better targeted diagnostics could make the iterative calibration loop much more efficient for modelers >
Lev: I just want to say that while the concept is strong, the method doesn't yet run on real hardware because of how much overhead you'd need to manage for all those branches >
Kai: That’s true. The fidelity we get from these complex architectures is still something we have to work hard on when we talk about cryogenics and measurement precision >
Mira: But the underlying theory is what’s interesting here; separating the total error into process contributions is a solid way to hypothesize where model deficiencies lie >
Lev: Yeah, it gives you a better hypothesis before you even start running simulations that take weeks on supercomputers >
Kai: So they show us how to get better-targeted diagnostics when dealing with model complexity and interacting physical processes that obscure the error origins through this Process Additive Neural Network approach >
Environmental Change Institute, University of Oxford · Atmospheric, Oceanic and Planetary Physics, University of Oxford · Department of Applied Mathematics and Theoretical Physics, University of Cambridge
physics.ao-ph
Submitted: 2026-10-14
Updated: 2026-11-04
Comments: Accepted at the NeurIPS 2026 Workshop on Tackling Climate Change with Machine Learning (TCCML). Corresponding author: yuiko.ichikawa@ouce.ox.ac.uk
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 90/100
The gist: This work introduces a Process Additive Neural Network (PANN) framework to disentangle compensating biases in atmospheric models where errors in different physical processes can offset each other and…
Key concepts
- Compensating Biases
- These occur when errors in one physical process cancel out the errors from another. In complex models, this makes it hard to tell if a total error is due to a specific physics issue or just an offset between competing processes.
- Process Additive Neural Network (PANN)
- This is a neural network structure where each branch handles one distinct physical process. The model learns by summing the outputs of these branches, allowing it to decompose a total observed error into contributions from individual physical components.
- Constrained Learning Architecture
- The architecture prevents different physical processes from mixing their information in the hidden layers. This structural constraint ensures that each branch focuses only on its assigned process, leading to clearer and more physically structured error attribution.
Terminology
Summary
This work introduces a Process Additive Neural Network (PANN) framework to disentangle compensating biases in atmospheric models where errors in different physical processes can offset each other and obscure their origins. p. 0 ichikawa@ouce. uk
Abstract
This work introduces a Process Additive Neural Network (PANN) framework to
disentangle compensating biases in atmospheric models, where errors in different
physical processes can offset each other and obscure their origins. Initial experiments focus on cloud-related
processes in OpenIFS SCM case studies, where cloud and convection tendencies
are strongly coupled and compensation is difficult to diagnose using standard approaches. The aim is not to replace human interpretation of physical processes, but
to extend it to regimes where model complexity, process interaction, and compensating errors make direct diagnosis difficult.Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper applies a branched neural network architecture to identify and analyse compensating errors in atmospheric models. In many atmospheric systems, the total observed behaviour results from multiple distinct physical processes acting in superposition. p. 0 1 Introduction
This work applies a branched neural network architecture to identify and analyse compensating errors
in atmospheric models. In many atmospheric systems, the total observed behaviour results from multiple distinct physical
processes acting in superposition. While the combined outcome is observable, the contributions of
individual processes are often hidden. This is particularly problematic when theoretical understanding of some processes is incomplete: errors in different processes can offset one another, making it
difficult to attribute errors correctly, leading to a failure to understand the true nature of the processes. Unfortunately, they are tightly coupled, and their humidity tendencies are strongly anticorrelated, as
convective detrainment feeds the cloud scheme [6]. Standard post-hoc interpretability methods, such
as Shapley Additive Explanations (SHAP), input-gradient saliency maps [7], or partial radiative perturbation [8], struggle in the presence of such high autocorrelation, leading to high variance in attributions and potential “disagreement” between explanation methods [9–14].Where the paper says it
Where the paper says it
Where the paper says it
How the training signal is built
The architecture follows the Neural Additive Model [17], with one branch per process and no cross-branch connections in the hidden layers. PANN is trained to attribute the error eˆ in specific-humidity tendency dQ to individual physical processes. p. 1 The details of each blocks are described in the Supplement. PANN
is trained to attribute the error eˆ in specific-humidity tendency dQ to individual physical processes. PANN
is trained to attribute the error eˆ in specific-humidity tendency dQ to individual physical processes. Each branch receives only the tendency of one process: dQval. Each branch receives only the tendency of one process: dQval. Branch outputs: eˆval are summed,
and that sum is trained to match the discrepancy between a verification error etruth (with no process
breakdown) and the predicted error eˆ. Branch outputs: eˆval are summed,
and that sum is trained to match the discrepancy between a verification error etruth (with no process
breakdown) and the predicted error eˆ. Individual process errors are never observed; they are inferred
as eˆval from this additive constraint. Two stages are used. Stage 1 plants a synthetic total error with known cloud and convection contributions and tests whether PANN recovers the prescribed branch contributions.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The method is designed to turn an additive physical constraint into an attribution mechanism by decomposing one observed total into process contributions. The design has three practical roles: First, it turns an additive physical constraint into an attribution mechanism. p. 1 This design has three practical roles. First, it turns an additive physical constraint into an attribution
mechanism: one observed total is decomposed into process contributions. First, it turns an additive physical constraint into an attribution
mechanism: one observed total is decomposed into process contributions. Second, connections that
the physics does not allow are removed a priori, so the hidden layers cannot mix convection with
cloud. Second, connections that
the physics does not allow are removed a priori, so the hidden layers cannot mix convection with
cloud. Third, the model remains simple enough that standard XAI tools (e. Third, the model remains simple enough that standard XAI tools (e. SHAP or LOCO) apply
with little change; because the branches do not mix, those tools need only consider combinations of a
few process heads rather than of all hidden units. First, a short dominance probe on the initial cross-validation fold estimates fixed branch weights. Under leave-one-covariate-out (LOCO) ordering, the importance of branch p is the drop in validation
R2 when that branch is removed:
Ip = R
2
(all branches) − R
2
(all branches \ p).Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Estimated error attributions
The study evaluates the PANN on two datasets: a synthetic target and the dQ difference between ECMWF OpenIFS and C18. In the synthetic experiment, PANN successfully predicts the total error variance and attributes it to individual branches. p. 2 Branch error variances are largest
in the mid-troposphere, where convection and cloud errors are simultaneously active, providing the
strongest test of separation. PANN successfully predicts the total error variance and attributes it to the
individual branches (Fig. In the synthetic experiment, where the truth
is prescribed, PANN recovers the attribution well. For the C18 experiment, however, the branch
contributions should be interpreted as a physically constrained diagnosis rather than as verified truth. 2 Error against the C18 cloud-permitting reference
Having validated PANN on synthetic targets, we apply it to the dQ difference between OpenIFS
and C18 (Fig. Panel (a) shows that error variance is largest in the lower troposphere (surface–
∼700 hPa) and enhanced in the equatorial convection belt, where moisture variability and cloudconvection activity are highest.Where the paper says it
Where the paper says it
Where the paper says it
Impact
PANN offers a process-aware way to diagnose compensating errors in weather and climate models. Parameterization workflows rely heavily on calibration against bulk metrics, which need not expose competing biases between schemes. p. 3 5 Impact
PANN offers a process-aware way to diagnose compensating errors in weather and climate models. Parameterization workflows rely heavily on calibration against bulk metrics, which need not expose
competing biases between schemes [2, 3]; PANN can diagnose when such compensation occurs and
indicate which parameterizations—and, in regional applications, which locations—should be prioritised for revision. Parameterization workflows rely heavily on calibration against bulk metrics, which need not expose
competing biases between schemes [2, 3]; PANN can diagnose when such compensation occurs and
indicate which parameterizations—and, in regional applications, which locations—should be prioritised for revision. By separating a total tendency discrepancy into process-level contributions, it can
guide tuning towards physically consistent fixes rather than net-bias reduction alone. Instead, it extends
process-based interpretation to problems in which several parameterizations act simultaneously and
their errors compensate. This capability will become increasingly important as atmospheric models
grow more complex. Nevertheless, cloud-related processes remain among the largest sources of uncertainty. A humidity attribution
can consequently provide useful constraints and hypotheses when extending the method to temperature, particularly if the cloud-error representation is held fixed or otherwise informed by the humidity
analysis (e. As computational resources increase, models may represent more of this
complexity explicitly; diagnosing the resulting interactions will remain challenging. Methods such as
PANN can help reveal patterns arising from interacting processes without requiring those patterns to
be reduced to a simpler system first. A valuable next step would therefore be to apply the same workflow to field
observations, for example from TWPICE [19]. This would test whether PANN can diagnose model
errors against observations, rather than only differences between two models. is a major bottleneck in parameterization development [22], better-targeted diagnostics could make
that loop more efficient. In the longer term, this may improve our understanding of the atmospheric
processes represented by these schemes and support more reliable weather and climate simulations.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Process Additive Neural Network (PANN) framework for error disentanglement. This system can identify
compensating biases in atmospheric models, where errors in different physical processes can offset each other and obscure their origins.
-
Attribution of process-level error variances and mean errors from model discrepancies. The system provides
process-level error attributions
which are used to estimateprocess-level bias
by summing the branch outputs. -
Constrained learning architecture for physically structured interpretability in complex systems. The PANN enforces a structure where
each branch corresponds to a distinct physical process and learns its own hidden representation,
preventing processes from mixing in hidden layers. -
Diagnosis of parameterization deficiencies during model tuning and calibration cycles. PANN can
diagnose when such compensation occurs and indicate which parameterizations—and, in regional applications, which locations—should be prioritised for revision.
-
Enhanced uncertainty quantification for atmospheric model components through variance analysis. By analyzing the contrast between error variance and mean-error patterns, the system can reveal
spatial and temporal mismatches in the occurrence of active convective/cloud grid points.
-
Robust explanation methods resilient to high feature correlation. PANN addresses issues where
strong correlations among input features can destabilize individual-feature attributions
by enforcing a parallel structure.
Abstract
This work introduces a Process Additive Neural Network (PANN) framework to disentangle compensating biases in atmospheric models, where errors in different physical processes can offset each other and obscure their origins. Using a single-column model, each PANN branch represents the tendency error associated with a separate physical process, and the summed branch outputs approximate the total error relative to a reference tendency. Once trained successfully, the branch outputs are expected to provide process-level error attributions and, when averaged, estimates of process-level bias. Initial experiments focus on cloud-related processes in OpenIFS SCM case studies, where cloud and convection tendencies are strongly coupled and compensation is difficult to diagnose using standard approaches. The aim is not to replace human interpretation of physical processes, but to extend it to regimes where model complexity, process interaction, and compensating errors make direct diagnosis difficult. By providing a physically structured form of machine-learning interpretability, PANN can help connect complex model behaviour with process-level explanations. This work supports both improved understanding of compensating errors and the development of more physically consistent weather and climate models.
Sources
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Feature Importance: A Closer Look at Shapley Values and LOCO
- Comparing Explanation Methods for Traditional Machine Learning Models Part 2: Quantifying Model Explainability Faithfulness and Improvements with Dimensionality Reduction
- Neural Additive Models: Interpretable Machine Learning with Neural Nets
- The many Shapley values for model explanation
Related papers
- Anthropogenic Heat in Urban Climate Systems: Forcing, Sensitivity, and Feedback
- NORi: An ML-Augmented Ocean Boundary Layer Parameterization
- A Checklist to assess the energy and carbon impacts of ML/AI applications in Earth System Modeling
- Skillful Thunderstorm Forecasts Learned Directly from Observations
- Wind vector retrieval from miniaturized wave-enabled sea-surface drifters
- Stochastic coupling of climate variables and ice volume over the Late Pleistocene glacial cycles