Correlated Systematic Uncertainties and Errors-on-Errors in Measurement Combinations: Methodology and Application to the 7-8 TeV ATLAS-CMS Top Quark Mass Combination
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.
Kai: We're looking at the paper: "Correlated Systematic Uncertainties and Errors-on-Errors in Measurement Combinations".
Mira: Here's the gist of the paper. The Gamma Variance Model (GVM) provides a statistical framework to handle uncertainties in systematic errors—informally called errors-on-errors—which is particularly useful when combining results from several measurements, such as the 7–8 TeV ATLAS-CMS top quark mass combination.
Kai: The paper introduces the Gamma Variance Model (GVM) as a statistical model incorporating uncertainties in the assignment of systematic errors.
Mira: This model allows for the treatment of both types of errors-on-errors, including those arising from ambiguities in defining a "1-sigma" error and those stemming from the estimation procedures themselves.
Kai: The GVM extends the likelihood function by treating the systematic variance as an adjustable parameter, denoted as vs, which is modeled as a gamma-distributed random variable.
Title and authors: Kai: So we're looking at this paper today, "Correlated Systematic Uncertainties and Errors-on-Errors in Measurement Combinations: Methodology and Application to the seven-eight TeV ATLAS-CMS Top Quark Mass Combination" <ref:2407.05322#pg1,Application to the 7-8 TeV ATLAS-CMS Top Quark Mass Combination>. Mira, what's the main story here?
Mira: It’s essentially about how we handle those tricky errors that aren't just simple plus or minus one. It tackles how we combine measurements when those systematic uncertainties are actually correlated in a more complicated way.
Lev: So if I had to put it simply, the paper is building a statistical framework, the Gamma Variance Model, to deal with uncertainties in assigning those systematic errors.
Kai: Right, so it’s about making sure that when we combine results from different places—like ATLAS and CMS—we don't just treat all those systematic errors as perfectly known constants.
Mira: Exactly. It introduces this model where the variance itself becomes a random variable, modeled as a gamma distribution, which lets us adjust how much we trust each systematic assignment.
The paper's summary: Kai: So what does the paper actually do? It sets up this Gamma Variance Model to treat those errors-on-errors, which are the uncertainties in error estimation itself.
Mira: They show a general relationship between two ways of doing things—the Best Linear Unbiased Estimator method and what they call the nuisance-parameter method.
Lev: And they generalize that nuisance-parameter approach so it can handle these errors-on-errors, which is crucial because standard methods often get stuck when correlations are non-trivial.
Kai: They then derive analytical formulas for figuring out the central values of things, the confidence intervals, and even the goodness-of-fit under these new conditions.
Mira: The application they use is to this specific seven–eight TeV ATLAS-CMS top quark mass combination, looking at how those uncertain systematics affect the final result <ref:2407.05322#pg1,7–8 TeV ATLAS-CMS top quark mass combination>.
Lev: And they find something interesting about that combination: it shows that the central value is pretty robust to having those systematic errors be uncertain one at a time.
The paper's improvements: Kai: So what are the specific improvements they suggest over how we usually do these kinds of combinations?
Mira: The big improvement is incorporating those errors-on-errors directly into the model, which allows the central value to become less sensitive to discrepancies between different input data values.
Lev: They also show that when you add a fictitious outlier measurement, within the Gamma Variance Model framework, that outlier gets assigned lesser weight as the error-on-error parameter increases.
Kai: That means if we have a weird data point, it doesn't totally wreck our result; it just gets down-weighted based on how uncertain we are about our own error assignments.
Mira: And they also show that the size of the confidence interval itself becomes sensitive to the internal consistency of your input data, meaning it inflates when measurements don't line up perfectly.
Conclusion: Kai: So wrapping this up, what's the main point about this work on "Correlated Systematic Uncertainties and Errors-on-Errors in Measurement Combinations: Methodology and Application to the seven-eight TeV ATLAS-CMS Top Quark Mass Combination" <ref:2407.05322#pg1,Application to the 7-8 TeV ATLAS-CMS Top Quark Mass Combination>?
Mira: The core message is that for combinations like this top quark mass combination, treating those systematic uncertainties as having errors within their errors makes the central value very stable.
Lev: It’s about quantifying how much uncertainty comes from the input data not being fully compatible, which we see reflected in a larger confidence interval when those discrepancies are present.
Kai: So essentially, this framework is a general procedure for figuring out which systematic uncertainties you actually need to worry about when they have associated errors-on-errors.
Mira: The study concludes that the ATLAS-CMS mt combination is robust against these types of uncertainties, and it’s a useful tool for future combinations where those correlations are non-trivial.
Lev: For me, it shows us how to model the overhead of running this on real hardware because we can see how much uncertainty is added by these complex error structures.
Kai: That covers what we have here. We'll keep an eye on how this Gamma Variance Model helps us build better measurements for the next generation of experiments.
Enzo Canonero, Glen Cowan
Royal Holloway, University of London
hep-ex
Submitted: 2024-07-07
Updated: 2025-07-03
Comments: 19 pages, 5 figures
Journal ref: Eur. Phys. J. C 85, 156 (2025)
DOI: 10.1140/epjc/s10052-025-13884-w
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 74/100
The gist: The Gamma Variance Model (GVM) provides a statistical framework to handle uncertainties in systematic errors—informally called errors-on-errors—which is particularly useful when combining results…
Key concepts
- Gamma Variance Model (GVM)
- A statistical framework that models uncertainties in systematic errors by treating the systematic variance as a random variable following a gamma distribution. This allows researchers to account for 'errors-on-errors,' which are uncertainties in how systematic errors themselves are assigned, rather than just fixed values.
- Best Linear Unbiased Estimator (BLUE) vs. Nuisance Parameter Method
- The paper establishes a general link between two methods for estimating results when systematic uncertainties are correlated. It shows how to extend the nuisance parameter method to include errors-on-errors, providing analytical formulas for calculating central values and confidence intervals under complex correlation conditions.
- Errors-on-Errors
- These are uncertainties arising from errors in assigning systematic error values, often related to ambiguities in defining a '1-sigma' error or estimation procedures. The GVM specifically models these as random variables, allowing the analysis to account for how uncertainty in the error assignment affects the final result.
- Robustness and Outlier Sensitivity
- The GVM demonstrates that while the central value of a measurement combination is stable against uncertain systematic errors, its confidence interval size is sensitive to internal data consistency. When an outlier is added, its influence on the combination decreases as the error-on-error parameter increases.
Terminology
Summary
The Gamma Variance Model (GVM) provides a statistical framework to handle uncertainties in systematic errors—informally called errors-on-errors—which is particularly useful when combining results from several measurements, such as the 7–8 TeV ATLAS-CMS top quark mass combination. p. 0 Ha Top quarks
Keywords: Gamma Variance Model, errors-on-errors, combining measurements, correlated systematics, nuisance parameters, Bartlett correction, ATLAS-CMS top-mass combination
1 Introduction
In Particle Physics analyses such as those at the Large
Hadron Collider (LHC), many results are already dominated by systematic uncertainties or will be shortly. In this
context, uncertainties in the assigned values of systematic
errors themselves – informally called errors-on-errors
–
become increasingly important.Where the paper says it
Methodology of Error Modeling
The paper introduces the Gamma Variance Model (GVM) as a statistical model incorporating uncertainties in the assignment of systematic errors. This model allows for the treatment of both types of errors-on-errors, including those arising from ambiguities in defining a (13)
This model extends Eq. (3) by adding, in the last line,
terms associated with treating σ
2
us
as an adjustable parameter of the model and its estimate vs as gamma distributed.1-sigma
error and those stemming from the estimation procedures themselves. The GVM extends the likelihood function by treating the systematic variance as an adjustable parameter, denoted as vs, which is modeled as a gamma-distributed random variable. p. 3Where the paper says it
Equivalence of Estimation Methods
The paper establishes a general relation between the Best Linear Unbiased Estimator (BLUE) method and the nuisance-parameter method when correlations induced by systematic uncertainties are non-trivial. It demonstrates how to generalize the nuisance-parameter approach to include errors-on-errors and derives analytical formulas for estimating central values, confidence intervals, and goodness-of-fit under these conditions. p. 0 , not ±1 or 0), and we then generalise the nuisance-parameter approach to include errors-on-errors. We then
present analytical formulas for estimating central values, confidence intervals, and goodness-of-fit when
errors-on-errors are incorporated into the statistical model. (3) is used instead. In fact, the two
methods are equivalent when the auxiliary measurements
are set to zero, us = 0.Where the paper says it
Where the paper says it
Application to Top Quark Mass Combination
The GVM is applied to the 7–8 TeV ATLAS-CMS top quark mass combination. The study investigates the impact on the combination’s central value and confidence interval when the largest sources of systematic uncertainties are treated as uncertain, one at a time. p. 0 Specifically, we investigate the impact on the
combination’s central value and confidence interval when
the largest sources of systematic uncertainties are treated
as uncertain. We present the results by considering the
largest systematic uncertainties as uncertain, one at a
time, and we vary their associated error-on-error parameters. This example illustrates the behavior of the GVM in scenarios
where the inputs of a combination are not entirely internally compatible. Unlike a standard combination, incorporating errors-on-errors makes the central value of the
combination, here taken as the maximum likelihood estimator for the parameter of interest, more robust to discrepancies between input data values.Where the paper says it
Where the paper says it
Sensitivity to Internal Consistency and Outliers
The GVM exhibits a key feature where its sensitivity to internal consistency of the input data becomes relevant when a fictitious measurement is added as an outlier. The central value of the combination becomes less sensitive to outliers, while the presence of discrepancies results in an increase in the confidence interval. p. 1 Specifically, the central value becomes less sensitive to outliers. At the same
time, the presence of discrepancies results in an increase
in the confidence interval for the parameter of interest. To illustrate the properties of the Gamma
Variance Model, we apply it to the 7–8 TeV ATLAS-CMS top quark mass combination. We also explore
a hypothetical scenario by artificially adding a fictitious measurement as an outlier to the combination,
illustrating a key feature of the Gamma Variance Model — its sensitivity to the internal consistency of
the input data — which could become relevant for future combinations.Where the paper says it
Where the paper says it
Analytical Formulas for Implementation
The paper provides a set of analytical formulas designed to simplify the practical implementation of uncertain systematic errors. These formulas are useful for reducing the computational burden in complex analyses. p. 5 (15) and (18)
prevent exact analytical maximization of the GVM loglikelihoods with respect to µ and θ, as well as the computation of confidence intervals for µ and the evaluation of
goodness-of-fit for the observed data. Although these tasks
can always be performed using numerical techniques, analytical formulas are particularly useful for reducing the
computational burden that can arise in complex analyses.Where the paper says it
Conclusions on Robustness
The study concludes that the ATLAS-CMS mt combination is robust, with the central value remaining very stable across a broad spectrum of assumed values for errors-on-errors. The confidence interval remains stable as well, though it exhibits up to a 10% increase when the b-JES systematic uncertainty is considered uncertain. p. 1 At the same
time, the presence of discrepancies results in an increase
in the confidence interval for the parameter of interest. This increase reflects the uncertainty stemming from the
fact that the measurements in the dataset are not entirely
compatible. We present the results by considering the
largest systematic uncertainties as uncertain, one at a
time, and we vary their associated error-on-error parameters. This approach serves as a general method to identify the systematic uncertainties to which a combination
is sensitive when errors-on-errors are considered.Where the paper says it
Where the paper says it
Summary of Key Findings
The Gamma Variance Model (GVM) provides a statistical framework to handle uncertainties in the assignment of systematic errors, informally called errors-on-errors. The model is of particular use in analyses that combine the results of several measurements. p. 0 The Gamma Variance Model is a statistical model that incorporates uncertainties in the assignment of systematic errors (informally called errors-on-errors). The model is of particular use in analyses
that combine the results of several measurements. In the past, combinations have been carried out using
two alternative approaches: the Best Linear Unbiased Estimator (BLUE) method or what we will call the
nuisance-parameter method. In this paper, we obtain a general relation between the BLUE and nuisanceparameter methods when the correlations induced by systematic uncertainties are non-trivial (i. , not ±1 or 0), and we then generalise the nuisance-parameter approach to include errors-on-errors. We then
present analytical formulas for estimating central values, confidence intervals, and goodness-of-fit when
errors-on-errors are incorporated into the statistical model. This framework is then applied
to the 7-8 TeV ATLAS-CMS top quark mass combination [7]. Specifically, we investigate the impact on the
combination’s central value and confidence interval when
the largest sources of systematic uncertainties are treated
as uncertain. This example illustrates the behavior of the GVM in scenarios
where the inputs of a combination are not entirely internally compatible. Unlike a standard combination, incorporating errors-on-errors makes the central value of the
combination, here taken as the maximum likelihood estimator for the parameter of interest, more robust to discrepancies between input data values. 05322v4 [hep-ex] 3 Jul 2025
EPJ manuscript No. (will be inserted by the editor)
Correlated Systematic Uncertainties and Errors-on-Errors in
Measurement Combinations with an Application to the
7
-
8 TeV
ATLAS-CMS Top Quark Mass Combination
Enzo Canonero and Glen Cowan
Physics Department, Royal Holloway, University of London, U.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The authors argue that the assumption of independent gamma distributions for estimates of variances is a plausible model that arises, e.g., if nuisance parameters are estimated through an average of Gaussian-distributed measurements. p. 3 Some studies carried out
in connection with [1] of the log-normal distribution or
a gamma distribution for the estimated standard deviations led to qualitatively similar results, but were not
pursued further when the following reasons to use gammadistributed v became apparent. First, suppose that us itself is found as an average of
some number of other Gaussian-distributed values us,i,
i = 1, . In the GVM, the value
that would typically be assigned to σ
2
us
in a standard combination is treated as an estimate of this parameter and
is denoted as vs. One should keep in mind that the choice of independent gamma distributions for the estimates v is not expected to produce a perfect model, rather an approximation that represents a potentially significant improvement
over the assumption of exactly specified variances.Where the paper says it
Where the paper says it
How the training signal is built
The profile likelihood is maximized with respect to the NPs using a recursive approach, expressing profiled values as ˆˆθs = ˆˆθ(0)s + ε 2s ˆˆθ(1)s + ε 4s ˆˆθ(2)s + · · ·. The first term in this series is given by ˆˆθ(0)s = X M p=1 C(0)-1 sp [X N i,j=1 Γ p j V-1 ij (y j - µ) + u p v p]. p. 5 (21) and equating terms of equal powers
of ε
2
s
, we obtain a recursive set of equations for the coefficients ˆˆθ
(n)
s . The first term is given by
ˆˆθ
(0)
s =
X
M
p=1
C
(0)−1
sp
X
N
i,j=1
Γ
p
i V
−1
ij (yj − µ) + up
vp
, (23)
where the matrix C
(0)
sp is defined as
C
(0)
sp =
X
N
i,j=1
Γ
s
i V
−1
ij Γ
p
j +
δsp
vs
(24)
and δsp is the Kronecker delta.Where the paper says it
Where the gains come from
The Bartlett correction is used to adjust the likelihood ratio by a scaling factor so that its distribution more closely matches a χ squared distribution, reducing error terms to order ε 4s, while also ensuring that p-values remain accurate up to terms of the same order. This correction is applied to the GOF statistic and confidence intervals in the GVM likelihoods. p. 6 However, the GVM
log-likelihoods in Eqs. (13) and (16) contain non-quadratic
terms, causing deviations from the χ
2 distribution by terms
of order ε
2
s
, which affect the precision of confidence intervals. These ε
2
s approximations are highly effective when the
condition specified in Eq. (29) is satisfied. However, when this
occurs, it does not mean that the GVM is inapplicable;
rather, it suggests that numerical maximization should be
used instead. Using the perturbative solutions for ˆˆθs, the log-likelihood
can be optimized with respect to µ to determine its best
estimate.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved Robust Parameter Estimation: The system can estimate central values that are
more robust to discrepancies between input data values
by incorporating errors-on-errors through the Gamma Variance Model, which makes the combinationless sensitive to outliers.
-
Non-Trivial Correlation Handling: The AI system can perform measurements combinations where systematic uncertainties induce correlations that are not simply plus or minus one, by using an extended nuisance parameter approach that introduces distinct NPs for each systematic source.
-
Uncertainty Quantification via Scaling: The system can quantify the impact of uncertain systematic errors on a combination by treating the variance sigma squared us as an adjustable parameter and its estimate vs as gamma distributed, allowing it to model how
the systematic error induced by source s in measurement yi
scales. -
Outlier Sensitivity Mitigation: When an outlier is added to the dataset, the system can demonstrate that
within the GVM framework as the error-on-error parameter increases, the outlier is assigned lesser weight within the combination,
causing a shift back towards original results. -
Confidence Interval Inflation Modeling: The AI can predict that
the size of the confidence interval is sensitive to the internal consistency of the input data,
specifically showing how it inflates when errors-on-errors are considered and data are less compatible.
Abstract
The Gamma Variance Model (GVM) is a statistical model that incorporates uncertainties in the assignment of systematic errors (informally called errors-on-errors). The model is of particular use in analyses that combine the results of several measurements. In the past, combinations have been carried out using two alternative approaches: the Best Linear Unbiased Estimator (BLUE) method or what we will call the nuisance-parameter method. In this paper we derive useful relations that allow one to connect the BLUE and nuisance-parameter methods when the correlations induced by systematic uncertainties are non-trivial (1, -1 or 0), and we generalise the nuisance-parameter approach to include errors-on-errors. We then illustrate some of the properties of the GVM by applying it to the 7-8 TeV ATLAS-CMS top quark mass combination. We present results by considering the largest systematic uncertainties as uncertain, one at a time, and we vary their associated error-on-error parameters. This procedure is useful for identifying the systematic uncertainties to which a combination is sensitive when they are themselves uncertain. We also explore the hypothetical scenario of including an outlier in the combination, which could become relevant for future combinations, by artificially adding a fictitious measurement to it. This example highlights a key feature of the GVM: its sensitivity to the internal consistency of the input data.
Sources
- Combination of measurements of the top quark mass from data collected by the ATLAS and CMS experiments at $\sqrt{s}$ = 7 and 8 TeV
- Combination of CDF and D0 results on the mass of the top quark using up to 9.7 fb$^{-1}$ at the Tevatron
- A study of systematic uncertainties within the MSHT PDF Framework
Related papers
- DrSai: an agentic AI system toward autonomous physics analysis workflows at BESIII
- CANTON- mu Proposal: A Next-Generation Muon g-2 Measurement at Sub-0.1 ppm Precision
- Search for dark photon dark matter near 1,meV using a superconducting mixer
- ANUBIS detector and its sensitivity to neutral long-lived particles
- How to scale your HEP ML models: A recipe for robust architecture comparisons at scale
- Search for the 16 O(ppp) to 13 C pi+ pi+ e+ Decay Mode in Super-Kamiokande Using Machine Learning Techniques