Silent coverage failures in rare-event searches and a degeneracy index that predicts them
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.
Kai: We're looking at the paper: "Silent coverage failures in rare-event searches and a degeneracy index that predicts them".
Mira: Here's the gist of the paper. Searches for new physics in low-background experiments infer a non-negative signal strength from few events and often report an upper limit.
Kai: The Poisson–Fisher degeneracy index IPF(δν; ϑ0) = (β, γ), which maps a specified expected-count deformation, after projection onto the complete fitted tangent space, to β, the signed fitted signal shift in profiled standard-error units, and γ, the Poisson–Fisher norm of the unabsorbed residual.
Mira: This index provides a local diagnostic for whether a physically specified model error can bias the fitted signal, which interval endpoint is threatened, and whether the fitted residual can reveal the error.
Kai: Pruning redundant reasoning tokens The paper introduces the Poisson–Fisher degeneracy index IPF = (β, γ), which pairs the signed standardized signal bias with the residual left outside the complete fitted tangent space.
Paper summary: Kai: So, to wrap up the paper, "Silent coverage failures in rare-event searches and a degeneracy index that predicts them," it’s about giving us this Poisson–Fisher degeneracy index, which maps model deformation to signal shift and residual mismatch.
Mira: The authors are showing how this index lets us screen potential coverage damage and goodness of fit detectability before we even start toy calibrations using just expected bin counts and local model derivatives.
Lev: Essentially, they’re giving researchers a way to diagnose whether a plausible model error can bias the fitted signal, which interval endpoint is threatened, and if the fitted residual can reveal that error.
Kai: The implication is that this index provides a local diagnostic for coverage failure in rare-event searches where we infer a non-negative signal strength from few events and often report an upper limit.
Mira: It helps us understand the tension between the interval construction and the data model, especially when regularity conditions underpinning standard asymptotic theory fail, like with low counts or nuisance parameters.
Lev: The title points to the fact that these failures can be silent—meaning the interval misses its frequentist target while our goodness of fit test still seems okay.
Kai: It’s a useful framework for guiding model checks and uncertainty treatment in experiments where we’re pushing low-background limits using few events, like in dark matter recoil spectra.
Conclusion: Kai: So, this paper introduces something called a Poisson Fisher degeneracy index that pairs how much the model is wrong with how much the signal prediction gets skewed and what’s left over in our residual.
Mira: It sounds like they're giving us a diagnostic tool to see if our model assumptions are messing up our final measurement when we only have a few events.
Lev: From an engineering side, that means before we spend time calibrating everything with toy numbers, we can check if the model itself is introducing bias or if the residual will even show us the error.
Kai: Exactly. The authors are showing this index helps screen potential coverage damage and goodness of fit issues before we even start our calibration work using just expected counts.
Mira: It’s about identifying those tricky situations where a plausible error in our model could hide a problem or make us think our interval is covering the right thing when it's not.
Lev: And they show how to use the sign of that index, that beta value, to tell if a positive bias threatens the lower endpoint or if a negative bias makes the upper endpoint undercover.
Kai: That’s really useful for understanding why some searches might fail and others might seem okay just because of how we framed our analysis.
Mira: It moves us away from just checking if the numbers fit, toward actually checking if the underlying physics model is compatible with what we're observing.
Lev: It’s a way to prioritize where we need to spend our time fixing things—is it a nuisance parameter that needs constraining, or is it a fundamental issue with how we modeled the signal itself?
Kai: And they show that this index isn't universal across all types of model errors, which is important because it tells us what kind of problem we’re likely dealing with.
Mira: It sets up a new way to think about uncertainty—not just as a single number on an axis, but as a relationship between the signal shift and the residual mismatch.
Lev: It gives us a way to guide those model checks when we're pushing the limits of what we can measure with low event statistics.
Kai: So it’s not just about finding new physics, it’s about making sure our search methodology is sound before we declare something found, or that we miss something important.
Mira: It really shifts the focus onto the connection between our theoretical model and the practical limits of what a detector can actually measure.
Department of Mechanical and Industrial Engineering, University of Brescia · Istituto Nazionale di Fisica Nucleare
hep-ex, stat.ME
Submitted: 2026-08-10
Updated: 2026-10-09
DOI: 10.1140/epjc/s10052-026-16570-7
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 82/100
The gist: Searches for new physics in low-background experiments infer a non-negative signal strength from few events and often report an upper limit.
Key concepts
- Poisson–Fisher degeneracy index IPF(δν; ϑ0)
- This index is a diagnostic tool that maps a specified expected-count deformation to two values: β and γ. It tells researchers whether a model error can bias the fitted signal, which interval endpoint is at risk, and if the residual mismatch can detect the error.
- Signed fitted signal shift (β)
- This coordinate represents the predicted shift in µ, expressed in profiled standard-error units. It measures how much a model deformation could potentially damage the inferential results of a search by biasing the fitted signal.
- Poisson–Fisher norm of the residual (γ)
- This non-negative coordinate is the Poisson–Fisher norm of the residual left outside all fitted directions. It quantifies how much mismatch exists between the model and data, indicating if a compatible residual-based goodness-of-fit test can be performed.
- Controlled model misspecification
- This involves intentionally deforming the nominal expected-count vector ν⋆(ϑ; s) by adding a perturbation δν. This covers various errors like unmodelled background tails, shifted dark-matter halos, or overestimated signal efficiency.
Terminology
Summary
Searches for new physics in low-background experiments infer a non-negative signal strength from few events and often report an upper limit. p. 0 2
Istituto Nazionale di Fisica Nucleare (INFN), Pavia, Italy. Abstract
Searches for new physics in low-background experiments infer a non-negative signal strength from
few events and often report an upper limit. We study how controlled model departures affect
lower- and upper-endpoint coverage for six interval procedures in an exact Poisson counting experiment, dark-matter recoil spectra, and a neutrinoless-double-beta-decay peak search. We introduce
the Poisson–Fisher degeneracy index IPF(δν; ϑ0) = (β, γ), which maps a specified expected-count
deformation, after projection onto the complete fitted tangent space, to β, the signed fitted signal shift
in profiled standard-error units, and γ, the Poisson–Fisher norm of the unabsorbed residual. Learned posteriors can be inaccurate even under
the simulator [19], simulator mismatch motivates
discrepancy models [20, 21], and conformal validity requires exchangeability [22, 23]. Fisher geometry, nuisance projection, and misspecification asymptotics supply important ingredients, but they do not by themselves provide
a single signed diagnostic for three linked questions in low-count, boundary-constrained inference: whether a physically specified model error
can bias the fitted signal, which interval endpoint
is threatened, and whether the fitted residual can
reveal the error.Where the paper says it
Where the paper says it
Where the paper says it
Pruning redundant reasoning tokens
The paper introduces the Poisson–Fisher degeneracy index IPF = (β, γ), which pairs the signed standardized signal bias with the residual left outside the complete fitted tangent space. The signed coordinate β is called and goodness-of-fit detectability before toy calibration, using only expected bin counts and local
model derivatives. Choose a reference point ϑ0 in the fitted
model.the predicted shift in µ, expressed in profiled standard-error units; it therefore measures the deformation’s potential inferential damage
. The non-negative coordinate γ is described as the Poisson–Fisher norm of the residual after all fitted directions have been profiled and measures the mismatch available to a compatible residual-based goodness-of-fit test
. The index is intended to screen potential coverage damage and goodness-of-fit detectability before toy calibration using only expected bin counts and local model derivatives. p. 4Where the paper says it
How the training signal is built
The authors define a controlled model misspecification as a deformation of the nominal expected-count vector ν⋆(ϑ; s) = ν0(ϑ) + δν(ϑ; s), where s ∈ [0, 1] is the severity. This formulation covers additive rate or shape contamination, signal-model deformation, and detector response error. p. 2 The severity s = 1 denotes a declared stresstest endpoint, not a universal physical scale. This
formulation covers additive rate or shape contamination, signal-model deformation, and detectorresponse error; the numerical stress-test endpoints
are defined below. Second, and more insidiously, model misspecification:
the likelihood (or simulator) used for inference is
never identical to the process that generated the
data. An unmodelled low-energy background tail,
an uncertain dark-matter halo model, or an optimistic detector response can therefore invalidate
classical coverage guarantees. The counting study uses only its integrated excess,
up to six events; the spectral study uses Ed =
1 keV and up to 40 events (Section 6). Shifted dark-matter halo.Where the paper says it
Where the paper says it
Where the paper says it
Where the gains come from
The index is used in a five-step procedure to guide model checks and uncertainty treatment. The sign of β indicates whether a positive bias threatens the lower endpoint (false rejection of µ = 0), while negative bias can make the upper endpoint undercover. p. 0 Locally,
the sign of β identifies the threatened endpoint, while γ provides a local indication of detectability
by the saturated-Poisson goodness-of-fit test used here at a 5% nominal type-I error rate. Across the
studied deformations, positive signal-like bias degrades discovery-side coverage while making upper
limits conservative; negative signal bias from overestimated signal efficiency can make the upper endpoint undercover. Calibration under the nominal simulator does not protect against misspecification
of that simulator relative to the data-generating process. A plausible deformation with large |β| and
small γ may therefore evade diagnosis and should be represented by a nuisance constrained with auxiliary information or included in a defensible envelope. There are no universal thresholds for a “large” |β|
or a “small” γ: an analysis should derive them
from the largest tolerable change in endpoint coverage and the required model-check power. Table 1
in Section 7 illustrates steps 2–4 for representative
configurations of the deformation families. coverage instead averages over a declared parameter distribution π(ϑ):
Pϑ∼π, x∼p(· | ϑ)
θ ∈ Rα(x)
≥ 1 − α. (3)
This is the finite-sample conformal guarantee
under exchangeability when the score is fixed
before calibration.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Inference methods and numerical protocol
The paper compares various inference methods including Feldman–Cousins, CLs, profile-likelihood, and Bayesian intervals. The counting studies use Feldman–Cousins and LF2I which employ model-specific Neyman calibration. p. 6 5 Inference methods and
numerical protocol
The counting studies use Feldman–Cousins, CLs,
profile-likelihood, and Bayesian intervals; the
spectral studies use profile likelihood, LF2I, and
split-conformal posterior scores. Feldman–Cousins
and LF2I use model-specific Neyman calibration, the baseline profile likelihood is asymptotic,
the CLs construction used here is conservative,
Bayesian intervals control posterior credibility,
and the conformal guarantee is prior-predictive
and marginal. 5 Inference methods and
numerical protocol
The counting studies use Feldman–Cousins, CLs,
profile-likelihood, and Bayesian intervals; the
spectral studies use profile likelihood, LF2I, and
split-conformal posterior scores. Feldman–Cousins
and LF2I use model-specific Neyman calibration, the baseline profile likelihood is asymptotic,
the CLs construction used here is conservative,
Bayesian intervals control posterior credibility,
and the conformal guarantee is prior-predictive
and marginal. Coverage estimation. At fixed ϑ, effective coverage (4) is estimated from
N nominal or misspecified datasets as the fraction
of regions containing θ, or of projected intervals containing
p
µ, with binomial standard error
c(1 − c)/N; for example, N ≈ 3600 gives 0.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved AI systems can implement a Poisson-Fisher degeneracy index (IPF) to map model deformations to their signed effect on signal inference (β) and residual detectability (γ). This allows the system to screen potential coverage damage and goodness-of-fit detectability before toy calibration, as described in Section 4.
-
The system can use the IPF coordinates (β, γ) to guide model checks by reading the sign of β; a positive β
threatens the lower endpoint
while a negative β makes the upper limit conservative. It can also use γ to indicate cases withlow residual-test power.
-
The AI can employ a coverage-sensitivity remedy by promoting signal-like deformations to constrained nuisances, which
restores coverage at a measurable cost in interval length,
as detailed in Section 6.4. This allows the system to report robust limits even when model misspecification occurs.
Abstract
Searches for new physics in low-background experiments infer a non-negative signal strength from few events and often report an upper limit. Nominal frequentist coverage requires both a valid interval construction and an adequate data model. We study how controlled model departures affect lower- and upper-endpoint coverage for six interval procedures in an exact Poisson counting experiment, dark-matter recoil spectra, and a neutrinoless-double-beta-decay peak search. We introduce the Poisson--Fisher degeneracy index I PF(δν; 0)=(β,γ), which maps a specified expected-count deformation, after projection onto the complete fitted tangent space, to β, the signed fitted signal shift in profiled standard-error units, and γ, the Poisson--Fisher norm of the unabsorbed residual. Locally, the sign of β identifies the threatened endpoint, while larger γ implies greater detectability by the saturated-Poisson goodness-of-fit test used here at a fixed 5% type-I error rate. Across the studied deformations, positive signal-like bias degrades discovery-side coverage while making upper limits conservative; negative signal bias from overestimated signal efficiency can make the upper endpoint undercover. Calibration under the nominal simulator does not protect against misspecification of that simulator relative to the data-generating process. A plausible deformation with large β and small γ may therefore evade diagnosis and should be represented by a nuisance constrained with auxiliary information or included in a defensible envelope. In the exactly collinear constrained-nuisance benchmark, modelling the deformation restores coverage at the evaluated grid points, at a quantifiable cost in interval sensitivity.
Sources
- First Dark Matter Search with Nuclear Recoils from the XENONnT Experiment
- Results from the Super Cryogenic Dark Matter Search (SuperCDMS) experiment at Soudan
- Final Results of GERDA on the Search for Neutrinoless Double-$\beta$ Decay
- Search for Majorana neutrinos exploiting millikelvin cryogenics with CUORE
- Search for the Majorana Nature of Neutrinos in the Inverted Mass Ordering Region with KamLAND-Zen
- Asymptotic formulae for likelihood-based tests of new physics
- A Unified Approach to the Classical Statistical Analysis of Small Signals
- Confidence Level Computation for Combining Searches with Small Statistics
- The frontier of simulation-based inference
- Likelihood-Free Frequentist Inference: Bridging Classical Statistics and Machine Learning for Reliable Simulator-Based Inference
- A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful
- Investigating the Impact of Model Misspecification in Neural Simulation-based Inference
- Conformal Prediction Under Covariate Shift
- Conformal prediction beyond exchangeability
- Astrophysical uncertainties on the local dark matter distribution and direct detection experiments
- Measurement of Higgs boson production in the diphoton decay channel in $pp$ collisions at center-of-mass energies of 7 and 8 TeV with the ATLAS detector
- Observation of the diphoton decay of the Higgs boson and measurement of its properties
Related papers
- DrSai: an agentic AI system toward autonomous physics analysis workflows at BESIII
- CANTON- mu Proposal: A Next-Generation Muon g-2 Measurement at Sub-0.1 ppm Precision
- Search for dark photon dark matter near 1,meV using a superconducting mixer
- ANUBIS detector and its sensitivity to neutral long-lived particles
- How to scale your HEP ML models: A recipe for robust architecture comparisons at scale
- Search for the 16 O(ppp) to 13 C pi+ pi+ e+ Decay Mode in Super-Kamiokande Using Machine Learning Techniques