SURFing to the Fundamental Limit of Jet Tagging
summary
The gist
The paper introduces the Surrogate Reference (SURF) method to validate generative models by comparing them against a tractable surrogate reference model to determine if modern jet taggers have…
In short
The SURF method validates generative models by comparing them against a simpler reference model to find their true limits. Using a surrogate model (EPiC-FM) to evaluate a complex GPT model revealed that GPT's separation power is artificially inflated, showing artifacts from the generative process rather than true performance limitations of jet taggers.
Key concepts
- Surrogate Reference (SURF) Method
- This method uses one tractable generative model (EPiC-FM) as a simple reference to test another complex one (GPT). It allows for exact statistical tests by training the target model on samples from the surrogate, helping to determine if the complex model is overstating its separation capabilities.
- Generative Models (GPT vs. EPiC-FM)
- The paper compares two models: GPT, which views jets as ordered sequences of tokens, and EPiC-FM, which treats jets as unordered particle clouds with kinematic features. The comparison shows that GPT's likelihood ratios inaccurately suggest a better separation than the simpler EPiC-FM model allows.
- Fundamental Limit
- This refers to the absolute maximum separation power achievable when distinguishing top jets from QCD jets. The study concludes that this limit is not far away from current state-of-the-art classifiers, suggesting that observed inflation in separation power comes from modeling artifacts, not classifier deficiencies.
Terminology used across episodes
This episode discusses
- SURFing to the Fundamental Limit of Jet Tagging · Paper Radio
- ParticleNet: Jet Tagging via Particle Clouds
- The Machine Learning Landscape of Top Taggers
- Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques
- An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging
- Explainable Equivariant Neural Networks for Particle Physics: PELICAN
- Particle Transformer for Jet Tagging
- A Lorentz-Equivariant Transformer for All of the LHC
- Solving Key Challenges in Collider Physics with Foundation Models
- A Method to Simultaneously Facilitate All Jet Physics Tasks
- Accuracy versus precision in boosted top tagging with the ATLAS detector
- OmniLearned: A Foundation Model Framework for All Tasks Involving Jet Physics
- The Fundamental Limit of Jet Tagging
- Learning the language of QCD jets with transformers
- Building Normalizing Flows with Stochastic Interpolants
- Flow Matching for Generative Modeling
- Improving and generalizing flow-based generative models with minibatch optimal transport
- EPiC-ly Fast Particle Cloud Generation with Flow-Matching and Diffusion
- Flow Matching Beyond Kinematics: Generating Jets with Particle-ID and Trajectory Displacement Information
- Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models · Paper Radio
- OmniJet- alpha: The first cross-task foundation model for particle physics
The paper
SURFing to the Fundamental Limit of Jet Tagging · Read on arXiv
Ian Pang, *, Darius A. Faroughy, †David Shih, ↓Ranit Das, Gregor Kasieczka
Rutgers University · Institute for Theoretical Physics, University of Heidelberg, Germany · Institut für Experimentalphysik, University of Hamburg
Transcript
Introduction to the show: ident: High Energy Physics: Phenomenology. The day in particle phenomenology: models, signatures and constraints.
Kai: I'm Kai.
Mira: And I'm Mira, here with Lev.
Kai: We're looking at the paper: "SURFing to the Fundamental Limit of Jet Tagging".
Paper summary: Mira: To recap what we've covered, this paper introduces the SURF method to validate generative models by using one model with tractable likelihoods, like EPiCFM, as a surrogate reference for another model like GPT.
Lev: The core idea is training the target GPT model on samples from that tractable surrogate EPiCFM, and then comparing the resulting likelihood ratios against those calculated directly from the surrogate.
Kai: What they found is that this comparison shows the NP optimal ROC curve derived from the GPT likelihoods actually far exceeds what's achievable by the EPiCFM surrogate itself.
Mira: That discrepancy demonstrates unambiguously that these GPT models are artificially inflating their apparent separation between top and QCD jets beyond the physical limit represented by EPiCFM.
Lev: They also pointed out that this overfitting happens right away, which makes those likelihood ratios inherently hard for actual classifiers to reproduce accurately.
Kai: The paper essentially argues against the idea that we're hitting some fundamental wall just because a new generative model looks better on paper.
Mira: Instead, they conclude that the inflated optimal behavior comes from artifacts in the GPT likelihood landscape, not from deficiencies in classifier performance as previously suggested.
Lev: So, for those of us building error correction systems for real hardware, this is important because it tells us what kind of noise we’re actually dealing with when we try to model these processes.
Kai: It shifts the focus from just chasing better classifiers to understanding the statistical properties of the generative models themselves.
Conclusion: Mira: So, looking at "SURFing to the Fundamental Limit of Jet Tagging," it seems like this work is more about validating tools than finding a new physical limit for jet tagging.
Lev: The authors are focused on using their SURF method to check the validity of generative models by comparing them against a tractable surrogate reference model, EPiCFM.
Kai: Ultimately, the conclusion they draw is that there's no evidence of an inflated fundamental limit for jet tagging itself based on this comparison.
Mira: The picture they present is that the real limit isn't far away from what we already have in state-of-the-art classifiers, considering how different ROC curves align.
Lev: They suggest that the consistency between classifiers trained on JetClass and EPiCFM samples points to this conclusion about where we stand right now.
Kai: So, to put it plainly, the results show that the performance limitations aren't where people thought they were, but rather in how these generative models are producing their likelihood ratios.
More episodes
- 2610.38463-Gauge-parameter cancellation and finite form factors in one-loop quark scattering in the refined Gribov-Zwanziger framework
- 2610.37198-Searches for new particle physics by methods of classical and neutrino astronomy
- 2505.01492-Photo- and Hadrodisintegration constraints on massive relics decaying into neutrinos
- 2505.20389-Dispersion relation of the neutrino plasma: Unifying fast, slow, and collisional instabilities
- 2511.10731-Keeping It Renormalizable: Minimal Baryogenesis induced Asymmetric Dark Matter
- 2404.04021-Vacuum correlations of the stress-energy-momentum tensor with constituent quarks
- 2503.02931-Relativistic Navier-Stokes description of the quark-gluon plasma radial flow
- 2606.06105-Bulk viscosity of a binary mixture: the role of the intra-species interaction
- 2607.03343-First look at the evaluation of two-loop Feynman integrals for radiative return processes
- 2607.04132-Left-right symmetry breaking in E 6 L times E 6 R occurs only in spacetime -- with possible implications for strong CP