JetCoRD: Reliability-Gated Correction of Jet Taggers over Two Frozen Teachers
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: High Energy Physics: Phenomenology. The day in particle phenomenology: models, signatures and constraints.
Kai: We're looking at the paper: "JetCoRD: Reliability-Gated Correction of Jet Taggers over Two Frozen Teachers".
Mira: Here's the gist of the paper. Modern jet taggers based on graph and transformer networks deliver state-of-the-art performance but are expensive to train and difficult to share across experiments The central innovation is a single per-sample reliability signal ri that simultaneously weights the distillation loss, controls prototype-based teacher repair, and anchors the inference-time gate gi that mixes teacher and student logits JetCoRD is an 82 k-parameter unified student distilling jointly from ATLAS GN2 (5 M params, 3 classes) and CMS ParT (2 M params, 10 classes) The student matches both teachers in overall accuracy and exceeds them at physics-actionable working points: +4.3% on b-vs-c at ε = 0.77, +1.5% on c-vs-u at ε = 0.30, +1.6% on T → bqq at ε = 0.5 and +1.4% on H → bb at ε = 0.5 Pruning redundant reasoning tokens The paper introduces JetCoRD, an 82 k-parameter cross-experiment jet-tagging student that distills both the ATLAS GN2 flavour tagger and the JetClass-trained ParT classifier This is accomplished at merely 1.2% of the teacher parameter budget, enabled by two key innovations: Adaptive Corrective Representation Distillation (A-CoRD) on the loss side and Reliability-Aware Inference fusion combined with a Class-Conditional Backbone on the architecture side How the training signal is built The reliability signal ri is defined as ri = ⊮ yˆT i = y i · σ(κ · m i), where m i = top1(t i) − top2(t i) This signal scales monotonically with the teacher’s logit margin, scaling to 0 if the teacher is wrong and to 1 when the teacher is right and confident The reliability-weighted distillation loss is defined as L w KD = (1/N) Σ i ri · T squared · KLσT(t i) / σT(s i) Furthermore, the class-adaptive repair coefficient βc is defined as βc = β0 · σ αc − s · accT c Where the gains come from The coupling between A-CoRD and RAI shares a single reliability concept, where A-CoRD’s training-time reliability signal ri becomes the anchor for the inference-time gate in RAI The final logits are a convex mixture s final i = gi · t i + (1 − gi) · s student i Ablation studies show that the gains come from the combination of RAI and A-CoRD, with RAI providing the dominant architectural uplift The learnable per-class βc converges to a sparse repair pattern that mirrors the teacher’s weak spots, directing essentially all of its repair budget to two specific classes — u in ATLAS, and H →4q in JetClass Pruning redundant reasoning tokens The paper demonstrates that KD achieves a near-zero bg margin (−0.04 %) while A-CoRD achieves a positive mean margin (+0.22 %) that concentrates on specific hard binary boundaries The gate gi has learned to not trust the teacher when the teacher is wrong, dropping by −0.20 (ATLAS) and −0.09 (JC) relative to the correct subset Where the gains come from The analysis reveals that A-CoRD helps when the downstream task is dominated by a specific hard binary boundary where the teacher’s per-class accuracy is sub-perfect KD suffices if the downstream task is dominated by average accuracy and the teacher’s per-class accuracies are well-matched to the desired operating points The reliability-coupled training/inference design appears, to our knowledge, novel in HEP distillation Pruning redundant reasoning tokens The paper concludes that A-CoRD + PCG yields the deepest bg margin (+0.40%), while the learnable per-class βc of A-CoRD focuses the repair effort onto exactly the two classes where the teachers are statistically weakest The reliability-coupled training/inference design appears, to our knowledge, novel in HEP distillation and is broadly applicable beyond jet tagging — anywhere a heavyweight teacher is imperfectly calibrated and a cheap student must respect its mistakes Where the gains come from The paper notes that A-CoRD shows a small but reproducible 2% regression on H →qql¯ @0.99 and t → blν@0.995 (teacher class accuracies 98.1 % and 98.6 % respectively) at the extreme tail of Ps/(Ps + Pb) This regression is a structural consequence of repairing teacher mistakes in representation space The paper suggests that a saturation mask on βc for classes where the teacher exceeds 97% accuracy and a residual student-head parameterisation are natural structural remedies The code and trained checkpoints are available at https://github.
Title and authors: Kai: So we're looking at this paper called "JetCoRD: Reliability-Gated Correction of Jet Taggers over Two Frozen Teachers," and it tackles how to use knowledge distillation across different experiments. It seems like the authors are trying to make a single, smaller student model that can learn from two very different, but very accurate, teacher models.
Mira: Yeah, it's interesting because the main problem they identified is that standard knowledge distillation relies on the assumption that the teacher is perfect at tagging jets. But when you look at high-purity working points in experiments like ATLAS or CMS, those teachers actually have a prediction error rate of around ten to thirty percent.
Lev: That makes sense from an engineering standpoint; if the teacher is consistently wrong, distilling its mistakes just makes the student learn to mimic the wrong answers >
Kai: Exactly. And what JetCoRD does is introduce a single per-sample reliability signal, which they call r sub i. This signal does three things at once: it weights how much you should distill each sample's information, it controls how the teacher's mistakes are repaired in the prototype stage, and it anchors that inference gate later on >
Mira: That sounds like a smart way to treat the teacher’s errors as actual data rather than just something to ignore. They define this reliability signal as r sub i equals negative y hat sub T i, which is y sub i multiplied by the sign of kappa times m sub i, where m sub i is the difference between the top one and top two logits >
Lev: So it scales monotonically with how confident the teacher is, scaling to zero if the teacher gets something wrong and one when it's right and confident, which sounds like a very direct way to measure trust >
Kai: Right. And this signal then feeds into a reliability-weighted distillation loss that they call L sub w KD, which uses this r sub i to weight the distillation loss based on the difference between the teacher and student logits >
Mira: Plus, there's this class adaptive repair coefficient beta c, which is defined by a combination of some base value beta zero and a term involving sigma alpha c s acc T c >
Lev: So it sounds like they are trying to dynamically allocate where the student should focus its learning based on how much the teacher is actually struggling with specific classes, like focusing on the two classes where the teachers are statistically weakest >
Kai: And this coupling between that training time signal and a later inference gate is what they call Reliability-Aware Inference fusion, or RAI. The final logits are a convex mixture of the teacher's logit and the student's logit, weighted by this gate g sub i >
Title and authors: Mira: That means when you make a prediction at test time, if the teacher is wrong about something specific, that gate g sub i drops low, and you rely more on what the student learned >
Lev: The ablation studies show that without this combination of A-CoRD and RAI, distillation alone achieves a near zero background rejection margin of negative zero point four percent, while A-CoRD alone manages to get a positive mean margin of plus zero point two two percent concentrated on specific hard boundaries >
Kai: That's a big difference. It shows that the architectural uplift from RAI is what provides the main gain when you are dealing with tasks dominated by a specific hard binary boundary where the teacher isn't perfect >
Mira: They also found that the learnable class adaptive coefficient beta c converges to a sparse repair pattern, directing essentially all of its repair budget to just two specific classes—u in ATLAS and H to four q in JetClass >
Lev: That focus on only those two classes is interesting because it suggests a very targeted way to fix the teacher's flaws rather than trying to fix everything at once >
Kai: And this leads them to suggest that A-CoRD plus PCG yields the deepest background rejection margin of plus zero point four percent, which happens when that class adaptive coefficient focuses the repair effort onto exactly those two weakest classes >
Mira: The paper does mention a limitation where A-CoRD shows a small but reproducible regression on H to q sub q slash epsilon at zero point nine nine and t to b l nu at zero point nine nine five, which is the extreme tail of Ps over Ps plus Pb >
Lev: That regression seems to be a structural consequence of how they are repairing those teacher mistakes in representation space, which suggests that maybe we need a saturation mask on beta c for classes where the teacher exceeds ninety seven percent accuracy >
Kai: So, overall, JetCoRD is about using this reliability concept—the training signal and the inference gate—to make distillation work even when your teachers are imperfect and from different experiments. It moves beyond just matching overall accuracy to actively correcting specific weaknesses >
Mira: The implication is that this method could be broadly applicable anywhere you have a heavyweight teacher that's imperfectly calibrated, and you want a cheaper student to respect those mistakes >
Lev: It sounds like a solid framework for moving from simple parameter matching to something more robust when dealing with real-world, messy detector data >
Kai: So we’re done with JetCoRD. Next up, we're looking at how this idea might play out in some of those other dark matter searches we talked about earlier.
The paper's summary: Kai: So we're looking at JetCoRD now, which is this model that tries to fix the problem of expensive jet taggers by having one student learn from two different, heavyweight teachers simultaneously across experiments.
Mira: Right, so the paper says they managed to make this single student model—the eighty-two thousand parameter one—distill knowledge from two very different systems at only a fraction of the teacher's total parameters.
Lev: And they say that this efficiency comes from two main things, Adaptive Corrective Representation Distillation on the loss side, and Reliability-Aware Inference fusion combined with a Class-Conditional Backbone on the architecture side.
Kai: That’s what I find interesting about it because they are using one single reliability signal, let's call it r sub i, to do three different jobs at once. It weights the distillation loss, it controls how the teacher's prototypes get repaired, and it anchors that inference gate later on.
Mira: Exactly. This signal r sub i is defined based on the top one and top two logits from the teacher, scaling with how confident they are about their tag. If they’re wrong, that signal drops to zero; if they’re right and certain, it goes up to one.
Lev: So when we look at the training loss, this reliability-weighted distillation loss uses that r sub i to decide which parts of the teacher's mistakes we should focus on correcting in the student model.
Kai: And that links directly into the inference stage with what they call Reliability-Aware Inference fusion, or RAI. Instead of just picking one logit, they use a convex mixture where a gate g sub i decides how much to trust the teacher versus the student for each sample.
Mira: The crucial part there is that this gate g sub i gets its anchor from that same training reliability signal r sub i. So, if the teacher was unreliable during training on a certain type of jet, that same information dictates how much we trust it at test time.
Lev: And they show through their ablation studies that this combination of A-CoRD and RAI is what gives the real performance boost, especially when the task depends on a very specific hard boundary where the teacher’s accuracy isn't perfect.
Kai: They also found that their learnable coefficients for class repair, beta c, really focus their effort on just two classes—u in ATLAS and H to four q in JetClass—which is pretty targeted.
Mira: That means the system doesn't waste its effort trying to fix every single mistake; it just targets the teacher's statistically weakest points. It’s a very specific kind of repair pattern.
Lev: But they did have one caveat, so watch this carefully: A-CoRD actually showed a small regression on some extreme tails of the measurement record, like at zero point nine nine for certain classes.
Kai: That regression seems to be just the structural consequence of repairing those mistakes in the representation space, which makes me think we might need some kind of saturation mask on those repair coefficients for classes where the teacher is already super accurate.
Mira: If you look at that, it suggests that this reliability coupling idea—linking training and inference through that signal—is pretty novel in how we do distillation in high energy physics.
Lev: It really moves beyond just matching overall accuracy; it’s about making the cheap student respect the teacher’s specific errors, which is a concept I think could be useful anywhere you have a heavy model that's imperfect and you need a lighter one to learn from its flaws.
The paper's improvements: Tom: So we're looking at what they suggest as improvements for JetCoRD now, which is this way to make that whole system even better than just combining the training and inference gates you already have.
Kai: The authors suggest that A-CoRD coupled with PCG yields the deepest background rejection margin of plus zero point four percent, which happens when they use that class adaptive coefficient beta c to focus all the repair effort onto those two weakest classes.
Mira: That’s a significant increase over what KD could do on its own, and it shows how focusing that repair budget helps push those specific metrics up.
Lev: And they also noted a limitation where A-CoRD showed a small regression on some extreme tails of the measurement record, like at zero point nine nine for certain classes.
Kai: They suggest that this might be a structural consequence of repairing those mistakes in representation space, and they propose using a saturation mask on beta c for those classes where the teacher is already over ninety seven percent accurate.
Mira: So it's not just about making the loss smaller; it's about understanding *why* the teacher is wrong at certain points and adjusting how much we trust its answer based on that knowledge.
Lev: From an engineering side, if we are running this on real hardware, that saturation mask idea sounds like something we could implement to stabilize the learning process when you hit those extreme limits.
Kai: It brings us back to the whole idea of linking training and inference through that reliability concept—that’s the core mechanism they keep emphasizing.
Mira: The implication is that this reliability-coupled design is applicable beyond just jet tagging, it’s about building a cheap student model that respects the known imperfections of any heavyweight teacher across different experiments.
Lev: It suggests that we can move past just matching overall accuracy and start actively correcting specific weaknesses, which is what makes this method useful for real data where you have these weird, hard binary boundaries.
Conclusion: Kai: So we’re wrapping up on JetCoRD, which is this paper about using a single student model to learn from two different jet taggers at once across experiments like ATLAS and CMS.
Mira: Basically, they developed a way to use one reliability signal that controls both the training loss and the test-time decision gate.
Lev: The key finding is that by coupling this training signal with an inference fusion mechanism, you get a student model that performs better than either teacher alone in specific working points.
Kai: They showed that this method can handle cross-experiment distillation efficiently, even when the teachers have different detector responses and class spaces.
Mira: The main implication is that we can stop assuming the teacher is perfect at tagging jets and instead treat its mistakes as actual information we need to learn from.
Lev: From an engineering standpoint, this means we’re building a system where the student learns to trust the teacher less when it’s wrong, which should make our error correction much more robust for real hardware deployment.
Kai: The method allows us to dynamically focus the learning effort onto exactly the classes where the teachers are weakest, like u in ATLAS and H to four q in JetClass.
Mira: That targeted repair pattern is smart; it shows that we don't need a brute-force fix for every class mistake if we know where those mistakes are concentrated.
Lev: It makes sense because trying to correct everything at once usually just overcomplicates the error correction overhead, and focusing on the weak points cuts down on that computational cost significantly.
Kai: So to recap, JetCoRD is this framework for reliable distillation using a single reliability signal across different jet tagging experiments.
Mira: It changes how we think about knowledge transfer, moving from just matching final accuracy to actively correcting specific errors in the representation space.
Lev: It’s a solid piece of work because it shows you can make a much smaller model perform better by respecting the imperfections of a larger one.
Kai: And that’s where we’re going for today with JetCoRD, but next up, we've got some papers looking at how to search for stopped long-lived particles in melting LHC detectors.
Institute of High Energy Physics, Chinese Academy of Sciences · University of Chinese Academy of Sciences
hep-ph
Submitted: 2026-07-17
Updated: 2026-11-03
Code: https://github.com/sysu17363020/JetCoRD
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 89/100
The gist: Modern jet taggers based on graph and transformer networks deliver state-of-the-art performance but are expensive to train and difficult to share across experiments The central innovation is a single…
Key concepts
- Reliability Signal ($r_i$)
- This signal measures how right and confident a teacher is for a specific sample. It scales from 0 (teacher is wrong) to 1 (teacher is right and confident), based on the difference between the top two predicted classes. This signal acts as a universal metric for judging teacher reliability.
- Adaptive Corrective Representation Distillation (A-CoRD)
- A-CoRD uses the reliability signal ($r_i$) to weight distillation loss. It treats teacher mistakes as valuable information during training, allowing the student to focus its learning on correcting the specific errors made by the teacher, rather than just matching its output.
- Reliability-Aware Inference fusion (RAI)
- RAI determines which prediction to trust at inference time using a gate ($g_i$). This gate is directly anchored by the training reliability signal. It creates a convex mixture of teacher and student logits, ensuring that the system dynamically trusts the teacher less when it is known to be unreliable.
- Class-Conditional Backbone ($eta_c$)
- This mechanism uses learnable coefficients ($eta_c$) to dynamically allocate repair effort based on class. These coefficients automatically contract toward zero for classes where the teacher is already highly accurate, focusing the student's correction budget only on the specific classes where the teacher makes mistakes.
Terminology
Summary
Modern jet taggers based on graph and transformer networks deliver state-of-the-art performance but are expensive to train and difficult to share across experiments The central innovation is a single per-sample reliability signal ri that simultaneously weights the distillation loss, controls prototype-based teacher repair, and anchors the inference-time gate gi that mixes teacher and student logits JetCoRD is an 82 k-parameter unified student distilling jointly from ATLAS GN2 (5 M params, 3 classes) and CMS ParT (2 M params, 10 classes) The student matches both teachers in overall accuracy and exceeds them at physics-actionable working points: +4. p. 03% on b-vs-c at ε = 0.77, +1.5% on c-vs-u at ε = 0.30, +1.6% on T → bqq at ε = 0. p. 05 and +1.4% on H → bb at ε = 0.5
Pruning redundant reasoning tokens
The paper introduces JetCoRD, an 82 k-parameter cross-experiment jet-tagging student that distills both the ATLAS GN2 flavour tagger and the JetClass-trained ParT classifier This is accomplished at merely 1.2% of the teacher parameter budget, enabled by two key innovations: Adaptive Corrective Representation Distillation (A-CoRD) on the loss side and Reliability-Aware Inference fusion combined with a Class-Conditional Backbone on the architecture side
How the training signal is built
The reliability signal ri is defined as ri = ⊮ yˆT i = y i · σ(κ · m i), where m i = top1(t i) − top2(t i) This signal scales monotonically with the teacher’s logit margin, scaling to 0 if the teacher is wrong and to 1 when the teacher is right and confident The reliability-weighted distillation loss is defined as L w KD = (1/N) Σ i ri · T squared · KLσT(t i) / σT(s i) Furthermore, the class-adaptive repair coefficient βc is defined as βc = β0 · σ αc − s · accT c
Where the gains come from
The coupling between A-CoRD and RAI shares a single reliability concept, where A-CoRD’s training-time reliability signal ri becomes the anchor for the inference-time gate in RAI The final logits are a convex mixture s final i = gi · t i + (1 − gi) · s student i Ablation studies show that the gains come from the combination of RAI and A-CoRD, with RAI providing the dominant architectural uplift The learnable per-class βc converges to a sparse repair pattern that mirrors the teacher’s weak spots, directing essentially all of its repair budget to two specific classes — u in ATLAS, and H →4q in JetClass
The paper demonstrates that KD achieves a near-zero bg margin (−0.04 %) while A-CoRD achieves a positive mean margin (+0. p. 422 %) that concentrates on specific hard binary boundaries The gate gi has learned to not trust the teacher when the teacher is wrong, dropping by −0. p. 520 (ATLAS) and −0. p. 009 (JC) relative to the correct subset
The analysis reveals that A-CoRD helps when the downstream task is dominated by a specific hard binary boundary where the teacher’s per-class accuracy is sub-perfect KD suffices if the downstream task is dominated by average accuracy and the teacher’s per-class accuracies are well-matched to the desired operating points The reliability-coupled training/inference design appears, to our knowledge, novel in HEP distillation
The paper concludes that A-CoRD + PCG yields the deepest bg margin (+0.40%), while the learnable per-class βc of A-CoRD focuses the repair effort onto exactly the two classes where the teachers are statistically weakest The reliability-coupled training/inference design appears, to our knowledge, novel in HEP distillation and is broadly applicable beyond jet tagging — anywhere a heavyweight teacher is imperfectly calibrated and a cheap student must respect its mistakes
The paper notes that A-CoRD shows a small but reproducible 2% regression on H →qql¯ @0.99 and t → blν@0.995 (teacher class accuracies 98. p. 31 % and 98.6 % respectively) at the extreme tail of Ps/(Ps + Pb) This regression is a structural consequence of repairing teacher mistakes in representation space The paper suggests that a saturation mask on βc for classes where the teacher exceeds 97% accuracy and a residual student-head parameterisation are natural structural remedies The code and trained checkpoints are available at https://github. p. 0
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Adaptive Corrective Representation Distillation (A-CoRD) allows for a
per-sample reliability signal ri that simultaneously weights the distillation loss, controls prototype-based teacher repair, and anchors the inference-time gate gi.
This enables the student totreat the teacher’s mistakes as information rather than as ground truth
during training. -
The system can perform cross-experiment distillation by jointly training a single 82 k-parameter student from two heavyweight teachers with
incompatible detector responses and disjoint class spaces,
such as ATLAS GN2 and CMS ParT. -
The Reliability-Aware Inference fusion (RAI) architecture uses a convex mixture of teacher and student logits, where the gate gi is anchored to the training reliability signal ri, ensuring
the same notion of 'trust the teacher' applies at train and test time.
-
The system can dynamically allocate its repair budget by using per-class learnable coefficients beta c that
automatically contract with the teacher’s per-class accuracy,
focusing on correcting only the two weakest classes (u in ATLAS, H to 4q in JetClass). -
The final inference stage uses a convex mixture of logits s final i = gi · ti + (1 − gi) · s student i, allowing for a continuous mix where gi approaches 1 when the teacher is right and approaches 0 when it is wrong.
Sources
- Distilling the Knowledge in a Neural Network
- Accelerating Resonance Searches via Signature-Oriented Pre-training
Related papers
- MadNIS at NLO
- The NANOGrav 15-year data set: A Limit on Continuous Spin Gravitons
- DIS-secting the universe with high-energy neutrinos
- Crowdsourcing Gravitational Waves from Superradiant Axions
- Classification of g-modes for neutron stars with a strong transition: Novel universal relation including slow stable hybrid stars
- Scalar axion field of toroidal electromagnetic pulses