Probabilistic, Structure-Intrinsic Clustering with a Hierarchical Embedding (PSICHE) in Time and Space Applied In Particle Physics Jet Reconstruction

arXiv:2609.16276 · hep-ex, hep-ph · Submitted 2026-09-14 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.

Kai: I'm Kai.

Mira: And I'm Mira, here with Lev.

Kai: We're looking at the paper: "Probabilistic, Structure-Intrinsic Clustering with a Hierarchical Embedding (PSICHE) in Time and Space Applied In Particle Physics Jet Reconstruction".

Mira: Here's the gist of the paper. Probabilistic, Structure-Intrinsic Clustering with an Hierarchical Embedding (PSICHE) in Time and Space Applied In Particle Physics Jet Reconstruction introduces a novel algorithm that addresses challenges in jet reconstruction by combining a variational Gaussian mixture model with Bayesian hierarchical clustering to learn dynamic jet sizes, multi-scale substructure, and spatiotemporal clustering Algorithm Foundation PSICHE is a probabilistic clustering algorithm that integrates the recursive tree structure of Bayesian Hierarchical Clustering with an embedded variational mixture model to learn high-level associations through the overarching tree structure while symbiotically resolving lower-level substructure with the mixture model The algorithm utilizes a variational Gaussian mixture model (vGMM) to capture local information as clusters within a self-contained group of observations Multiple vGMMs are embedded into a higher-level, treeconsistent model for multi-scale learning, allowing for structure at multiple scales to be simultaneously learned in a self-consistent framework Structure Learning Mechanisms The algorithm incorporates several notable algorithmic additions beyond the core combination of vGMM and BHC These additions include: 1.

Kai: Observational Data Precisions and Weights, which are incorporated into the model likelihood to encode uncertainties and relative importance of data points.

Kai: Subcluster Initialization and Regulation via Hierarchical Inheritance, where the number of subclusters is set by the posteriors from parent vGMMs in a merge, regulating subcluster multiplicity.

Paper summary: Mira: So looking at the paper "Probabilistic, Structure-Intrinsic Clustering with a Hierarchical Embedding (PSICHE) in Time and Space Applied In Particle Physics Jet Reconstruction," the authors are essentially proposing this framework to solve the problems we face when reconstructing jets.

Kai: The core idea is using PSICHE to dynamically learn jet sizes and multiplicity while simultaneously capturing multi-scale substructure through a combination of Bayesian Hierarchical Clustering and a variational mixture model. Lev It’s trying to do the hard part: learning these complex associations in a single probabilistic framework instead of doing them separately.

Mira: What this implies for the field is that we might be able to get a better handle on jet substructure even when dealing with the real mess of pileup contamination and varying jet sizes. Kai It moves beyond just looking at one fixed scale of clustering; it allows for learning structure across different scales in a self-consistent way.

Lev: From an engineering view, it suggests that if we can implement this inference on real hardware, we get a method to handle the inherent uncertainties and dynamic nature of these measurements in a way that respects the underlying physics constraints like energy conservation.

Mira: And the authors flag some limitations themselves; they mention needing more than two clustering scales beyond a simple variational mixture model to avoid issues like non-identifiability. Kai So while it’s a solid approach for jet reconstruction, we still have to be careful about the assumptions baked into those models and what information we can actually extract from them.

Conclusion: Kai: So, we’re wrapping up our look at PSICHE now. The title itself is pretty dense, right? Probabilistic, Structure-Intrinsic Clustering with a Hierarchical Embedding in Time and Space Applied In Particle Physics Jet Reconstruction.

Mira: Yeah it sounds like a lot of math packed into one name but what the authors are actually doing is taking jet reconstruction and making it probabilistic through this clustering method.

Kai: Right, they’re using this recursive tree structure from Bayesian Hierarchical Clustering combined with a variational mixture model to learn how jets look across different scales in both time and space.

Lev: From an engineering standpoint, they’re tackling the dynamic nature of jets—how their size and multiplicity change—and trying to figure out that multi-scale substructure simultaneously.

Mira: And they do this by using observational data precisions as weights and regulating the number of subclusters based on how parent models merge, which is a clever way to manage complexity.

Kai: So, what does this actually mean for the world when you take these ideas out of the particle physics lab and apply them to real detector data?

Lev: It means we could potentially mitigate pileup contamination better. The paper suggests a way to filter out subclusters that don't fit the expected resolved structure in both time and space based on their probability of being correct.

Mira: It shifts jet reconstruction from being a fixed-scale problem to something dynamic where you can probe structure at multiple levels all at once.

Kai: It’s about moving away from a single snapshot of what the jet looks like to understanding the whole history and scale of that object.

Lev: And if we can get those learned observables, we might actually be able to use them for things like jet tagging or even searching for new physics scenarios, like hidden valley situations.

Mira: It’s a solid framework but they also point out that to avoid mathematical headaches with non-identifiability, you need to go beyond just two clustering scales.

Kai: So the takeaway is this PSICHE method offers a new way to handle the complexity of jet substructure by combining hierarchical structure learning with local probabilistic modeling.

Lev: It’s a big leap from standard methods, but we still have those limitations they mentioned about needing more than two scales to keep things stable.

University of Kansas

hep-ex, hep-ph

Submitted: 2026-09-14

Updated: 2026-11-03

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: Probabilistic, Structure-Intrinsic Clustering with an Hierarchical Embedding (PSICHE) in Time and Space Applied In Particle Physics Jet Reconstruction introduces a novel algorithm that addresses…

Key concepts

Variational Gaussian Mixture Model (vGMM)
A vGMM is a statistical tool used to capture local information within a group of observations. It models data as a mixture of several Gaussian distributions, allowing the algorithm to identify distinct, localized clusters or substructures in the particle data.
Bayesian Hierarchical Clustering (BHC)
BHC uses a recursive tree structure to learn high-level associations across multiple scales. In this context, it helps organize the complex jet data into a hierarchical framework where larger structures are built from smaller, more local groupings.
Structure-Intrinsic Clustering
This approach means that the algorithm learns the underlying physical structure directly from the data's inherent properties rather than relying on pre-defined models. It seeks to discover how particle emissions naturally group together across different spatial and temporal scales.
Hierarchical Embedding
This refers to embedding a lower-level clustering model, like the vGMM, into a higher-level tree structure (the BHC). This allows the algorithm to simultaneously learn fine substructure within clusters while maintaining a consistent, multi-scale organization of the entire system.

Terminology

Summary

Probabilistic, Structure-Intrinsic Clustering with an Hierarchical Embedding (PSICHE) in Time and Space Applied In Particle Physics Jet Reconstruction introduces a novel algorithm that addresses challenges in jet reconstruction by combining a variational Gaussian mixture model with Bayesian hierarchical clustering to learn dynamic jet sizes, multi-scale substructure, and spatiotemporal clustering

Algorithm Foundation

PSICHE is a probabilistic clustering algorithm that integrates the recursive tree structure of Bayesian Hierarchical Clustering with an embedded variational mixture model to learn high-level associations through the overarching tree structure while symbiotically resolving lower-level substructure with the mixture model The algorithm utilizes a variational Gaussian mixture model (vGMM) to capture local information as clusters within a self-contained group of observations Multiple vGMMs are embedded into a higher-level, treeconsistent model for multi-scale learning, allowing for structure at multiple scales to be simultaneously learned in a self-consistent framework

Structure Learning Mechanisms

The algorithm incorporates several notable algorithmic additions beyond the core combination of vGMM and BHC These additions include:

  1. Observational Data Precisions and Weights, which are incorporated into the model likelihood to encode uncertainties and relative importance of data points. p. 4

    Where the paper says it

    Observational Measurement Precisions and Weights In order to accommodate experimental variations among observations, the vGMM likelihood, introduced in Equation 1, is augmented with terms that dictate both the overall importance of an observation within the model, as well as its importance relative to others. The concept of observational importance is incorporated in PSICHE by associating data weights Ω ≡ {ωn} and measurement precisions (inverse covariances) Λ∗ ≡ {Λ ∗ n} with all observations X ≡ xn.

  2. Subcluster Initialization and Regulation via Hierarchical Inheritance, where the number of subclusters is set by the posteriors from parent vGMMs in a merge, regulating subcluster multiplicity. p. 5

    Where the paper says it

    Specifically, the number of subclusters and their starting exponential-family distributions are set by the posteriors from the parent vGMMs considered in a merge. A representation of this inheritance scheme is illustrated in Figure 5.

  3. Subcluster Regulation via Merged Model Comparisons, which uses a measure of “overlap” between subclusters to decide on potential mergers, ensuring parsimony. p. 5

    Where the paper says it

    Whether the assumption on subcluster multiplicity is sensible in all cases and domains is debatable, but here it can be desirable to temper unnecessary complexity in unsupervised learning by ensuring that the inductive, tree-consistent structure of the BHC is consistently communicated to vGMMs. Subcluster Regulation via Merged Model Comparisons Even with this inheritance scheme for vGMM initializations, it is not necessarily the case that mixture models will converge to global optima, particularly regarding the subcluster multiplicity degree of freedom.

Physical Translation and Constraints

The probabilistic model is translated into physical jets by applying the BHC clusters to physically-meaningful groupings of subatomic particle emissions and depositions, where the BHC clusters correspond to physically-meaningful groupings of subatomic particle emissions and depositions – jets – following from the collimated decay of a single particle whose identity and properties may be of interest The model incorporates domain-specific inductive biases such as transforming observations into a local jet reference frame to mimic the projective geometry of particle detectors Furthermore, the choice of weights, where weights should be proportional to energy, is crucial for preserving four-vector summation and providing infrared (IR) and collinear (C) safety.

Application in Jet Reconstruction

PSICHE is applied to jet clustering by grouping emissions into jets corresponding to exclusive or hard assignments of the BHC partitions, with further jet subclustering performed by the embedded vGMM, allowing for probabilistic or soft assignments between subclusters The algorithm learns dynamic jet sizes and multiplicity simultaneously, and its timing-aware design allows it to study the deleterious effects of pileup contamination in jets A novel technique for pileup mitigation is presented by removing subclusters inconsistent with the expected resolved, two-pronged, time-coherent structure based on their probabilistic responsibilities

Future Directions

Future developments include incorporating more than two clustering scales beyond a vGMM embedded into a BHC tree structure to avoid non-identifiability and singularities Replacing normal distributions in the vGMM with alternative, less generic likelihood models for observations could incorporate even more information about low-level detector elements The paper also suggests leveraging learned jet and subcluster observables for jet tagging and BSM physics searches, such as Hidden Valley scenarios.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Dynamic jet size learning: The improved AI system can dynamically learn variable jet sizes without having to a priori specify any of these algorithmic inputs, as PSICHE inherently accommodates jets of varying sizes and types.

  2. Multi-scale feature learning: The system can perform unsupervised, multi-scale learning of emergent features like jet substructure and multiplicity by embedding multiple variational Gaussian mixture models into a tree-based hierarchical model.

  3. Spatiotemporal clustering: The AI can cluster in space and time by incorporating experimental resolutions as precisions among the clustering-domain dimensions, natively including precision timing measurements of emissions and depositions.

  4. Uncertainty incorporation: The system can incorporate domain specific experimental uncertainties through observational data precisions and weights, allowing it to model the inclusion of observational uncertainties into the model likelihood.

  5. Pileup mitigation: The system can employ a pileup mitigation technique leveraging PSICHE’s dynamic, multi-scale, spatiotemporal clustering to recover expected resonances even in extreme pileup conditions.

  6. Jet characterization: The improved AI system can characterize jets across multiple scales by using the subcluster posterior sufficient statistics to describe the jet as a whole for kinematic characterization across multiple jet scales.

  7. Jet tagging: The system can leverage posterior sufficient statistics of the vGMM for jet tagging, using features like prong multiplicity and shape to identify specific jet types.

  8. BSM physics searches: The AI can elucidate non-trivial substructure in jets relevant to Beyond Standard Model physics by exploiting timing-awareness and co-learning multiple scales, which is crucial for searching for Hidden Valley scenarios.

Abstract

Jet reconstruction is an active and open area of particle physics research, with questions related to jet size, multiplicity, substructure, and experimental performance in the presence of pileup and noise. Analogous challenges are ubiquitous throughout other applications of unsupervised learning in the context of clustering and association. This paper introduces a new algorithm for clustering in the context of jet reconstruction: Probabilistic, Structure-Intrinsic Clustering with a Hierarchical Embedding (PSICHE). PSICHE addresses several challenges and shortcomings of existing jet clustering methods with novel features, including but not limited to: (i) dynamically-learned and variable jet (cluster) sizes, (ii) unsupervised, multi-scale learning of emergent features, like jet substructure and multiplicity, (iii) clustering in space and time, and (iv) incorporation of domain-specific experimental uncertainties. All of these properties are achieved in a self-consistent, probabilistic framework that is computationally tractable. Furthermore, demonstrations of these features are presented for a range of jet clustering examples in LHC scenarios for various physics phenomena, clustering inputs, pileup conditions, and detector performance parameters.

Related papers