Towards anomaly detection searches for new physics signatures including Higgs bosons with weakly supervised machine learning

summary

The gist

The Higgs And X Anomaly Detection (HAXAD) strategy offers a principled approach to searching for anomalies occurring in association with a Higgs boson by combining machine-learning-based feature…

In short

The HAXAD strategy searches for new physics anomalies associated with Higgs boson decays by combining machine learning feature embeddings with background estimation and weakly supervised classification. It introduces two novel embedding methods—an unsupervised VAE and a semi-supervised contrastive encoder—to create latent spaces that separate signal from background, enabling more precise anomaly detection.

Key concepts

Unsupervised Embedding (VAE)
This method uses a Variational Autoencoder trained on simulated continuum diphoton backgrounds and SM Higgs processes. It learns an event embedding by predicting the mean and variance of a seven-dimensional latent space, which captures features from 19 event-level inputs, including jet information and lepton flavor.
Semi-supervised Embedding (Contrastive Encoder)
This encoder uses a particle transformer architecture to build a six-dimensional latent space. It is trained using a contrastive loss that pulls events belonging to the same process closer together and pushes different processes apart, creating clear signal-background separation for anomaly detection.
Background Estimation
The background in the signal region is modeled by combining a non-resonant component, described by a normalizing flow model with six rational quadratic spline layers, and a resonant component from SM Higgs production. This combined estimate is used to train the final classifier.
Weakly Supervised Classification
Boosted Decision Trees (BDTs) are trained on the latent features derived from the embeddings. These classifiers distinguish between the estimated background and pseudo-data, utilizing a weighted binary cross-entropy loss to identify potential anomalies.

Terminology used across episodes

This episode discusses

The paper

Towards anomaly detection searches for new physics signatures including Higgs bosons with weakly supervised machine learning · Read on arXiv

Chi Lung Cheng, Julia Gonski, Julie Khalilieh Romman, Runze Li, Qibin Liu, Benjamin Nachman, Dennis Noll, Liangyu Wua

Fundamental Physics Directorate, SLAC National Laboratory · Department of Particle Physics and Astrophysics, Stanford University

Transcript

Introduction to the show: ident: High Energy Physics: Experiment. The day in high-energy experiment: detectors, searches and limits.

Kai: I'm Kai.

Mira: And I'm Mira, here with Lev.

Kai: We're looking at the paper: "Towards anomaly detection searches for new physics signatures including Higgs bosons with weakly supervised machine learning".

Mira: Here's the gist of the paper. The Higgs And X Anomaly Detection (HAXAD) strategy offers a principled approach to searching for anomalies occurring in association with a Higgs boson by combining machine-learning-based feature embedding, background estimation, and weakly supervised classification The discovery of new Beyond the Standard Model (BSM) physics is a key objective of research at high energy particle colliders Higgs boson final states provide a particularly generic and thus compelling handle for AD at colliders As the only fundamental scalar and a field whose couplings set the masses of Standard Model (SM) particles, the Higgs boson naturally acts as a portal through which many BSM sectors can communicate with the SM Moreover, several Higgs decay–1– modes such as H → γγ offer clean reconstruction and good mass resolution, enabling precise comparisons between data and reference distributions and thereby amplifying the reach of AD methods that search for localized or subtle discrepancies in high-dimensional feature space Embedding Methods The work introduces entirely new embedding strategies for the first stage, which in turn shape the background estimation and classification performed in the defined latent space Two new embedding methods are considered which both improve the performance of the entire HAXAD pipeline Unsupervised Embedding: This method is a fully unsupervised VAE, trained on MC simulations of the continuum diphoton background together with the SM Higgs boson processes The training sample consists of the three continuum components (γγ+jets, ttγγ ¯, and llγγ+jets) with diphoton invariant mass in the full analysis window 105 < mγγ < 145 GeV, together with the four SM Higgs production modes (ggH, VBF, V H, and ttH¯) restricted to the signal region 120 < mγγ < 130 GeV The model acts on 19 event-level features, which include the same nine input features used in the previous VAE model plus pT, η, ϕ, and flavor of the leading two leptons, and a b-tag identification flag for the leading two jets The encoder predicts the mean and variance of a diagonal Gaussian posterior over a seven-dimensional latent space, from which the latent vector is sampled via reparameterization The VAE training loss consists of a mean-squared-error reconstruction term for the continuous features and a binary cross-entropy term for the binary flags, combined with the Kullback-Leibler divergence (KL) divergence of the latent posterior with respect to a standard normal prior, weighted by β = 0.1 The training sample is randomly split into training and validation sets containing 80% and 20% of the events, respectively The model is trained with a batch size of 4096, using the Adam optimizer with an initial learning rate of 10−3 and cosine annealing to 10−6 over 300 epochs For each event, a single latent vector sampled from the encoder posterior serves as the event embedding used by all downstream anomaly detection stages, so that the embedding also carries the width of the learned posterior Semi-supervised Embedding: This method features the contrastive encoder design introduced in Ref.

43: , which builds a semi-supervised latent space for weakly supervised anomaly detection The encoder body is based on a particle transformer architecture and has 1.1M trainable parameters The inputs of this encoder are the same as described in Ref.

43: , consisting of both low-level and high-level features The output block of the encoder is inspired by the VAE, which predicts a mean and variance from which the latent vector is sampled via reparameterization A six-dimensional latent space is used, following the same design as in Ref.

43: During training, a two-layer MLP projection head is applied after the output block and discarded at inference time, a practice that has been shown to improve the quality of the learned embedding The contrastive encoder is trained with the loss: L = Lcon + λ KL(z ∥ N (0, 0.1)) with Lcon = X i∈I −1 P(i) X p∈P(i) log exp(Zi · Zp/τ) P a∈A(i) exp(Zi · Za/τ) The contrastive loss temperature is set to τ = 0.1 and the KL weight λ is set to be 0.1 By design, the contrastive loss in Eq.

Title and authors: Kai: So we're looking at this paper called "Towards anomaly detection searches for new physics signatures including Higgs bosons with weakly supervised machine learning". We just talked about what HAXAD is, so now let's talk about what the authors are actually trying to achieve with this specific title and their approach.

Mira: They are focusing on developing a principled way to search for anomalies when you have a Higgs boson present, using machine learning techniques that don't require perfect labels.

Lev: It’s interesting they specifically mention weakly supervised classification because, as we know from the literature, getting clean labels for these rare processes is tough.

Kai: They are building this pipeline by combining three main things: feature embedding based on machine learning, background estimation, and then using a weakly supervised classifier to sort the data.

Mira: That's the core mechanism they propose for their search strategy.

The paper's summary: Kai: To summarize this paper, they introduce entirely new embedding strategies for the first stage that then shape how we estimate the background and classify things in a latent space.

Mira: They propose two distinct embedding methods to improve the whole pipeline, one unsupervised and one semi-supervised.

Lev: The unsupervised method is a fully unsupervised Variational Autoencoder, trained on Monte Carlo simulations of continuum diphoton backgrounds alongside SM Higgs production processes.

Kai: This VAE is trained on nineteen event-level features, including input features like p T, eta, phi, and the flavor of the leading two leptons.

Mira: The encoder predicts a mean and variance for a seven-dimensional latent space, which they then sample from to get an event embedding.

Lev: And that latent vector carries the width of the learned posterior with it, which is something that would be important when we think about running this on real hardware.

Kai: The semi-supervised method uses a contrastive encoder design, inspired by prior work, and builds a semi-supervised latent space using particle transformer architecture.

Mira: This encoder is trained with a specific contrastive loss function that pulls events from the same process together and pushes different processes apart in the six-dimensional latent space.

The paper's improvements: Kai: The paper highlights some key improvements over existing methods, especially concerning how they handle the training data and signal separation.

Mira: They point out that their unsupervised VAE trains on a large set of simulated training samples, which includes eight million events from the continuum gamma gamma background and two point nine million events from SM Higgs boson processes.

Lev: That is a significant sample size for training an unsupervised model, and it shows how they’re trying to capture the continuous features alongside the discrete flags.

Kai: They also show that the semi-supervised encoder has robustness for unseen signals by testing its extrapolation capability, meaning it works even if you have mass parameters not in the training sample.

Mira: That's important because it shows a capability beyond just fitting known data patterns; it suggests a better way to handle physics we haven't seen before.

Lev: From an engineering standpoint, that robustness is key for real hardware deployment, showing that the latent space separation isn't just overfitting to the training set.

Conclusion: Kai: So wrapping up this paper on "Towards anomaly detection searches for new physics signatures including Higgs bosons with weakly supervised machine learning," the authors show how combining these embedding and classification steps creates a framework for searching for BSM signals.

Mira: The main implication is that they provide a principled approach to AD that doesn't strictly require perfect labels on training instances, moving beyond just full labeled training sets.

Lev: For running this on real hardware, the key challenge will be managing the computational cost of both these complex embedding methods and then running the downstream weakly supervised classifier efficiently.

Kai: So, we get model independent limits by constraining any anomalous production without reference to a specific signal hypothesis, which is pretty powerful for discovery searches.

Mira: They also discuss deriving model dependent limits through a signal injection scan procedure, which gives a more realistic constraint on BSM production cross sections.

Lev: That injection scan part sounds like it would be computationally intensive but provides the most realistic constraint on what we might actually see in an experiment.

Kai: So, the paper lays out a clear path for using these machine learning tools to search for Higgs anomalies, and it sets up a way to get limits that are both model independent and model dependent.

More episodes

← Home