Scaling laws for amplitude surrogates
summary
The gist
Scaling laws describing the dependence of neural network performance on the amount of training data, spent compute…
In short
This work investigates scaling laws for neural network performance when approximating particle physics amplitudes using amplitude surrogates. It finds that these scaling laws depend mainly on the number of external particles in a process. This allows researchers to predict necessary training data, compute, and network size needed to achieve specific precision targets.
Key concepts
- Scaling Laws
- These are mathematical rules describing how the performance (loss) of a neural network changes as you increase training data (Dtrain), network size (N), or computational cost (C). They help estimate the resources required for high-precision approximations.
- Intrinsic Dimension
- This is the minimum number of independent variables needed to describe the local geometry of the data manifold. For amplitude surrogates, this dimension is fixed and determined solely by the number of degrees of freedom in the interaction amplitude, which relates directly to external particles.
- Amplitude Surrogates
- These are neural networks trained to approximate complex particle physics interaction amplitudes. The paper uses these as a tool to explore how computational resources scale with the complexity of the physical process being modeled.
- Power Laws
- The scaling laws are expressed as power laws, like L(X, Y, Z) = (Xc/X) αX + KX(Y, Z). These mathematical expressions allow for a systematic parametrization of how test loss depends on the variables N (size), Dtrain (data), and C (compute).
Terminology used across episodes
This episode discusses
- Scaling laws for amplitude surrogates · Paper Radio
- Machine Learning and LHC Event Generation
- Modern Machine Learning for LHC Physicists
- Exploring phase space with Neural Importance Sampling
- i-flow: High-dimensional Integration and Sampling with Normalizing Flows
- Event Generation with Normalizing Flows
- MadNIS -- Neural Multi-Channel Importance Sampling
- Efficient phase-space generation for hadron collider event simulation
- The MadNIS Reloaded
- Accelerating HEP simulations with Neural Importance Sampling
- Differentiable MadNIS-Lite
- Sampling NNLO QCD phase space with normalizing flows
- Efficient many-jet event generation with Flow Matching
- (Machine) Learning amplitudes for faster event generation
- Using neural networks for efficient evaluation of high multiplicity scattering amplitudes
- Optimising simulations for diphoton production at hadron colliders using amplitude neural networks
- A factorisation-aware Matrix element emulator
- Targeting Multi-Loop Integrals with Neural Networks
- Loop Amplitudes from Precision Networks
- Unweighting multijet event generation using factorisation-aware neural networks
- One-loop matrix element emulation with factorisation awareness
The paper
Scaling laws for amplitude surrogates · Read on arXiv
Institut für Theoretische Physik, University of Heidelberg · Department of Physics, Harvard University · LPNHE, Sorbonne University, CNRS/IN2P3
Scaling laws describing the dependence of neural network performance on the amount of training data, the spent compute, and the network size have emerged across a huge variety of machine learning task and datasets. In this work, we systematically investigate these scaling laws in the context of amplitude surrogates for particle physics. We show that the scaling coefficients are connected to the number of external particles of the process. Our results demonstrate that scaling laws are a useful tool to achieve desired precision targets.
Transcript
Introduction to the show: ident: High Energy Physics: Phenomenology. The day in particle phenomenology: models, signatures and constraints.
Kai: I'm Kai.
Mira: And I'm Mira, here with Lev.
Kai: We're looking at the paper: "Scaling laws for amplitude surrogates".
Mira: Here's the gist of the paper. Scaling laws describing the dependence of neural network performance on the amount of training data, spent compute, and network size have emerged across a huge variety of machine learning task and datasets In this work, we systematically investigate these scaling laws in the context of amplitude surrogates for particle physics.
Kai: Scaling Laws in Deep Learning Context Universal approximation theorems state that a sufficiently wide or deep neural network can in principle approximate any continuous function with an arbitrary degree of precision However, reaching very high precision levels with a neural network often requires a large investment of training resources involving scaling up the data points Dtrain, the number of network parameters N and the compute C.
Mira: A simple solution to prevent this issue is to make use of the empirical scaling laws that parametrize the test loss as a set of power laws on the variables N, Dtrain and C.
Kai: The general expression for the scaling laws is L(X, Y, Z) = (Xc/X) αX + KX(Y, Z), where (X, Y, Z) denotes any ordering of the variables (N, Dtrain, C).
Title and authors: Kai: So, this paper, "Scaling laws for amplitude surrogates," by Bahl, Bresó-Pla, Butter, and Iturriza Ramirez from Heidelberg and Harvard, is looking at how neural networks perform when approximating scattering amplitudes.
Mira: They start by setting the stage with the universal approximation theorems in machine learning which say a wide or deep network can approximate any continuous function to any precision.
Kai: But they point out that reaching high precision often demands a massive investment in training resources, scaling up the data points Dtrain, the network parameters N, and the compute C.
Lev: And this is where their work comes in because they introduce empirical scaling laws that parametrize the test loss as power laws on those variables, which is a practical way to manage that resource trade-off.
Mira: They show that they can use these scaling laws to achieve desired precision targets, and the authors demonstrate this using a case study of the quark antiquark annihilation into a top antiquark and a Higgs boson production, qq¯ → t¯tH.
Kai: And they find that the intrinsic dimension of the input data manifold for these processes is always fixed by the number of degrees of freedom in our datasets, which matches the degrees of freedom in the interaction amplitude itself.
Lev: That means if you know how many external particles are involved in a process, you immediately know a fundamental property—the intrinsic dimension—which helps narrow down what kind of network we need.
The paper's summary: Kai: To summarize, the paper systematically investigates these scaling laws across a large variety of practically relevant processes that cover different interaction patterns in particle physics.
Mira: They observe that the qualitative scaling behavior is consistent across different processes, and they specifically see the slopes of the compute and training data curves decreasing as you increase the number of external particles.
Lev: That decrease in slope is what's important because it shows that for more complex interactions, you actually get better returns on your compute or data investment compared to simpler ones.
Kai: They also compare two different architectures they studied: a standard multi-layer perceptron that uses the four-vectors of the particles as input and a Lorentz Local Canonicalization Transformer.
Mira: They show that scaling laws across these different processes are largely dependent on the number of external particles, suggesting that this physical property is the main driver of performance scaling.
Lev: The paper also notes a practical implication: for processes like qq¯ → t¯tH, using a low-cost surrogate allows you to set an anchor point for those scaling curves using a small training dataset and low FLOPs.
The paper's improvements: Kai: The authors suggest two main improvements to make this tool more useful for researchers in the field. First, they want to be able to perform predictive resource estimation.
Mira: They propose that you can set an anchor point for the scaling curves by using a low-cost surrogate to predict exactly what resources you need on any physical process to hit a certain precision target.
Lev: That's useful because it moves us from just fitting the data to actually predicting how much GPU time or data storage we'll need before we even start training.
Kai: The second improvement focuses on how we use the established power law behavior for performance prediction across different quantities.
Mira: They suggest that if one of these resource quantities isn't large enough, increasing the other resources won't actually improve the performance further, which helps us understand where saturation happens.
Lev: So instead of blindly scaling up everything, we can use this framework to characterize the conditions under which we hit a performance ceiling.
Conclusion: Kai: So, wrapping up on "Scaling laws for amplitude surrogates," the paper shows that scaling coefficients are tied to the number of external particles and provides a useful tool for achieving precision targets in particle physics simulations.
Mira: The key finding is that we can predict resource needs using empirical scaling laws, and they specifically show how this behavior holds across diverse interaction patterns.
Lev: And from my side, it confirms that the intrinsic dimension being fixed by the number of external particles is a solid constraint for what kind of network structure we should be designing.
Kai: It really helps us think about building architectures, like using things such as the LLoCa-Transformer, which they show can be more stable when dealing with complex amplitudes with high permutation symmetry.
Mira: Overall, this work provides a way to get a conservative estimate for the resources needed to reach any desired precision target by setting those lower bounds on the scaling coefficients.
Lev: It's a good way to ground our resource planning in theoretical constraints derived from the physics of the interaction itself.
More episodes
- 2610.38463-Gauge-parameter cancellation and finite form factors in one-loop quark scattering in the refined Gribov-Zwanziger framework
- 2610.37198-Searches for new particle physics by methods of classical and neutrino astronomy
- 2511.15779-SURFing to the Fundamental Limit of Jet Tagging
- 2505.01492-Photo- and Hadrodisintegration constraints on massive relics decaying into neutrinos
- 2505.20389-Dispersion relation of the neutrino plasma: Unifying fast, slow, and collisional instabilities
- 2511.10731-Keeping It Renormalizable: Minimal Baryogenesis induced Asymmetric Dark Matter
- 2404.04021-Vacuum correlations of the stress-energy-momentum tensor with constituent quarks
- 2503.02931-Relativistic Navier-Stokes description of the quark-gluon plasma radial flow
- 2606.06105-Bulk viscosity of a binary mixture: the role of the intra-species interaction
- 2607.03343-First look at the evaluation of two-loop Feynman integrals for radiative return processes