Scaling laws for amplitude surrogates
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: High Energy Physics: Phenomenology. The day in particle phenomenology: models, signatures and constraints.
Kai: I'm Kai.
Mira: And I'm Mira, here with Lev.
Kai: We're looking at the paper: "Scaling laws for amplitude surrogates".
Mira: Here's the gist of the paper. Scaling laws describing the dependence of neural network performance on the amount of training data, spent compute, and network size have emerged across a huge variety of machine learning task and datasets In this work, we systematically investigate these scaling laws in the context of amplitude surrogates for particle physics.
Kai: Scaling Laws in Deep Learning Context Universal approximation theorems state that a sufficiently wide or deep neural network can in principle approximate any continuous function with an arbitrary degree of precision However, reaching very high precision levels with a neural network often requires a large investment of training resources involving scaling up the data points Dtrain, the number of network parameters N and the compute C.
Mira: A simple solution to prevent this issue is to make use of the empirical scaling laws that parametrize the test loss as a set of power laws on the variables N, Dtrain and C.
Kai: The general expression for the scaling laws is L(X, Y, Z) = (Xc/X) αX + KX(Y, Z), where (X, Y, Z) denotes any ordering of the variables (N, Dtrain, C).
Title and authors: Kai: So, this paper, "Scaling laws for amplitude surrogates," by Bahl, Bresó-Pla, Butter, and Iturriza Ramirez from Heidelberg and Harvard, is looking at how neural networks perform when approximating scattering amplitudes.
Mira: They start by setting the stage with the universal approximation theorems in machine learning which say a wide or deep network can approximate any continuous function to any precision.
Kai: But they point out that reaching high precision often demands a massive investment in training resources, scaling up the data points Dtrain, the network parameters N, and the compute C.
Lev: And this is where their work comes in because they introduce empirical scaling laws that parametrize the test loss as power laws on those variables, which is a practical way to manage that resource trade-off.
Mira: They show that they can use these scaling laws to achieve desired precision targets, and the authors demonstrate this using a case study of the quark antiquark annihilation into a top antiquark and a Higgs boson production, qq¯ → t¯tH.
Kai: And they find that the intrinsic dimension of the input data manifold for these processes is always fixed by the number of degrees of freedom in our datasets, which matches the degrees of freedom in the interaction amplitude itself.
Lev: That means if you know how many external particles are involved in a process, you immediately know a fundamental property—the intrinsic dimension—which helps narrow down what kind of network we need.
The paper's summary: Kai: To summarize, the paper systematically investigates these scaling laws across a large variety of practically relevant processes that cover different interaction patterns in particle physics.
Mira: They observe that the qualitative scaling behavior is consistent across different processes, and they specifically see the slopes of the compute and training data curves decreasing as you increase the number of external particles.
Lev: That decrease in slope is what's important because it shows that for more complex interactions, you actually get better returns on your compute or data investment compared to simpler ones.
Kai: They also compare two different architectures they studied: a standard multi-layer perceptron that uses the four-vectors of the particles as input and a Lorentz Local Canonicalization Transformer.
Mira: They show that scaling laws across these different processes are largely dependent on the number of external particles, suggesting that this physical property is the main driver of performance scaling.
Lev: The paper also notes a practical implication: for processes like qq¯ → t¯tH, using a low-cost surrogate allows you to set an anchor point for those scaling curves using a small training dataset and low FLOPs.
The paper's improvements: Kai: The authors suggest two main improvements to make this tool more useful for researchers in the field. First, they want to be able to perform predictive resource estimation.
Mira: They propose that you can set an anchor point for the scaling curves by using a low-cost surrogate to predict exactly what resources you need on any physical process to hit a certain precision target.
Lev: That's useful because it moves us from just fitting the data to actually predicting how much GPU time or data storage we'll need before we even start training.
Kai: The second improvement focuses on how we use the established power law behavior for performance prediction across different quantities.
Mira: They suggest that if one of these resource quantities isn't large enough, increasing the other resources won't actually improve the performance further, which helps us understand where saturation happens.
Lev: So instead of blindly scaling up everything, we can use this framework to characterize the conditions under which we hit a performance ceiling.
Conclusion: Kai: So, wrapping up on "Scaling laws for amplitude surrogates," the paper shows that scaling coefficients are tied to the number of external particles and provides a useful tool for achieving precision targets in particle physics simulations.
Mira: The key finding is that we can predict resource needs using empirical scaling laws, and they specifically show how this behavior holds across diverse interaction patterns.
Lev: And from my side, it confirms that the intrinsic dimension being fixed by the number of external particles is a solid constraint for what kind of network structure we should be designing.
Kai: It really helps us think about building architectures, like using things such as the LLoCa-Transformer, which they show can be more stable when dealing with complex amplitudes with high permutation symmetry.
Mira: Overall, this work provides a way to get a conservative estimate for the resources needed to reach any desired precision target by setting those lower bounds on the scaling coefficients.
Lev: It's a good way to ground our resource planning in theoretical constraints derived from the physics of the interaction itself.
Institut für Theoretische Physik, University of Heidelberg · Department of Physics, Harvard University · LPNHE, Sorbonne University, CNRS/IN2P3
hep-ph, cs.LG
Submitted: 2026-01-19
Updated: 2026-11-05
Comments: 45 pages, 20 figures, v2: revised version
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 91/100
The gist: Scaling laws describing the dependence of neural network performance on the amount of training data, spent compute…
Key concepts
- Scaling Laws
- These are mathematical rules describing how the performance (loss) of a neural network changes as you increase training data (Dtrain), network size (N), or computational cost (C). They help estimate the resources required for high-precision approximations.
- Intrinsic Dimension
- This is the minimum number of independent variables needed to describe the local geometry of the data manifold. For amplitude surrogates, this dimension is fixed and determined solely by the number of degrees of freedom in the interaction amplitude, which relates directly to external particles.
- Amplitude Surrogates
- These are neural networks trained to approximate complex particle physics interaction amplitudes. The paper uses these as a tool to explore how computational resources scale with the complexity of the physical process being modeled.
- Power Laws
- The scaling laws are expressed as power laws, like L(X, Y, Z) = (Xc/X) αX + KX(Y, Z). These mathematical expressions allow for a systematic parametrization of how test loss depends on the variables N (size), Dtrain (data), and C (compute).
Terminology
Summary
Scaling laws describing the dependence of neural network performance on the amount of training data, spent compute, and network size have emerged across a huge variety of machine learning task and datasets In this work, we systematically investigate these scaling laws in the context of amplitude surrogates for particle physics. p. 0 SciPost Physics Submission
Scaling laws for amplitude surrogates
Henning Bahl1
, Victor Bresó-Pla2
, Anja Butter1,3, and Joaquín Iturriza Ramirez3
1 Institut für Theoretische Physik, Universität Heidelberg, Germany
2 Department of Physics, Harvard University, 02138 Cambridge, MA, USA
3 LPNHE, Sorbonne Université, Université Paris Cité, CNRS/IN2P3, Paris, France
Abstract
Scaling laws describing the dependence of neural network performance on the amount
of training data, the spent compute, and the network size have emerged across a huge
variety of machine learning task and datasets. In this work, we systematically investigate
these scaling laws in the context of amplitude surrogates for particle physics.Where the paper says it
Scaling Laws in Deep Learning Context
Universal approximation theorems state that a sufficiently wide or deep neural network can in principle approximate any continuous function with an arbitrary degree of precision However, reaching very high precision levels with a neural network often requires a large investment of training resources involving scaling up the data points Dtrain, the number of network parameters N and the compute C. A simple solution to prevent this issue is to make use of the empirical scaling laws that parametrize the test loss as a set of power laws on the variables N, Dtrain and C. p. 2 In the worst
case scenario, one of the variables being too small can result in a performance saturation that
cannot be overcome with any additional increase on the other parameters. A simple solution to prevent this issue is to make use of the empirical scaling laws that
parametrize the test loss of a neural network as a set of power laws on the variables N, Dtrain
and C. These laws represent a valuable tool to predict the training budget that is needed to
reach any performance level and also characterize the conditions under which saturation can
happen. The general expression for the scaling laws is:
L(X, Y, Z) = (Xc/X)
αX + KX
(Y, Z), (1)
where (X, Y, Z) denotes any ordering of the variables (N, Dtrain, C)*.Where the paper says it
Where the paper says it
How the Training Signal is Built
The intrinsic dimension can be trivially computed for each of the studied processes as a function of the number of particles in the final state nf through the equation d = 3nf − 4. The intrinsic dimension will always match the number of degrees of freedom of our datasets, which is clear if we consider the definition of the intrinsic dimension as the minimum number of variables needed to parametrize a local environment inside the data manifold. p. 4 This characterization for the scaling laws is very useful for amplitude surrogates, since
the intrinsic dimension will always match the number of degrees of freedom of the interaction
amplitude. This is clear if we consider the definition of the intrinsic dimension as the minimum
number of variables needed to parametrize a local environment inside the data manifold. As for the batch size, we observe in practice that the
scaling for our studied tasks does not depend on a range of batch sizes from 26
to 216
. This characterization for the scaling laws is very useful for amplitude surrogates, since
the intrinsic dimension will always match the number of degrees of freedom of the interaction
amplitude.Where the paper says it
Where the paper says it
Where the Gains Come From
The scaling coefficients are connected to the number of external particles involved in the process. The results demonstrate that scaling laws across different processes depend mostly on the number of external particles involved in the interaction. p. 1 We also determine the differences in scaling across architectures
by considering a simple multilayer perceptron (MLP) versus the more sophisticated Lorentz
Local Canonicalization Transformer (LLoCa-Transformer) [25,26]. As a result of our analysis,
we demonstrate that scaling laws across different processes depend mostly on the number of
external particles involved in the interaction. As for the batch size, we observe in practice that the
scaling for our studied tasks does not depend on a range of batch sizes from 26
to 216
. This characterization for the scaling laws is very useful for amplitude surrogates, since
the intrinsic dimension will always match the number of degrees of freedom of the interaction
amplitude. This behavior implies that whenever the training is limited by either Y or Z, network
performance will be saturated at the plateau induced by KX
(Y, Z) no matter how much X is
increased. On the flip side, once X becomes the only limiting factor during training, the loss
will depend exclusively on X and KX
(Y, Z) will converge to the fundamental lower bound of
the loss. We show
that the scaling coefficients are connected to the number of external particles of the
process. Our results demonstrate that scaling laws are a useful tool to achieve desired
precision targets.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Scaling Across Processes
We study a large variety of practically relevant processes that cover a diverse set of interaction patterns. We observe the same qualitative scaling behavior as with the qq¯ → t¯tH interaction for other processes, and we clearly see that the slopes of the C and Dtrain scaling curves decrease with the number of external particles, as expected from the lower bounds on the scaling coefficients shown in Eqs. p. 1 We also determine the differences in scaling across architectures
by considering a simple multilayer perceptron (MLP) versus the more sophisticated Lorentz
Local Canonicalization Transformer (LLoCa-Transformer) [25,26]. As a result of our analysis,
we demonstrate that scaling laws across different processes depend mostly on the number of
external particles involved in the interaction. from the underlying data manifold, but this does not seem to compromise the validity
of the lower bound in practice, as we will demonstrate below. A mathematical derivation for
classical models connecting the scaling coefficient to the smoothness of the target function can
be found in Ref.Where the paper says it
Where the paper says it
Model Comparison and Prediction
We study two architectures: a multi-layer perceptron that uses the momentum invariant pairs of the particle’s four-vectors as inputs (MLP-I) and the Lorentz Local Canonicalization (LLoCa) Transformer. We demonstrate that scaling laws across different processes depend mostly on the number of external particles involved in the interaction. p. 1 We also determine the differences in scaling across architectures
by considering a simple multilayer perceptron (MLP) versus the more sophisticated Lorentz
Local Canonicalization Transformer (LLoCa-Transformer) [25,26]. As a result of our analysis,
we demonstrate that scaling laws across different processes depend mostly on the number of
external particles involved in the interaction. 1 Network architectures
We study two architectures: a multi-layer perceptron that uses the momentum invariant pairs
of the particle’s four-vectors as inputs (MLP-I) and the Lorentz Local Canonicalization (LLoCa)
Transformer [25,26], which achieves Lorentz equivariance by transforming each particle’s four
vector into a local frame. Both of these networks are Lorentz-invariant by construction, which has been proven to
be an important property to maximise surrogate precision. These conditions set t¯tH production as a prime target for surrogate methods
specialized in achieving a high degree of precision under a very strict training data budget. Despite the effectiveness of neural networks at small training dataset sizes, it has been
clearly observed that the interpolation quality stops improving once the performance hit a
certain threshold (see Fig. We show
that the scaling coefficients are connected to the number of external particles of the
process. Our results demonstrate that scaling laws are a useful tool to achieve desired
precision targets.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Predictive Resource Estimation For amplitude surrogates can be performed by setting an anchor point for scaling curves using a low-cost surrogate to predict
the necessary resources needed to reach a certain precision target on any physical process.
-
Model performance prediction can be made using the established power law behavior where
If one of these quantities is not sufficiently large, increasing the other resources will not further improve the performance.
-
Surrogate training budgets can be determined by analyzing scaling laws such as
L(X, Y, Z) = (Xc/X) αX + KX (Y, Z), (1)
to predictthe training budget that is needed to reach any performance level and also characterize the conditions under which saturation can happen.
-
The intrinsic dimension of the input data manifold can be estimated by using the twoNN method, which allows for
a simple method to predict the minimum amount of resources needed to reach a certain surrogate precision target on any physical process
based on the fact thatthe intrinsic dimension is known and completely fixed by the number of external particles.
-
Architectures like LLoCa-Transformer can be used to boost performance when dealing with complex amplitudes that feature high degrees of permutation symmetry, leading to
a more stable scaling against hyperparamater changes
in specific processes.
Abstract
Scaling laws describing the dependence of neural network performance on the amount of training data, the spent compute, and the network size have emerged across a huge variety of machine learning task and datasets. In this work, we systematically investigate these scaling laws in the context of amplitude surrogates for particle physics. We show that the scaling coefficients are connected to the number of external particles of the process. Our results demonstrate that scaling laws are a useful tool to achieve desired precision targets.
Sources
- Machine Learning and LHC Event Generation
- Modern Machine Learning for LHC Physicists
- Exploring phase space with Neural Importance Sampling
- i-flow: High-dimensional Integration and Sampling with Normalizing Flows
- Event Generation with Normalizing Flows
- MadNIS -- Neural Multi-Channel Importance Sampling
- Efficient phase-space generation for hadron collider event simulation
- The MadNIS Reloaded
- Accelerating HEP simulations with Neural Importance Sampling
- Differentiable MadNIS-Lite
- Sampling NNLO QCD phase space with normalizing flows
- Efficient many-jet event generation with Flow Matching
- (Machine) Learning amplitudes for faster event generation
- Using neural networks for efficient evaluation of high multiplicity scattering amplitudes
- Optimising simulations for diphoton production at hadron colliders using amplitude neural networks
- A factorisation-aware Matrix element emulator
- Targeting Multi-Loop Integrals with Neural Networks
- Loop Amplitudes from Precision Networks
- Unweighting multijet event generation using factorisation-aware neural networks
- One-loop matrix element emulation with factorisation awareness
Related papers
- MadNIS at NLO
- The NANOGrav 15-year data set: A Limit on Continuous Spin Gravitons
- DIS-secting the universe with high-energy neutrinos
- Crowdsourcing Gravitational Waves from Superradiant Axions
- Classification of g-modes for neutron stars with a strong transition: Novel universal relation including slow stable hybrid stars
- Scalar axion field of toroidal electromagnetic pulses