Learning Scattering Amplitudes with Transformer Reinforcement Learning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: High Energy Physics: Theory. The day in high-energy theory: gravity, strings, fields and the questions left open.
Kai: We're looking at the paper: "Learning Scattering Amplitudes with Transformer Reinforcement Learning".
Mira: Here's the gist of the paper. We introduce a transformer reinforcement learning algorithm that learns to solve high loop-level scattering amplitudes in planar N = 4 Super Yang-Mills theory Our algorithm improves on previous transformer-only results by incorporating previously derived symmetries and relationships into the learning algorithm This results in a greatly decreased fraction of the solution that needs to be known a priori to solve the entire problem An additional benefit is that our algorithm also ensures that every output obeys the set of known relationships Rather than predicting all coefficients independently, the model proposes assignments that are propagated through exact linear relations, while MCTS searches over assignments when propagation alone is insufficient This is crucial to the generalization of machine learning approaches to higher loops, as without this, there is no way to overcome the factorially scaling of state sizes and compare to results derived via other methods.
Kai: Problem Formulation in QFT Scattering amplitudes are central objects in quantum field theory (QFT), encoding transition probabilities between asymptotic particle states Planar N = 4 Super Yang-Mills (SYM) theory has emerged as a key testing ground for such developments due to its high degree of symmetry and simplified analytic structure The difficulty of traditional perturbative computations based on Feynman diagrams scales factorially with both loop order and the number of external particles, motivating the development of alternative frameworks Recent work has shown that the symbolic structure of amplitudes can be treated as a sequence modeling problem, making it amenable to modern machine learning approaches such as transformers The shortcomings of this method are that it requires a vast majority of the final answer to be known so that it may be used as training data Furthermore, if one only greedily samples the resulting probability distribution, the outcome is almost guaranteed to violate the set of known relationships.
Mira: Symbolic Representation and Constraints The problem is framed as learning a mapping between discrete sequences and integer coefficients using the symbology representation of the form factor The L-loop symbol takes the form S
F(L): = X li1, …, li2L Cli1, …, li2L li1 ⊗ · · · ⊗ li2L, (1) The naive total number of possible words grows as 6(2L), leading to an exponential combinatorial explosion Despite the large combinatorial space, the symbol space is extremely sparse, and this sparsity is largely governed by adjacency constraints Forbidden adjacent pairs include the set of forbidden pairs such as one in
1: .
Kai: Words must be constructed with an alternating structure where (a, b, c) represents any string of letters “a”, “b”, or “c”, (d, e, f) represents a string of letters “d”, “e”, or “f” where only one of these characters is repeated.
Paper summary: Kai: So, wrapping up this discussion on "Learning Scattering Amplitudes with Transformer Reinforcement Learning," what's the bigger picture here for people outside of pure theory?
Mira: It shows that we can use these large language model architectures, like transformers, to tackle problems in quantum field theory where the state spaces are incredibly complex. They're not just guessing; they’re using learned patterns guided by known physical laws to find solutions that respect those laws.
Lev: It’s about showing how integrating structure—the known linear relations—into the learning pipeline actually makes the model better for extrapolation to higher loop orders, which is exactly where traditional methods hit a wall because of that factorial scaling.
Kai: The authors' work, using their transformer reinforcement learning approach to solve these amplitudes, suggests a viable path toward using machine learning for these kinds of calculations rather than just relying on massive, brute-force perturbative computations.
Mira: They’ve demonstrated that by structuring the problem as a sequential search where propagation handles most of the work and Monte Carlo tree search handles the difficult parts, you can reconstruct results with very little initial training data.
Lev: The paper points out a limitation, though: they mention adding a smaller set of constructed non-homogeneous relationships just to stop the transformer from outputting the trivial solution where all coefficients are zero. That’s an arbitrary constraint added for stability in the search.
Kai: So, while they show reconstruction up to loop order five with about five percent data, the method relies on those extra constraints to keep it from collapsing into a useless answer, which is a bit of an engineering patch.
Mira: It’s a practical demonstration that combining AI with established physical symmetries can be effective for exploring these high-dimensional spaces, even if you still need some careful tuning on those auxiliary rules.
Lev: For anyone thinking about implementing this kind of approach for real computational physics, the next step is figuring out how efficiently that MCTS search runs when the number of coefficients gets much larger than what they tested here. That’s where the real bottleneck is likely to be.
Kai: So, it’s a promising direction showing how transformer reinforcement learning can be steered by physical constraints to solve very hard problems in quantum field theory amplitudes.
Conclusion: Kai: So this paper is about using these transformer reinforcement learning algorithms to figure out those scattering amplitudes in planar N equals four super yang mian theory, which is some pretty hard quantum field theory stuff that usually involves a ton of tedious calculations.
Mira: Right, so the authors are showing how they can use transformers to learn the mapping between the symbols of these amplitudes and the actual integer coefficients, and they’re doing it by incorporating known symmetries into the learning process.
Lev: From an error correction standpoint, that incorporation of symmetry is key because it means we don't have to train on every single possible coefficient; we only need enough data to learn how those relationships connect things.
Kai: It sounds like they’re trying to bypass the need for knowing almost everything beforehand, which is a big deal when you look at the complexity scaling factorials in these amplitudes.
Mira: Exactly, and the way they structure it—where propagation handles most of the work and mcts explores alternatives—that's how they manage to get a complete result even when you only feed it a small fraction of the total data.
Lev: But I still have to ask, what happens when we move this from theory onto actual hardware? How much overhead does that sequential search and mcts add on top of what we’d need for real quantum error correction setups?
Kai: That’s the next thing to look at, right? Because if we can get the learning framework working on these symbols, it opens up a way to calculate things that are just too big for traditional computers.
Institute for Gravitation and the Cosmos, Pennsylvania State University · School of Physical Science and Engineering, Tongji University, Shanghai, China · School of Computer Science, Georgia Institute of Technology, USA · School of Physics, Georgia Institute of Technology, USA · Lawrence Berkeley National Laboratory
hep-th
Submitted: 2026-09-26
Updated: 2026-10-31
Comments: Submitted to ML4PS 2026
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 88/100
The gist: We introduce a transformer reinforcement learning algorithm that learns to solve high loop-level scattering amplitudes in planar N = 4 Super Yang-Mills theory Our algorithm improves on previous…
Key concepts
- Planar N=4 SYM
- This is a specific type of quantum field theory that serves as a crucial testing ground for calculating scattering amplitudes. It possesses high symmetry, which simplifies the mathematical structure compared to more complex theories. The goal is to use this simplified structure to compute these amplitudes.
- Symbolic Representation (Form Factor)
- Scattering amplitudes are represented using a symbolic form factor notation, like S[F(L)], which involves sequences of letters and coefficients. While the total number of possibilities is huge, the actual space is very sparse due to strict adjacency constraints, making it manageable for learning.
- Sequential Search with MCTS
- The algorithm reconstructs the amplitude by having a transformer propose coefficient assignments. These assignments are then checked against exact linear relations derived from symmetries. If propagation alone isn't enough, Monte Carlo Tree Search (MCTS) explores alternatives to find consistent solutions.
Terminology
Summary
We introduce a transformer reinforcement learning algorithm that learns to solve high loop-level scattering amplitudes in planar N = 4 Super Yang-Mills theory Our algorithm improves on previous transformer-only results by incorporating previously derived symmetries and relationships into the learning algorithm This results in a greatly decreased fraction of the solution that needs to be known a priori to solve the entire problem An additional benefit is that our algorithm also ensures that every output obeys the set of known relationships Rather than predicting all coefficients independently, the model proposes assignments that are propagated through exact linear relations, while MCTS searches over assignments when propagation alone is insufficient This is crucial to the generalization of machine learning approaches to higher loops, as without this, there is no way to overcome the factorially scaling of state sizes and compare to results derived via other methods. p. 0 Rather than predicting all coefficients independently, the model proposes assignments that are propagated through exact linear relations, while MCTS searches
over assignments when propagation alone is insufficient. This is crucial to the
generalization of machine learning approaches to higher loops, as without this,
there is no way to overcome the factorially scaling of state sizes and compare to
results derived via other methods.Where the paper says it
Problem Formulation in QFT
Scattering amplitudes are central objects in quantum field theory (QFT), encoding transition probabilities between asymptotic particle states Planar N = 4 Super Yang-Mills (SYM) theory has emerged as a key testing ground for such developments due to its high degree of symmetry and simplified analytic structure The difficulty of traditional perturbative computations based on Feynman diagrams scales factorially with both loop order and the number of external particles, motivating the development of alternative frameworks Recent work has shown that the symbolic structure of amplitudes can be treated as a sequence modeling problem, making it amenable to modern machine learning approaches such as transformers The shortcomings of this method are that it requires a vast majority of the final answer to be known so that it may be used as training data Furthermore, if one only greedily samples the resulting probability distribution, the outcome is almost guaranteed to violate the set of known relationships.
Symbolic Representation and Constraints
The problem is framed as learning a mapping between discrete sequences and integer coefficients using the symbology representation of the form factor The L-loop symbol takes the form S[F(L)] = X li1, …, li2L Cli1, …, li2L li1 ⊗ · · · ⊗ li2L, (1) The naive total number of possible words grows as 6(2L), leading to an exponential combinatorial explosion Despite the large combinatorial space, the symbol space is extremely sparse, and this sparsity is largely governed by adjacency constraints Forbidden adjacent pairs include the set of forbidden pairs such as one in [1]. Words must be constructed with an alternating structure where (a, b, c) represents any string of letters “a”, “b”, or “c”, (d, e, f) represents a string of letters “d”, “e”, or “f” where only one of these characters is repeated. p. 1 Words must be constructed with an alternating {(a, b, c)(d, e, f)(a, b, c). (d, e, f)} structure, where (a, b, c) represents any string of letters “a”, “b”, or “c”, (d, e, f) represents a
string of letters “d”, “e”, “f” where only one of these characters is repeated.Where the paper says it
Reconstruction via Sequential Search
The reconstruction process is formulated as a sequential search problem: a transformer proposes coefficient assignments, exact relations propagate their consequences, and MCTS explores alternatives when propagation alone is insufficient This allows us to use substantially fewer labeled coefficients while ensuring consistency with the imposed relations The minimal suffix representation technique constructs a compact representation of the states using constructed set of relationships [3]. This representation results in smaller state sizes but larger token alphabet, and each word is reduced to 2L − k + 1 tokens in length. p. 2 This is the trade-off of this representation: smaller state sizes but larger token alphabet. This
representation not only decreases the overall number of terms in the state but also decreases the size
of each word to 2l − k + 1 tokens in length.Where the paper says it
Training Strategy and Results
The pretraining phase involves creating roots where each root is a random subset of the a priori known subset The policy head of the transformer is trained with normal supervised learning between the most likely coefficient from its probability distribution The value head, however, is trained based on the MSE between its predicted length of the terminal path and its actual length. The results applied to L = 5 symbol with the K = 5 representation show that the search assigns a coefficient to all 12,543 keys that satisfies every relationship and agrees with [3], up to an additional cyclic transformation. p. 3 We used a known fraction of f ∈ {5, 10, 15, 20, 25, 30, 40, 50}%. At
every fraction that we tested, the search assigns a coefficient to all 12,543 keys that satisfies every
relationship and agrees with [3], up to an additional cyclic transformation. This is the regime where the majority of the work is done by learned priors. Figure 2: Ablation study on the L = 5, K = 5 symbol for known fractions f from 5% to 50%
(darker = larger f)
This demonstrates that the pretrained transformer combined with propagation granted by the symmetries of N = 4 SYM and [1], along with the Monte-Carlo Tree Search, reconstructs the complete
L = 5 symbol of the 3g → H with as little as 5% of its coefficients.Where the paper says it
Where the paper says it
Conclusion on Consistency
By construction, any violation ends the MCTS search path; every completed output of this algorithm satisfies all these imposed relationships This is not guaranteed in the transformer-only approach as there is no additional structure to constrain its output. The work was submitted to ML4PS on 19th September 2026. p. 4 This is not guaranteed in the transformer-only approach as there is no additional
structure to constrain its output. Note added: This work was submitted to ML4PS on 19th September 2026.Where the paper says it
Acknowledgments
We thank Lance Dixon for fruitful discussions This research used computing resources allocated to AG’s NESAP ‘AI Physicist’ project at the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory. p. 4 This research used computing resources allocated
to AG’s NESAP ‘AI Physicist’ project at the National Energy Research Scientific Computing Center (NERSC), a U. Department of Energy Office of Science User Facility located at Lawrence
Berkeley National Laboratory.Where the paper says it
References
[1] Tianji Cai, Franc¸ois Charton, Kyle Cranmer, Lance J. Dixon, Garrett W. p. 4 References
[1] Tianji Cai, Franc¸ois Charton, Kyle Cranmer, Lance J. Dixon, Garrett W. Dixon, Garrett W. Merz, and Matthias
Wilhelm.Where the paper says it
Where the paper says it
[2] Tianji Cai, Garrett W. Merz, Franc¸ois Charton, Niklas Nolte, Matthias Wilhelm, Kyle Cranmer, and Lance J. p. 4 [2] Tianji Cai, Garrett W. Merz, Franc¸ois Charton, Niklas Nolte, Matthias Wilhelm, Kyle Cranmer,
and Lance J.Where the paper says it
[3] Lance J. Dixon, Omer Gurdogan, Andrew J. p. 4 [3] Lance J. Dixon, Omer Gurdogan, Andrew J. Dixon, Omer Gurdogan, Andrew J. McLeod, and Matthias Wilhelm.Where the paper says it
Where the paper says it
[4] Alexander B. Goncharov, Marcus Spradlin, C. p. 4 [4] Alexander B. Goncharov, Marcus Spradlin, C. Goncharov, Marcus Spradlin, C. Vergu, and Anastasia Volovich.Where the paper says it
Where the paper says it
[5] Piyush Jha, Jake Rudolph, Victoria Knapp-Perez, Max Fieg, Aishik Ghosh, and Vijay Ganesh. CDRL: Certification-Driven Reinforcement Learning for Neutrino Flavor Model Discovery
[6] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improved learning of high loop-level scattering amplitudes in planar N = 4 Super Yang-Mills theory by incorporating previously derived symmetries and relationships into the learning algorithm to ensure
every output obeys the set of known relationships.
This allows for agreatly decreased fraction of the solution that needs to be known a priori to solve the entire problem
and guarantees consistency with imposed constraints. -
Development of a reconstruction method where
a transformer proposes coefficient assignments, exact relations propagate their consequences, and MCTS explores alternatives when propagation alone is insufficient.
This sequential search approach allows for solving problems without needingthe vast majority of the final answer to be known so that it may be used as training data.
-
Creation of a sparse representation technique using suffixes to construct a
compact representation of the states
where each element in the independent basis is added to the alphabet and each suffix is replaced by a representative token. This trade-off results insmaller state sizes but larger token alphabet,
making it possible to tackle larger loop levels. -
A pretraining strategy for transformers that uses a set of roots propagated using the set of relationships to find words that
participate in the highest number of relationships that only have unknowns.
This guides the transformer's search space effectively before running Monte Carlo Tree Search. -
A policy head trained with
normal supervised learning between the most likely coefficient from its probability distribution
and a value head trained based onthe MSE between its predicted length of the terminal path and its actual length.
This dual-head training provides better guidance for coefficient prediction and a long-range approximation to state consistency.
Abstract
We introduce a transformer reinforcement learning algorithm that learns to solve high loop-level scattering amplitudes in planar N = 4 Super Yang-Mills theory. Our algorithm improves on previous transformer-only results by incorporating previously derived symmetries and relationships into the learning algorithm. This results in a greatly decreased fraction of the solution that needs to be known a priori to solve the entire problem. An additional benefit is that our algorithm also ensures that every output obeys the set of known relationships. Rather than predicting all coefficients independently, the model proposes assignments that are propagated through exact linear relations, while MCTS searches over assignments when propagation alone is insufficient. This is crucial to the generalization of machine learning approaches to higher loops, as without this, there is no way to overcome the factorially scaling of state sizes and compare to results derived via other methods. Using the symbology representation of the form factor, we frame the problem as learning a mapping between discrete sequences and integer coefficients.
Related papers
- The facts on quantum self-dual Einstein gravity and self-dual Yang-Mills theory
- Quantum Langlands correspondence for Painlev'e II
- Fortuitous BMN Operators from Multiple Cores
- Building far-from-equilibrium effective field theories using shift symmetries
- Inflation, Cosmic Perturbations and Non-Gaussianities
- Wavefunctions of AdS 3 Universes and T-deformed Torus Partition Functions