Quantum Attention by Overlap Interference: Predicting Classical and Many-Body Quantum Sequences
quant-ph, cs.CL, cs.LG
Submitted: 2026-02-06
Updated: 2026-09-24
Comments: 4 + 14 pages, 3 figures
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: We propose a variational quantum implementation of self-attention (QSA)-the core operation in transformers and large language models-which predicts future elements of a sequence by forming
Terminology
Abstract
We propose a variational quantum implementation of self-attention (QSA)-the core operation in transformers and large language models-which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and a degree- k polynomial kernel, and estimates a loss based on Rényi- 1/2 entropic functionals via two observables' expectation values, avoiding the decoding of amplitude-encoded predictions into classical probabilities. QSA also accommodates a constrained, trainable data-embedding tying state overlaps to data-level similarities. Its dominant end-to-end training complexity scales as O (μ-1k 2Td), versus O (T d k+1) of the fairest classical comparison, with μ a training signal; we show numerically that this allows a complexity advantage in the regime where sequence length T dominates the embedding size d. In simulations, our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising trajectories-establishing trainable attention as a practical primitive for quantum dynamical modeling.
Sources
- Attention Is All You Need
- Quantum Vision Transformers
- Beyond Classical Attention: Quantum Attention for Scalable Computation
- GQHAN: A Grover-inspired Quantum Hard Attention Network
- GPT on a Quantum Computer
- Quantum Transformer: Accelerating model inference via quantum linear algebra
- Quixer: A Quantum Transformer Model
- Optical decoder learning for fiber communication at the quantum limit
- A learning theory for quantum photonic processors and beyond
- Non-Linear Transformations of Quantum Amplitudes: Exponential Improvement, Generalization, and Applications
Related papers
- Reconquering Bell sampling on qudits: stabilizer learning and testing, quantum pseudorandomness bounds, and more
- Encrypted clones can leak: Classification of informative subsets in Quantum Encrypted Cloning
- Polynomial-time classical and quantum simulation of quantum impurity models
- A quasiparticle-protected superconducting qubit
- Theory of quantum-enhanced interferometry with general Markovian light sources
- A convergent hierarchy of spectral gap certificates for qubit Hamiltonians