Modeling quantum neural network gradient with reinforcement learning
quant-ph, cs.ET, cs.NE
Submitted: 2026-09-25
Updated: 2026-09-25
Comments: NeurIPS 2026 Main Track (Poster). OpenReview: https://openreview.net/forum?id=lfrvu8xfsh
License: http://creativecommons.org/licenses/by/4.0/
The gist: Training quantum neural networks (QNNs) on near-term hardware remains hampered by two compounding difficulties: the exponential vanishing of gradient variance known as the barren plateau, and the O(L
Terminology
Abstract
Training quantum neural networks (QNNs) on near-term hardware remains hampered by two compounding difficulties: the exponential vanishing of gradient variance known as the barren plateau, and the O(L times 2 n) time and memory cost of differentiating through an n-qubit, L-layer circuit. We propose RLQ-Grad, a reinforcement-learning-based optimizer in which a classical policy π ϕ (a spectrally-normalized PPO agent) learns to propose parameter updates directly, conditioned on the QNN's current parameters, loss, accuracy, and previous update. Because the surrogate gradient is emitted by a classical network rather than obtained by differentiating through the unitary U(θ), its variance is not constrained by the barren plateau concentration bound, and its cost scales with the number of trainable parameters rather than the Hilbert-space dimension. We prove these properties formally and verify them on a hardware-efficient ansatz across four supervised benchmarks with up to n=20 qubits. RLQ-Grad preserves a near-flat gradient-variance curve where backpropagation, parameter-shift, and adjoint differentiation decay by 1 to 2 orders of magnitude. Accounting for the full training pipeline (PPO rollouts, actor-critic updates, and optimizer states), RLQ-Grad needs under 2 MB of memory and runs 2490 times, 7876 times, and 673 times faster per iteration than these three methods at n=20. It improves top-1 accuracy by up to +10% over gradient-based baselines on circuits of up to 12 qubits, and matches dedicated barren plateau mitigation methods on CIFAR-10 at 14 to 20 qubits, where evolutionary and gradient-free optimizers collapse to chance.
Sources
- Learning to learn with quantum neural networks via classical neural networks
- Reinforcement Learning for Variational Quantum Circuits Design
- TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search
- Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems
- Quantum circuit optimization with deep reinforcement learning
- Quantum Architecture Search via Deep Reinforcement Learning
- Optimizing Quantum Variational Circuits with Deep Reinforcement Learning
- Efficient calculation of gradients in classical simulations of variational quantum algorithms
- PennyLane: Automatic differentiation of hybrid quantum-classical computations
- Adam: A Method for Stochastic Optimization
- SGDR: Stochastic Gradient Descent with Warm Restarts
- A Unified Theory of Quantum Neural Network Loss Landscapes
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- The CMA Evolution Strategy: A Tutorial
- Hardware-efficient ansatz without barren plateaus in any depth
- Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves
Related papers
- Reconquering Bell sampling on qudits: stabilizer learning and testing, quantum pseudorandomness bounds, and more
- Encrypted clones can leak: Classification of informative subsets in Quantum Encrypted Cloning
- Polynomial-time classical and quantum simulation of quantum impurity models
- A quasiparticle-protected superconducting qubit
- Theory of quantum-enhanced interferometry with general Markovian light sources
- A convergent hierarchy of spectral gap certificates for qubit Hamiltonians