Scalable Hessian-vector products for tensor networks with application to quantum circuit compression
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Quantum Radio. Generated commentary on the latest quantum physics and condensed matter papers.
Kai: We're looking at the paper: "Scalable Hessian-vector products for tensor networks with application to quantum circuit compression".
Title and authors: Kai: So this paper is called "Scalable Hessian-vector products for tensor networks with application to quantum circuit compression". It sounds like it’s tackling the problem of finding the second derivative information, which you need for good optimization, but doing it in a way that doesn't explode when your system gets big.
Mira: Right. The title suggests they are looking at tensor networks, which are how we usually represent these large quantum circuits and systems. And they’re focusing on Hessian-vector products, or HVPs, which is the tool you use to get second-order information without building the whole Hessian matrix upfront.
Lev: I wonder if this is useful for real hardware. Because when you're running something on a real quantum computer, memory and time are so tight that constructing a full Hessian matrix is just not an option for large systems, right?
Kai: Exactly. Standard first-order methods get stuck in local minima because they only look at the slope, but second-order methods need this curvature info. This paper claims to provide a scalable way to get that information without the memory explosion that comes with traditional Hessian calculations.
Mira: It’s about bypassing that bottleneck by using an analytical HVP kernel designed for any combination of linear maps you have in your network. That’s the core idea they are pushing here.
Lev: If it can handle arbitrary compositions of linear maps, that sounds flexible, but what happens when we move from a small toy circuit to something with thousands of parameters? Does it still scale well then?
Kai: That’s where their methodology gets interesting. They use recursive tangent-state propagation to guarantee scalability. They claim the maximum virtual bond dimension stays strictly bounded during this process.
Mira: And they achieve that by defining an augmented virtual space, which is just a direct sum of two identical virtual spaces, making the bond dimension bounded by two times the intermediate state’s bound. It sounds like a structural fix for memory growth.
Lev: Bounded bond dimensions are crucial for me. If the intermediate state size keeps growing linearly with the circuit depth, even if it's bounded by some constant times the previous step, that still means we might run out of RAM or time on real hardware.
Kai: The point is they show this structural property holds, which gives us a theoretical guarantee that it won't grow uncontrollably as you increase the size of your network. So, they’re addressing the scalability issue directly in their derivation.
The paper's summary: Mira: Okay, so if we look at the actual summary of "Scalable Hessian-vector products for tensor networks with application to quantum circuit compression", it boils down to this: they developed an analytical HVP kernel that works for any combination of linear maps.
Kai: They found that both forward-over-reverse and reverse-over-reverse HVP approaches naturally converge into the same two-pass algorithmic structure. This means you don't have to choose one specific way to calculate it; the method converges regardless of your approach, as long as you use this structure.
Lev: That convergence into an identical structure is a big deal for implementation. It simplifies things because we don't have to worry about which pass we pick for the forward or backward propagation when we need that Hessian-vector product.
Mira: And the kernel they derive is computed as the directional derivative of a complex gradient, and they show this is mathematically equivalent to something called Algorithm one <ref:2604.20384#pg1>. That links their high-level theory directly back to a concrete computational step.
Kai: They also claim that this resulting kernel provides up to four orders of magnitude improvement in fidelity when used for quantum circuit compression compared to just using naive Trotterization, which is what most people do by default.
Lev: Four orders of magnitude is a huge jump for accuracy, especially when we think about the overhead. If we can get that much better fidelity with a method that avoids explicit Hessian construction, that’s very compelling for practical use on real machines.
Mira: The main implication here is unifying flexibility with efficiency. They manage to combine the flexibility of adjoint methods with the computational efficiency of tensor network specific algorithms, which is what they call bridging those two paradigms.
The paper's improvements: Kai: When we look at the suggested improvements in this paper, one big thing is that they get second-order curvature information without having to explicitly construct the full Hessian matrix. That’s a massive win for large systems where that matrix just becomes too big to handle.
Lev: So, instead of calculating every single derivative interaction simultaneously, you only calculate what you need—the vector product—and you do it efficiently using this kernel. That cuts the complexity dramatically for global optimization tasks on large networks.
Mira: Another major point is that they show this framework can be integrated into a Riemannian trust-region algorithm specifically for quantum circuit compression. This means they are applying this HVP kernel to optimize the Hilbert-Schmidt distance between your target unitary and your ansatz.
Kai: And when you use that optimization, compared to first-order methods like Riemannian ADAM, the convergence is significantly smoother and more monotonic. That smoothness comes from having that second-order information available during the step selection process.
Lev: A smoother convergence profile means fewer wild jumps in parameter space, which translates directly into a more stable run on hardware. It also suggests that we can adapt the step size adaptively better because we have a better sense of the local curvature.
Mira: They also point out that for translationally invariant circuits, they have an analytical HVP kernel that can be computed by summing local directional derivatives across layers while keeping the structure memory-efficient. That’s a practical hint for applying it to periodic systems.
Kai: And they also provide a diagnostic tool there, which is the ability to derive exact second-order geometry information, like the Riemannian Hessian spectrum. This tells you about the ill-conditioning in your optimization landscape, which helps explain why first-order methods get stuck.
Conclusion: Mira: So to wrap up on "Scalable Hessian-vector products for tensor networks with application to quantum circuit compression", this paper shows that we can access global second-order curvature information without having to build the full Hessian matrix, and they guarantee scalability through recursive propagation techniques.
Kai: It gives us a scalable lens into the second-order geometry of large systems, and it sets up a framework that can be extended to other areas like variational Monte Carlo simulations or even learning circuits with infinite tensor networks.
Lev: From an engineering standpoint, the guarantee on bounded virtual bond dimensions during recursive propagation is key. That structural property ensures computational feasibility for deep circuits and large many-body systems when we move this to real hardware.
Mira: And they showed that this approach can be used for quantum circuit compression, leading to much smoother and more monotonic convergence than first-order methods like Riemannian ADAM, while achieving a four orders of magnitude improvement in fidelity over naive Trotterization.
Kai: So the final thought is that they provide a way to get robust global updates for large networks by using this analytical HVP kernel. It’s a very powerful tool for understanding and optimizing these complex quantum systems.
Lev: I just think the ability to diagnose the Riemannian Hessian spectrum is valuable because it helps us understand exactly where the optimization landscape is getting messy, which is something we need when we start scaling up.
Mira: Yeah, so this paper gives us a concrete method for handling second-order geometry on these large systems in a scalable way. It’s a solid piece of work that points the way forward.
Isabel Nha Minh Le, Roeland Wiersema, Christian B. Mendl
Technical University of Munich · Munich Center for Quantum Science and Technology · Center for Computational Quantum Physics
quant-ph, physics.comp-ph
Submitted: 2026-04-22
Updated: 2026-11-14
Comments: 15 pages + 10 pages appendix, 7 figures + 2 figures in the appendix
Code: https://github.com/INMLe/rqcopt-hvp
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 79/100
The gist: Optimizing tensor networks with standard first-order methods often leads to slow convergence and entrapment in local minima.
Key concepts
- Hessian-vector product (HVP) Kernel
- This is an analytical tool derived from the tensor network's structure that calculates a specific part of the Hessian matrix without constructing the entire matrix. It allows for second-order optimization, which helps find better solutions in complex optimization landscapes efficiently.
- Recursive Tangent-State Propagation
- This is a method used to compute the HVP. It works by propagating states through the network recursively, ensuring that the memory required for these intermediate states remains strictly bounded. This structural property guarantees that the computation scales efficiently even for very large systems.
- Riemannian Trust-Region Framework
- This is an optimization strategy used to compress quantum circuits. Instead of taking a single large step, it uses the HVP kernel to evaluate the curvature of the optimization landscape. This allows for adaptive step size adjustments, leading to smoother and more reliable convergence than standard first-order methods.
- Virtual Bond Dimension
- This is a measure related to how complex or large the intermediate states in the tensor network are during computation. The paper proves that by using a specific augmented virtual space, this dimension stays strictly bounded, which is key to ensuring that the computational memory requirements do not grow uncontrollably.
Terminology
Summary
Optimizing tensor networks with standard first-order methods often leads to slow convergence and entrapment in local minima. p. 0 This work introduces an analytical Hessian-vector product kernel designed for arbitrary compositions of linear maps, leveraging recursive tangent-state propagation to guarantee scalability. p. 0 The second-order approach demonstrates up to a four-order-of-magnitude improvement in fidelity over naive Trotterization while delivering significantly smoother convergence than conventional first-order methods. p. 0
Derivation of the Hessian Kernel
The authors derive an analytical HVP kernel for arbitrary compositions of linear maps by exploiting the multi-linear structure inherent to tensor networks. This derivation reveals that both forward-over-reverse and reverse-over-reverse HVP approaches naturally converge into an identical two-pass algorithmic structure. p. 1 The final HVP is computed as the directional derivative of the complex gradient, which is shown to be equivalent to Algorithm 1. p. 1
Scalability and Tangent States
The framework guarantees scalability by demonstrating that the maximum virtual bond dimension of the tangent states remains strictly bounded. This is achieved by defining an augmented virtual space Vaug as the direct sum of two identical virtual spaces, Vaug = V ⊕ V, which allows for an exact representation whose bond dimension is strictly bounded by 2χ. p. 7
Application to Quantum Circuits
The HVP kernel is integrated into a Riemannian trust-region framework for quantum circuit compression. This approach optimizes the Hilbert-Schmidt distance between a target unitary and an ansatz by evaluating the Euclidean gradient and Hessian via Algorithm 2. p. 4 The second-order optimization yields significantly smoother and more monotonic convergence compared to first-order methods like Riemannian ADAM. p. 0
Performance on Spin Chains
The practical utility is demonstrated by optimizing Trotter circuits for transverse-field Ising and Heisenberg spin chains using the second-order Riemannian trust-region algorithm. This second-order approach successfully recovers and validates previous findings, achieving an improvement in approximation accuracy of up to four orders of magnitude over naive Trotterization. p. 8 The convergence behavior shows that the trust-region method yields significantly smoother and monotonic convergence than Riemannian ADAM by adaptively constraining the step size. p. 9
Conclusion
The paper concludes that the proposed HVP kernel provides access to global second-order curvature information without explicitly constructing the full Hessian matrix. This framework enables robust global updates for large-scale networks and offers a scalable lens into the second-order geometry of large systems. p. 1 The scalability is guaranteed by maintaining bounded virtual bond dimensions during recursive propagation. p. 6 The framework can be extended to other domains such as variational Monte Carlo simulations and learning circuits with infinite tensor networks. p. 0 The Riemannian HVP can be obtained by first taking the directional derivative of the Riemannian gradient and then projecting this result back to the tangent space. p. 6 The final HVP is computed as a superposition of two outer products, establishing exact equivalence with previous formulations. p. 3 The final HVP is computed as the gradient of the directional derivative, which simplifies to Algorithm 1. p. 6 The final HVP is computed as the gradient of the directional derivative, which simplifies to Algorithm 3. p. 6
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Analytical Hessian-vector products for tensor networks allow for second-order optimization of TN parameters without explicitly constructing a full Hessian matrix, bypassing its prohibitive quadratic scaling with system size. This enables efficient global network optimization for large-scale systems previously inaccessible to second-order methods.
-
The recursive tangent-state propagation algorithm guarantees scalability by ensuring the maximum virtual bond dimension of the tangent states remains strictly bounded by two times the maximum bond dimension of the intermediate state, avoiding unscalable linear memory growth. This structural property ensures computational feasibility even for deep circuits and large many-body systems.
-
The framework can be integrated into a Riemannian trust-region algorithm for quantum circuit compression, achieving up to four orders of magnitude improvement in fidelity over naive Trotterization by utilizing the second-order curvature information from the HVP kernel. This leads to significantly smoother and more monotonic convergence compared to first-order methods like Riemannian ADAM.
-
The system can utilize a unified two-pass algorithm (forward and backward passes) to compute both the gradient of the loss function and its Hessian-vector product simultaneously within a single computational budget. This concurrent execution allows for the simultaneous accumulation of second-order curvature information without requiring separate, costly evaluations.
-
For translationally invariant circuits, the framework provides an analytical HVP kernel that can be computed by summing local directional derivatives across layers while maintaining a memory-efficient structure. This allows for efficient evaluation of the Hessian-vector product in periodic systems where the perturbation direction is constant within each layer.
-
The method allows for the derivation of exact second-order geometry information, such as the Riemannian Hessian spectrum, which reveals strong ill-conditioning in optimization landscapes. This diagnostic tool helps understand why first-order methods exhibit instability and stagnation in high-curvature regions of the manifold.
Sources
- Scalable quantum dynamics compilation via quantum machine learning
- High-Performance Contraction of Quantum Circuits for Riemannian Optimization
- Regularized second-order optimization of tensor-network Born machines
- A short tutorial on Wirtinger Calculus with applications in quantum information
- Learning Circuits with Infinite Tensor Networks
- Variational Monte Carlo (VMC) with row-update Projected Entangled-Pair States (PEPS) and its applications in quantum spin glasses
Related papers
- Computational regimes in matrix-product-state-based quantum trajectory simulations
- Toward chemically accurate quantum simulations on quantum hardware: A quantum-HPC hybrid approach
- Transversal surface-code game with reconfigurable qubits
- Fulqrum: a generalized framework for quantum subspace diagonalization
- Flux-Activated Resonant Control of a Bosonic Quantum Memory
- Machine learning the effects of many quantum measurements