Reduced and Mixed-Precision Block Householder QR Factorization
math.NA, cs.NA
Submitted: 2026-10-28
Updated: 2026-10-28
Comments: 22 pages
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in hardware architecture have highlighted the importance of reduced-precision arithmetic for improving performance and reducing energy consumption in large-scale scientific…
Abstract
Recent advances in hardware architecture have highlighted the importance of reduced-precision arithmetic for improving performance and reducing energy consumption in large-scale scientific applications. However, uniformly low-precision methods can suffer from numerical instability, especially in fundamental linear algebra routines. Mixed-precision strategies therefore play an important role in balancing efficiency, stability, and accuracy. In this paper, we focus on the block Householder QR decomposition using the WY representation and study its behavior in reduced- and mixed-precision settings. Starting from a uniform half-precision variant, we show that performing norm computations with fp32 accumulation is necessary to avoid overflow and underflow on large matrices. Based on this observation, we introduce mpWYQR, a mixed-precision algorithm that increases the precision of only selected computations and avoids most of the precision conversions required by the existing mixed-precision blocked QR algorithm (mpBQR3) introduced in [L. M. Yang, A. Fox, and G. Sanders, SIAM J. Sci. Comp., 43.3 (2021), pp. A1723--A1753]. Our analysis shows that mpWYQR accuracy is governed by low-precision unit roundoff, and that its error bound exceeds that of mpBQR3 by only a constant factor, with identical heuristic bounds. In experiments across a range of matrix sizes and condition numbers, we show that mpWYQR retains mpBQR3 accuracy while running up to 18 times faster and using less memory. We also discuss the performance cost of computing inner products in higher precision, and the library support needed for an efficient GPU implementation.
Related papers
- Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
- A Neural-preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions
- Advanced Linear Algebra with Applications - Part I (Numerical linear algebra for PDEs, machine learning, and data assimilation)
- Operator learning meets inverse problems: A probabilistic perspective
- Second-order consistency for learning chaotic dynamics via randomized Jacobian matching
- Windowed thinning and query complexity for the bouncy particle and Zigzag samplers