Riemannian Optimization for Hadamard Products of Low-Rank Matrices

arXiv:2606.01216 · cs.LG, math.OC · Submitted 2026-05-31 · Read on arXiv

cs.LG, math.OC

Submitted: 2026-05-31

Updated: 2026-08-31

Comments: Added second-order discussion

License: http://creativecommons.org/licenses/by/4.0/

The gist: The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of

Terminology

Abstract

The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under these symmetries. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. The versatile framework of Riemannian quotient optimization enables both first-order and second-order Riemannian methods, the latter through a closed-form connection and the Riemannian Hessian. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.

Sources

Related papers