Non-Euclidean Gradient Descent Has Many Edges of Stability
The paper investigates how stability thresholds behave in non-Euclidean gradient descent, comparing Euclidean and other norms. It finds that Hilbertian norms are uniquely characterized by an almost-everywhere sharp stability threshold for quadratic objectives. For non-Hilbertian norms, multiple distinct stability edges can exist, leading to the coexistence of stable and unstable basins at the same step size.