Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
The study investigated if restricting logistic regression data to have equal magnitudes guarantees global convergence for all step sizes below a stability threshold in gradient descent. In one dimension, this restriction was sufficient for convergence up to a certain step size limit. However, in higher dimensions, even with equal-norm data, cycling behavior can still occur, proving the restriction is not universally sufficient.