Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization
This episode examines a paper on sign-based random reshuffling algorithms for non-convex optimization. The hosts discuss how the naive method, SignRR, has a fundamental flaw where discarding gradient magnitude causes convergence to stall. They conclude that a fix is needed, proposing SignRVR, a variance-reduced version that achieves clean convergence rates without extra assumptions.