RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
RanSOM introduces Randomized Second-Order Momentum to fix bias in non-convex optimization by replacing fixed step sizes with randomized ones drawn from distributions with mean $\eta_t$. This eliminates curvature-induced bias using Stein's identities and avoids costly auxiliary queries, achieving optimal convergence rates for both unconstrained and constrained problems.