Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam
The paper presents a method using Armijo backtracking as a low-cost probe to measure the local sharpness of a loss landscape. This technique acts as a safeguard for Adam optimization, allowing it to detect and cap excessively large learning rates that would otherwise cause training divergence. It is designed to be calibration-free and highly transferable across various architectures.