Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate
The paper analyzes Sharpness-Aware Minimization (SAM) algorithms, specifically GSAM, to find an $\epsilon$-approximation of the problem. It investigates two strategies: using an increasing batch size with a constant learning rate and using a constant batch size with a decaying learning rate. Results show that increasing batch sizes lead to lower test errors and better sharpness compared to constant batch sizes, suggesting it is computationally efficient for generalization.