Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
The episode discusses "Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise," a paper by Jun-Kun Wang and Maria-Eleni Sfyraki. Hosts review how the work unifies several optimizers, providing theoretical guarantees and robust variants for training AI models using real-world, heavy-tailed noise.