Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods
The episode discusses a paper titled "Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods" by Chayti, Doikov, and Jaggi. The hosts explain how this paper unifies various stochastic and variance-reduced second-order optimization methods using a 'helper framework.' They conclude that this framework allows for faster algorithms, such as 'lazy' Hessian updates, which save computation time in high dimensions.