Non-Stationary Functional Bilevel Optimization
The episode discusses "Non-Stationary Functional Bilevel Optimization," focusing on how to train machine learning models when underlying dynamics or objective functions change over time. The proposed method, SmoothFBO, uses a time-smoothed stochastic hypergradient estimator with a window parameter to stabilize outer loop updates and reduce variance in non-stationary environments.