Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
The episode discusses a paper titled "Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning." The hosts explore how this work provides a theoretical framework, called Neural LoFi, to explain how deep neural networks learn internal representations through iterative spectral procedures. They conclude that this framework offers a tractable way to study feature selection and suggests improvements for building more dynamic and flexible AI systems.