Neural Tangent Kernel Perspective on Parameter-Space Symmetries
The episode discusses a paper titled "Neural Tangent Kernel Perspective on Parameter-Space Symmetries." The hosts explore how linearization in wide neural networks is not accidental but a consequence of weak derivative correlations at initialization. They conclude that this relationship is equivalent and provides practical tools for measuring and controlling network behavior.