Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision
The episode discusses the paper "Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision," which proposes using wavelets to find sparse weight updates during fine-tuning of large vision models. The hosts explain how this method achieves better performance and parameter efficiency compared to existing techniques like LoRA, highlighting its benefits across image generation and classification tasks.