LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates

Name: LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates
Uploaded: 2026-09-07T00:22:44.897425+00:00
Duration: 428 s
Description: This episode discusses the paper 'LoRA-TSD,' a new optimizer that addresses geometric disconnect in LoRA updates. The hosts explain how it uses Tangent-Space Spectral Descent and Muon-Style Updates to treat the update as movement along a specific curved path.The method is shown to be more stable and efficient than existing baselines across multiple benchmarks.

This episode discusses the paper 'LoRA-TSD,' a new optimizer that addresses geometric disconnect in LoRA updates. The hosts explain how it uses Tangent-Space Spectral Descent and Muon-Style Updates to treat the update as movement along a specific curved path.The method is shown to be more stable and efficient than existing baselines across multiple benchmarks.

Open the full episode