A Dynamical Theory of LoRA in Continual Learning

Name: A Dynamical Theory of LoRA in Continual Learning
Uploaded: 2026-10-04T21:27:54.724375+00:00
Duration: 673 s
Description: This work develops a closed system of differential equations to precisely model how Low-Rank Adaptation (LoRA) affects feature learning and forgetting across sequential tasks. It shows that LoRA adaptation slows down initial learning but state-dependent masking significantly reduces catastrophic forgetting while maintaining the ability to learn new tasks.

This work develops a closed system of differential equations to precisely model how Low-Rank Adaptation (LoRA) affects feature learning and forgetting across sequential tasks. It shows that LoRA adaptation slows down initial learning but state-dependent masking significantly reduces catastrophic forgetting while maintaining the ability to learn new tasks.

Open the full episode