Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning
The Low-rank Decomposition and Adaptation (LoDA) framework addresses limitations in continual learning by decomposing LoRA updates into general and task-specific subspaces. It uses energy-based objectives to build a general subspace for shared knowledge transfer and an isolated subspace for task-specific adaptation, improving knowledge sharing while minimizing interference from past tasks.