Representation Drift in Continual Learning: LoRA versus N-gram Memory

Name: Representation Drift in Continual Learning: LoRA versus N-gram Memory
Uploaded: 2026-10-18T22:50:53.362518+00:00
Duration: 661 s
Description: The study compares three methods for online learning: updating N-gram memory, training a shared LoRA adapter, and training a fresh LoRA per task. It found that N-gram memory keeps the base fixed while suffering high drift on old tasks due to shared rows. In contrast, LoRA methods show higher overall drift but are more stable than N-gram in certain aspects.

The study compares three methods for online learning: updating N-gram memory, training a shared LoRA adapter, and training a fresh LoRA per task. It found that N-gram memory keeps the base fixed while suffering high drift on old tasks due to shared rows. In contrast, LoRA methods show higher overall drift but are more stable than N-gram in certain aspects.

Open the full episode