Specializing Without Forgetting: Analyzing Knowledge Preservation in Multilingual Model Adaptation
Researchers tested five ways to align parameters in multilingual models to prevent catastrophic forgetting when specializing them into language experts. Findings show that no single method is best for everything; layer freezing preserves comprehension, post-hoc reversion boosts translation quality, and L2 regularization maintains perplexity. The optimal strategy depends entirely on the specific task goals.