Mitigating Model Merging Conflicts via Knowledge-Aware Task Vector Purification
The Purifying Task Vector (PAVE) is a method to improve model merging performance by refining task vectors within a knowledge-aware subspace. PAVE uses context-oriented decomposition and a spectral rank allocation strategy to identify and prune redundant, task-irrelevant components in model weights. This purification resolves conflicts arising from redundancy, leading to significant performance gains on merging tasks.