MuEdit: An efficient multi-task editing method towards inter-domain knowledge conflicts

arXiv:2611.14910 · cs.AI, cs.CL · Submitted 2026-11-11 · Read on arXiv

cs.AI, cs.CL

Submitted: 2026-11-11

Updated: 2026-11-11

Comments: 30 pages,accepted by NeurIPS 2026

License: http://creativecommons.org/licenses/by-sa/4.0/

The gist: Large language models (LLMs) require frequent knowledge updates across heterogeneous domains, but existing editing methods fail when applied sequentially to multi-task settings.

Terminology

Abstract

Large language models (LLMs) require frequent knowledge updates across heterogeneous domains, but existing editing methods fail when applied sequentially to multi-task settings. We trace this collapse to inter-domain null-space misalignment: heterogeneous tasks induce distinct Jacobian geometries, causing their common null space to shrink dramatically and leaving insufficient admissible directions for conflict-free updates. We propose the Conflict Index to quantify this geometric interference, and introduce Mu-Edit, which mitigates multi-task conflicts via (1) conflict-aware ordering to minimize cumulative interference, and (2) dynamic low-rank approximation to expand the null space when ordering alone is insufficient. Experiments across five functional domains and three backbones show that Mu-Edit significantly outperforms existing baselines in multi-task editing performance while preserving general capabilities, and remains effective under incremental task arrival.

Sources

Related papers