MuEdit: An efficient multi-task editing method towards inter-domain knowledge conflicts
cs.AI, cs.CL
Submitted: 2026-11-11
Updated: 2026-11-11
Comments: 30 pages,accepted by NeurIPS 2026
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Large language models (LLMs) require frequent knowledge updates across heterogeneous domains, but existing editing methods fail when applied sequentially to multi-task settings.
Terminology
Abstract
Large language models (LLMs) require frequent knowledge updates across heterogeneous domains, but existing editing methods fail when applied sequentially to multi-task settings. We trace this collapse to inter-domain null-space misalignment: heterogeneous tasks induce distinct Jacobian geometries, causing their common null space to shrink dramatically and leaving insufficient admissible directions for conflict-free updates. We propose the Conflict Index to quantify this geometric interference, and introduce Mu-Edit, which mitigates multi-task conflicts via (1) conflict-aware ordering to minimize cumulative interference, and (2) dynamic low-rank approximation to expand the null space when ordering alone is insufficient. Experiments across five functional domains and three backbones show that Mu-Edit significantly outperforms existing baselines in multi-task editing performance while preserving general capabilities, and remains effective under incremental task arrival.
Sources
- Heterogeneous Agent Collaborative Reinforcement Learning
- Mass-Editing Memory in a Transformer
- Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models
- Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimization
- Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs
- Does Your Reasoning Model Implicitly Know When to Stop Thinking?
- Fast Model Editing at Scale
- AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
- Perturbation-Restrained Sequential Model Editing
- EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models
- EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- AnyEdit: Edit Any Knowledge Encoded in Language Models
- The Llama 3 Herd of Models
- Qwen2.5 Technical Report
- CodeSearchNet Challenge: Evaluating the State of Semantic Code Search
- Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis
- FAME: Towards Factual Multi-Task Model Editing
- InstructEdit: Instruction-based Knowledge Editing for Large Language Models
- Commonsense Knowledge Editing Based on Free-Text in LLMs
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents