BrainSurgery: Reproducible and Flexible Declarative Weight Manipulations for Model Editing and Upcycling
cs.LG, cs.CL
Submitted: 2026-06-08
Updated: 2026-10-28
Code: https://github.com/schneiderkamplab/brainsurgery
License: http://creativecommons.org/licenses/by/4.0/
The gist: Checkpoint transformations such as model merging, precision casting, and low-rank rewriting are commonly implemented through Python code or separate task-specific tools.
Terminology
Abstract
Checkpoint transformations such as model merging, precision casting, and low-rank rewriting are commonly implemented through Python code or separate task-specific tools. We present BrainSurgery, a tool that consolidates these operations into reusable declarative YAML plans over checkpoint tensors with built-in targeting, validation, diffing, and sharded export; a Web UI supports interactive inspection and plan construction. We compare BrainSurgery with three specialized systems across 13 settings, 11 benchmarks, 8 models, 4 architectures, and model sizes from 1.7B to 32B parameters. Without use-case-specific framework customization, BrainSurgery reproduces their model edits with comparable run cost, often lower memory usage, and close downstream performance. We further perform correctness, preservation, robustness, and scaling checks. Finally, a coding-agent study shows that agents can reliably generate and review BrainSurgery plans, which are substantially shorter and more structurally consistent than equivalent Python implementations. These results support BrainSurgery as a flexible alternative to maintaining separate tools for heterogeneous checkpoint-editing tasks.
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- nnterp: A Standardized Interface for Mechanistic Interpretability of Transformers
- NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals
- Arcee's MergeKit: A Toolkit for Merging Large Language Models
- LoRA: Low-Rank Adaptation of Large Language Models
- NeuroSurgeon: A Toolkit for Subnetwork Analysis
- Mass-Editing Memory in a Transformer
- Interpreto: An Explainability Library for Transformers
- PHLoRA: data-free Post-hoc Low-Rank Adapter extraction from full-rank checkpoint
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis