No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attacks
Simiao Xie, Chuancheng Shi, Shangze Li, Wenhua Wu, Fei Shen, Ying Zhou, Zhiyong Wang, Tat-Seng Chua
cs.AI, cs.CR
Submitted: 2026-08-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: With the rapid release of open-weight large foundation models, safety threats are shifting from black-box jailbreaks to neuron-level white-box attacks that directly identify and manipulate
Terminology
Abstract
With the rapid release of open-weight large foundation models, safety threats are shifting from black-box jailbreaks to neuron-level white-box attacks that directly identify and manipulate safety-related neurons. Existing alignment methods often investigate the safety behavior on a small number of neurons, creating fragile single point of failure with limited redundancy. To address this issue, we propose distributed safety alignment (DSA), which redundantly encodes safety capabilities across multiple computational neurons, ensuring that the model maintains its safety baseline even when critical safety neurons are disrupted. Specifically, we localize the intervention to the inputs of the down-projection layers in language-side feed-forward networks and treat each feature coordinate as the activation of an individual neuron. DSA then combines neuron activations with loss gradients to compute a direction-aware first-order Taylor score that globally identifies the neurons that contribute most to the current refusal behavior of the model. Finally, targeted disruption via deterministic masking and stochastic dropout is coupled, forcing the model to abandon narrow safety neurons and redundantly encode safety behavior across multiple compensatory neurons. Extensive experiments show that DSA substantially improves robustness against white-box neuron-level safety attacks while preserving the model's general language and multimodal utility.
Sources
- Phi-4 Technical Report
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Mechanistic Interpretability for AI Safety -- A Review
- Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
- DNA: Uncovering Universal Latent Forgery Knowledge
- Gemma: Open Models Based on Gemini Research and Technology
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
- Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation
- OrthoEraser: Coupled-Neuron Orthogonal Projection for Concept Erasure
- A StrongREJECT for Empty Jailbreaks
- SafeNeuron: Neuron-Level Safety Alignment for Large Language Models
- Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language Models
- Improving Alignment and Robustness with Circuit Breakers
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems