What Shapes Emergent Misalignment? Insights from Training Dynamics, Model Priors, and Data
Yuchen Zhang, Anietta Weckauff, Diego Garcia-Olano, Maksym Andriushchenko
cs.AI, cs.LG, stat.ML
Submitted: 2026-06-18
Code: https://github.com/aisa-group/em-generalization
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Understanding Emergent Misalignment via Feature Superposition Geometry
- Sharpness-Aware Minimization Improves Language Model Generalization
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!
- Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs
- Cyclical Learning Rates for Training Neural Networks
- Convergent Linear Representations of Emergent Misalignment
- Emergent Misalignment is Easy, Narrow Misalignment is Hard
- The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety
- Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment
- In-Training Defenses against Emergent Misalignment in Language Models
- Model Organisms for Emergent Misalignment
- Emergently Misaligned Language Models Show Behavioral Self-Awareness That Shifts With Subsequent Realignment
- Persona Features Control Emergent Misalignment
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems