How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift
The study investigated how task adaptation changes alignment in LLMs across 15 dimensions like safety and factuality. It found that supervised fine-tuning causes the most significant alignment drift, but KL regularization helps reduce this drift. Behavioral changes are also visible in internal model representations.