Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study
The episode discusses a paper quantifying the trustworthiness cost of domain adaptation in small language models (SLMs) across different architectures and fine-tuning methods. The study found that baseline adaptation does not significantly harm factual calibration, but specific safety fine-tuning methods like Dark Experience Replay and Task Arithmetic LoRA can reduce adversarial robustness, especially for smaller models.