Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation

Fine-tuning can introduce unwanted behaviors through deliberate poisoning or unexpected generalization from seemingly benign data. Because these effects may be difficult to anticipate or detect, we seek robustness in the fine-tuning process itself. Our premise is that datasets independently collected for the same task would share the intended capability while differing in their unwanted effects. This creates an opportunity to learn from agreement across sources while reducing dependence on any individual source. We introduce Multi-Source Consensus Distillation (MSCD): we fine-tune one model on each dataset, use agreement among these models to generate replacement training responses, and train a fresh student on the resulting data. We develop minimum and base-relative rules for combining the models' predictions, with extensions that accommodate useful behaviors learned by only some models or expressed in different words. Experiments on controlled poisoning, subliminal learning, and emergent misalignment show suppression of unwanted behavior while retaining shared benefits. In a practical language-understanding task, MSCD retains intent-classification capability while reducing unsafe medical advice introduced during fine-tuning. Comparing distilled students with direct consensus decoding shows that distillation closely preserves the measured desirable and undesirable behavior rates while replacing multiple model evaluations per generated token with a single student evaluation.