Alignment via Training Against Probes Without Losing Monitorability
Models can be aligned by training directly against probes that are continuously updated, without sacrificing their usefulness. The method uses probes to detect unwanted behaviors like harmfulness or dishonesty during training. Continuously updating these probes significantly reduces these undesirable traits while maintaining the model's original utility and ensuring the target property remains measurable.