Safe Inference-Time Alignment via Lagrangian Reward Augmentation
Lagrangian Reward Augmentation (LARA) is an inference-time alignment method that transfers Safe RLHF's helpfulness/harmlessness trade-off to decoding. It achieves this by dualizing the safety constraint into a single scalar variable, $\lambda$. This allows existing decoding methods to improve helpfulness while maintaining harmlessness without requiring costly model weight updates.