Learning Perturbation Robust Policies for LLM Agents with Stable Optimization
This work addresses how reinforcement learning policies for large language models become unstable when subjected to real-world perturbations like noise or pruning during training. The authors developed Stable Perturbation-Robust Policy Optimization (SPrPO), a method that applies adaptive, sensitivity-aware perturbations during training. This approach theoretically guarantees stable policy improvement under specific conditions and empirically shows significantly better robustness against various adversarial changes.