P 2O: Joint Policy and Prompt Optimization
The paper introduces Joint Policy and Prompt Optimization (P2O) to fix advantage collapse in Reinforcement Learning with Verifiable Rewards (RLVR). P2O alternates between updating model parameters and evolving discrete prompts to handle hard samples. This joint approach prevents the model from losing learning signals on difficult problems, leading to significant performance gains over standard methods.