Semifactual Credit-Augmented Policy Optimization
The study introduced Semifactual Credit-Augmented Policy Optimization (SCAPO) to improve large language model reasoning in reinforcement learning by addressing sensitivity to irrelevant prompt features. SCAPO measures token probability drift caused by minor prompt changes and uses this signal to reduce the advantages given to unstable tokens during training, leading to better accuracy without retraining the model.