Right Answer, Wrong Tokens: Tail-Aware Credit Calibration for RLVR
The paper addresses Positive-Credit Contamination in Group Relative Policy Optimization (GRPO), where uniform credit assignment incorrectly reinforces erroneous, low-probability tail tokens alongside correct ones. The proposed Tail-Aware Credit Calibration (TACO) method introduces a risk score based on token rarity to softly suppress positive credit for risky tokens without eliminating their learning opportunities, leading to improved reasoning performance and training stability.