Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?
The study investigates whether gains reported by extractors using Reinforcement Learning with Verifiable Rewards (RLVR) come from better answers or simply a change in answer format. By scoring model outputs with different extraction methods, the research found that 82% of out-of-domain multiple-choice gains are 'format gain,' meaning the RL training moved answers into a box where extractors look first, not because the underlying answers improved.