Expected Value Alignment for Generative Reward Modeling in Formal Mathematics Verification
The episode discusses the paper "Expected Value Alignment for Generative Reward Modeling in Formal Mathematics Verification." It addresses a conflict between traditional value-head models, which lack textual rationale, and generative models, which maintain text but struggle with continuous numerical alignment. The authors propose extracting continuous scores from the model's token distribution to provide a nuanced measure of intellectual rigor and uncertainty.