ReNFT: Repairing Mode Collapse in Reward Post-Training via Internal Probability-Mass Recalibration
The episode discusses ReNFT, a method for repairing mode collapse in reward-trained AI models. The authors argue that this issue is reversible and can be fixed internally by using two internal routes—a base route and an unconditional route—to generate counterfactual candidates. This allows the model to achieve high diversity while maintaining high quality.