Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training
The paper examines Evolution Strategies (ES) and GRPO in LLM post-training. ES achieves high peak accuracy but struggles with forgetting prior knowledge during sequential learning, while its updates are broad. GRPO offers targeted adjustments and stability. The hosts conclude that the choice between these two methods depends on whether an application requires peak performance or stable, continuous learning.