When Rewards Depend on the Group, GRPO Plays a Tournament
cs.LG, cs.CL
Submitted: 2026-11-02
Updated: 2026-11-02
Terminology
Sources
- Sample What You Say: Aligning Language Models to Sample the Distributions They State
- SPIRAL: Learning to Search and Aggregate
- SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning
- Jointly Reinforcing Diversity and Quality in Language Model Generations
- Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
- LLM Collaboration With Multi-Agent Reinforcement Learning
- Understanding R1-Zero-Like Training: A Critical Perspective
- Poly-EPO: Training Exploratory Reasoning Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering
- Outcome-based Exploration for LLM Reasoning
- On Advantage Estimates for Max@K Policy Gradients
- What is the Alignment Objective of GRPO?
- Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problems
- DSDR: Dual-Scale Diversity Regularization for Exploration in LLM Reasoning
- Text Embeddings by Weakly-Supervised Contrastive Pre-training
- Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
- It Takes Two: Your GRPO Is Secretly DPO
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis