SALT: When More Rollouts Don't Help in Group-Based Policy Optimization and How to Make Them Matter
cs.LG
Submitted: 2026-06-04
Updated: 2026-11-04
Code: https://github.com/meta-llama/llama3
Terminology
Sources
- Program Synthesis with Large Language Models
- Group Policy Gradient
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Group-in-Group Policy Optimization for LLM Agent Training
- Reinforcement Learning with Deep Energy-Based Policies
- Fast Generalized Neural Tangent Kernel Statistics via Trace Estimation
- Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
- ScRPO: From Errors to Insights
- Let's Verify Step by Step
- Asynchronous Methods for Deep Reinforcement Learning
- Training language models to follow instructions with human feedback
- Estimating Training Data Influence by Tracing Gradient Descent
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- On Entropy Control in LLM-RL Algorithms
- HybridFlow: A Flexible and Efficient RLHF Framework
- Qwen3 Technical Report
- Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis