Probe the Harness: Setup Checks for Stale-Data RL Comparisons in Language Models
cs.LG, cs.CL
Submitted: 2026-10-02
Updated: 2026-10-02
Sources
- Training Verifiers to Solve Math Word Problems
- AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
- Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
- A-3PO: Accelerating Asynchronous LLM Training with Staleness-aware Proximal Policy Approximation
- ASymPO: Asymmetric-Scale Policy Optimization for Asynchronous LLM Post-Training Without Behavior Information
- Delightful Distributed Policy Gradient
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- HybridFlow: A Flexible and Efficient RLHF Framework
- Scaling Laws for Collapse in Asynchronous GRPO
- How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
- Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
- Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale Data on LLMs?
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks