Cross-Behavior Policy Optimization: Unified Learning from Offline Demonstrations and On- and Off-Policy Rollouts via a Shared Regression Target
cs.LG
Submitted: 2026-10-09
Updated: 2026-10-09
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Sources
- Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning
- Soft Policy Optimization: Online Off-Policy RL for Sequence Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
- Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
- Solving Quantitative Reasoning Problems with Language Models
- RePO: Replay-Enhanced Policy Optimization
- Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- RL-finetuning LLMs from on- and off-policy data with a single algorithm
- Buffer Matters: Unleashing the Power of Off-Policy Reinforcement Learning in Large Language Model Reasoning
- Off-Policy Value-Based Reinforcement Learning for Large Language Models
- Group Sequence Policy Optimization
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks