When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?
Xuanfei Ren, Tengyang Xie
stat.ML, cs.LG
Submitted: 2026-06-16
Comments: 69 pages
Project page: https://rltheorybook.github.io
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Directed Exploration in Reinforcement Learning from Linear Temporal Logic
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedback
- Outcome-Based Online Reinforcement Learning: Algorithms and Fundamental Limits
- Training Verifiers to Solve Math Word Problems
- Foundations of Reinforcement Learning and Interactive Decision Making
- Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation
- The Role of Inherent Bellman Error in Offline Reinforcement Learning with Linear Function Approximation
- Maximum Reward Formulation In Reinforcement Learning
- Risk-sensitive Reinforcement Learning Based on Convex Scoring Functions
- OpenAI o1 System Card
- Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models
- Do We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical Perspective
- Offline Reinforcement Learning with Implicit Q-Learning
- Near-optimal Regret Using Policy Optimization in Online MDPs with Aggregate Bandit Feedback
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
- Hybrid RL: Using Both Offline and Online Data Can Make RL Efficient
- Solving math word problems with process- and outcome-based feedback
- To the Max: Reinventing Reward in Reinforcement Learning
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches