Provable Zeroth-Order Safe Exploration for Reinforcement Learning from Human Feedback
cs.LG
Submitted: 2026-11-05
Updated: 2026-11-05
Terminology
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
- Lyapunov-based Safe Policy Optimization for Continuous Control
- Safe RLHF: Safe Reinforcement Learning from Human Feedback
- Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
- LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking
- Worst-case iteration bounds for log barrier methods on problems with nonconvex constraints
- PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference
- Safe Posterior Sampling for Constrained MDPs with Bounded Constraint Violation
- Learning-based Model Predictive Control for Safe Exploration and Reinforcement Learning
- Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback
- Sample-Efficient Constrained Reinforcement Learning with General Parameterization
- Qwen2.5 Technical Report
- Reward Constrained Policy Optimization
- Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
- Making RL with Preference-based Feedback Efficient via Randomization
- Provable Reward-Agnostic Preference-Based Reinforcement Learning
- Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysis
- Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis