Reinforcement Learning with Reference-Based Rewards for Language Model Reasoning and Alignment
cs.CL, cs.AI, cs.LG
Submitted: 2025-04-14
Updated: 2026-11-18
Code: https://github.com/mzhaoshuai/RefAlign
Terminology
Sources
- GPT-4 Technical Report
- Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
- A General Theoretical Paradigm to Understand Learning from Human Preferences
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Fine-grained Image Captioning with CLIP Reward
- The Llama 3 Herd of Models
- Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
- Direct Language Model Alignment from Online AI Feedback
- Teaching Large Language Models to Reason with Reinforcement Learning
- CLIPScore: A Reference-free Evaluation Metric for Image Captioning
- LoRA: Low-Rank Adaptation of Large Language Models
- REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
- OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
- Mistral 7B
- Language Models (Mostly) Know What They Know
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- Teaching Models to Express Their Uncertainty in Words
- DeepSeek-V3 Technical Report
- WebGPT: Browser-assisted question-answering with human feedback
- Disentangling Length from Quality in Direct Preference Optimization
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data