LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback

arXiv:2607.29559 · cs.AI, cs.RO · Submitted 2026-07-31 · Read on arXiv

cs.AI, cs.RO

Submitted: 2026-07-31

Updated: 2026-10-12

Terminology

• Reward Function — 5× in this paper · explained in 3 episodes
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes

Sources

• Concrete Problems in AI Safety
• Vector Policy Optimization: Training for Diversity Improves Test-Time Search
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• Multi-Principal Assistance Games
• A Practical Guide to Multi-Objective Reinforcement Learning and Planning
• Playing Atari with Deep Reinforcement Learning
• Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Tasks
• Benchmarks and Algorithms for Offline Preference-Based Reward Learning
• General Preference Reinforcement Learning
• MODULI: Unlocking Preference Generalization via Diffusion Models for Offline Multi-Objective Reinforcement Learning

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems