On Generalizing Preference-based Reinforcement Learning to Multi-Objective Problems
This work generalizes preference-based reinforcement learning by formalizing a setting where human experts can label trajectory pairs as incomparable, not just preferred or dispreferred. It introduces a novel Multi-Objective Bradley-Terry model (MOBT) to learn from these comparisons, allowing the agent to recover multidimensional reward functions and traverse the Pareto frontier in multi-objective problems.