Preference-based Reinforcement Learning for Robotics: Advancements and Open Challenges

Preference-based reinforcement learning (PbRL) is a promising approach to enable robots to learn from human feedback and adapt to individual preferences. In PbRL, a human teacher provides comparative preferences between robot behaviors, which are used to learn a reward function that aligns with the human’s expectations. While PbRL offers advantages over traditional approaches like reinforcement learning and learning from demonstration, it still faces challenges that limit its real-world applicability, such as the need for large amounts of human feedback, the difficulty of learning from implicit or intermittent feedback, and the limited generalization capabilities. This survey provides an overview of recent advances in PbRL for robotics, focusing on techniques aimed at addressing these key challenges. By identifying the strengths and limitations of existing approaches, we provide a roadmap for future research directions to realize the full potential of PbRL in enabling robots to learn and adapt to complex, real-world environments.