Can you see how I learn? Human observers' inferences about Reinforcement Learning agents' learning processes
Bernhard Hilpert, Muhan Hou, Kim Baraka, Joost Broekens
cs.HC, cs.AI, cs.RO
Submitted: 2026-08-21
Updated: 2026-08-24
License: http://creativecommons.org/licenses/by/4.0/
The gist: Reinforcement Learning (RL) agents often exhibit learning behaviors that are not intuitively interpretable by human observers, which can result in suboptimal feedback in collaborative teaching
Terminology
Abstract
Reinforcement Learning (RL) agents often exhibit learning behaviors that are not intuitively interpretable by human observers, which can result in suboptimal feedback in collaborative teaching settings. Yet, how humans perceive and interpret RL agent's learning behavior is largely unknown. In a bottom-up approach with two experiments, this work provides a data-driven understanding of the factors of human observers' understanding of the agent's learning process. A novel, observation-based paradigm to directly assess human inferences about agent learning was developed. In an exploratory interview study (N=9), we identify four core themes in human interpretations: Agent Goals, Knowledge, Decision Making, and Learning Mechanisms. A second confirmatory study (N=34) applied an expanded version of the paradigm across two tasks (navigation/manipulation) and two RL algorithms (tabular/function approximation). Analyses of 816 responses confirmed the reliability of the paradigm and refined the thematic framework, revealing how these themes evolve over time and interrelate. Our findings provide a human-centered understanding of how people make sense of agent learning, offering actionable insights for designing interpretable RL systems and improving transparency in Human-Robot Interaction.
Sources
- Agent-Agnostic Human-in-the-Loop Reinforcement Learning
- OpenAI Gym
- Utility-based Adaptive Teaching Strategies using Bayesian Theory of Mind
- A Review of Communicating Robot Learning during Human-Robot Interaction
- "Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations
Related papers
- Calibrating LLM Explanations to Human Decisions with Reinforcement Learning
- Rethinking Opinion Measurement Interfaces for Human-in-the-Loop Optimization
- EduGage: A Multimodal Dataset and Benchmark for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning
- EvoDesign: Agentic Editable Diagram Creation via Design Expertise Evolution
- HAGI++: Head-Assisted Gaze Imputation and Generation
- Explanation User Interfaces: A Systematic Literature Review