Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind

Abstract

Adaptation to user preferences and the ability to infer and interpret human beliefs and intents, known as the Theory of Mind (ToM), are two critical aspects of effective human-robot collaboration. Despite their importance, very few studies have investigated the impact of adaptive robots with ToM capabilities. This work presents an exploratory comparative study to investigate how social robots simulating ToM capabilities affect user performance and perception. We design a two-tier architecture. The Q-learning agent on the first layer learns the robot’s higher-level behaviour. On the second layer, a heuristic-based ToM infers the user’s intended strategy and is responsible for implementing the robot’s assistance, as well as providing the motivation behind its choice. We conducted a user study in a real-world setting, involving 56 participants who interacted with either an adaptive robot simulating ToM capabilities, or with a robot lacking such capabilities. Our results suggest that participants in the ToM condition performed better, accepted the robot’s assistance more often, and perceived its ability to adapt, predict and detect their intentions to a greater extent. Our preliminary findings could inform future research and pave the way for the design of robot architectures for adaptive behaviour with ToM capabilities, where the choice of action is decoupled from its implementation.