Decoding Engagement: Exploring the Influence of Social Communication Cues in Human-Robot Conversational Scenarios

Enhancing human-robot interaction (HRI) requires a deep understanding of engagement. This paper explores how to capture engagement within conversational HRI by examining multiple non-verbal communication cues. Specifically, we analyze the engagement of 16 participants in the context of a reminiscence task, which establishes a foundational dialogue between the robot and the user. Head pose and voice activation were the non-verbal communication cues used in this study. We developed a methodology that identifies and correlates these cues with engagement levels during specific epochs of interaction. By contrasting these cues against video annotations and self-reported feedback, we refine our understanding of engagement. Our results demonstrate that a multimodal approach to analysing engagement significantly outperforms unimodal analysis, where 11 out of 16 cases (68%) correlate with engagement, increasing accuracy compared to using them independently. Critically, the study reveals that objective and subjective methods to analyse engagement closely align with users' perceptions of engagement, emphasising the importance of integrated communication strategies in HRI.