Identifying eye movement behavior indicators of social competence during conversation listening: A study using HoloLens 2

Aims: This study aims to investigate the relationship between eye movement behaviors and social competence, particularly focusing on listening patterns in multi-person conversations. The goal is to identify objective, quantifiable behavioral indicators of social competence to inform the development of assessment tools and training interventions. Methods: A three-person setting was designed with two conversational partners and one primary listener, and immersive eye-tracking data were collected using Microsoft HoloLens 2 during conversations on naturalistic topics. Social competence was rated by a clinical psychiatrist using standardized behavioral criteria, while analysis targeted three pre-selected indicators: nodding frequency, selective attention allocation to speaker’s regions of interest (head versus shoulders), and vertical gaze stability during listening. Results: The results revealed that nodding frequency showed a strong positive correlation with social competence scores, indicating its potential as a robust nonverbal biomarker. Participants with higher social competence demonstrated greater attention to the speaker’s head region while minimizing focus on less informative areas such as the shoulders. Furthermore, individuals with higher scores exhibited significantly lower vertical gaze dispersion, reflecting more focused and stable attentional control during social listening. Conclusion: This study establishes reliable eye movement-based indicators of social competence, highlighting their potential for assessing and enhancing social skills in real-world interactions. By integrating multimodal behavioral analysis, the findings provide a theoretical and technical foundation for developing personalized, real-time feedback systems for social competence training.