Automatic measurement of social gaze during naturalistic conversations in autism

Standardized, granular measurement of social communication behaviors, such as social gaze during natural interactions, is needed for a range of psychiatric applications including diagnosis and detecting clinical change in conditions such as autism. Computational approaches show promise in automatically measuring social behaviors within natural settings. This study aims to automatically measure social gaze features from videos of dyadic conversations, characterize autism-related differences, and capture individual-level differences. 46 autistic Participants and 36 neurotypical Participants, aged 8-29 years, engaged in a brief video-recorded conversation with a research staff member (Partner). An automated social gaze detector was trained to detect whether each partner was looking at the other and achieved 89% cross-validated accuracy with human annotations. Comparing detailed automatic gaze measurements, autistic Participants spent less time looking at Partners and engaging in mutual gaze than neurotypical Participants did. They also initiated mutual gaze less frequently and had shorter mutual gaze episodes, but did not differ in mutual gaze counts. A social gaze summary score fusing individual variables correlated specifically with ADOS-2 Social Affect scores and not Restricted and Repetitive Behavior scores. A cross-validated multivariate classification model using gaze features was able to make individual-level diagnostic prediction (autism versus neurotypical) with 73% accuracy. This study provides a framework for automatically quantifying social gaze behaviors with high granularity and demonstrates its use in psychiatric research. This framework has a potential for enhancing measurement of social skills and tracking therapeutic progress in autism and other psychiatric conditions.