Integration of Musculoskeletal and Autonomic Ocular Cues in Virtual Humans: Effects on Emotion Perception and Authenticity

Virtual human facial emotion expression is typically driven by musculoskeletal control, whereas autonomic physiological cues remain underused. We conducted a virtual reality study with a 5×5 factorial design that crossed five levels of musculoskeletal valence with five ocular states ranging from high parasympathetic activity to high sympathetic activity. Ocular physiology was implemented through parametric control of pupil diameter and scleral redness. Nineteen participants rated each stimulus for perceived valence and authenticity. While musculoskeletal cues were the primary driver of perceived valence, ocular physiology produced significant modulatory effects, even at the resolution of current headsets. This indicates that observers are sensitive to subtle ocular changes. For authenticity there was no direct effect of physiological cues, but there was a significant interaction with musculoskeletal cues, showing that authenticity judgments depend on their combination. Together, these findings suggest that adding physiologically plausible ocular signals enhances the social believability and improves the perception of emotions in virtual humans, with implications for training, serious games, social robotics, and behavioral science.