The Point of Pointing: Deictic Gestures Modulate Attentional Shifts and Cognitive Load in Simultaneous Interpreting

Abstract Spoken language processing requires the integration of acoustic signals with visual information. Deictic gestures, such as pointing, may facilitate this integration by constraining the referential domain before linguistic disambiguation takes place. However, it remains unclear how multimodal cues modulate processing under increased cognitive load. Simultaneous interpreting provides a task environment for the examination of speech processing under substantial cognitive load. In this study, we combined the Visual World Paradigm (VWP) with pupillometry to investigate how congruent, incongruent, and neutral pointing gestures influence attention and cognitive load in 24 professional simultaneous interpreters. Eye-tracking revealed that congruent gestures elicited early anticipatory visual target fixations. Incongruent gestures directed gaze toward visual competitors before rapid correction, while the absence of gestures delayed visual target identification. Counterintuitively, the gesture-absent condition produced greater pupil dilation than the incongruent condition, while congruent and incongruent conditions did not differ significantly in cognitive load. These results challenge the assumption that cue accuracy drives processing cost and instead suggest that the presence of a visual cue, even a misleading one, is the critical variable that reduces cognitive demands.