Delayed Auditory Feedback and the N170: investigating the effect of the modality of the learning material

The N170 is an event-related potential component that plays an important role in visual processing and is particularly prominent for faces (Bentin et al., 1996; Itier & Taylor, 2004; for a review, see Yovel, 2016) and words (for a review, see Carreiras et al., 2014). It peaks over the occipitotemporal cortex about 170 ms after the presentation of faces or words (for a review, see Sigurdardottir et al., 2021). For both faces and words, an origin has been described in or near the fusiform gyrus (Brem et al., 2006; Deffke et al., 2007; Gao et al., 2019; Iidaka et al., 2006), which contains specialized regions for face perception (Kanwisher et al., 1997), word processing (Cohen et al., 2002) and other stimulus categories (for an overview see Weiner & Zilles, 2016). In the last decade, the N170 has also been investigated in response to feedback stimuli during feedback learning, and studies found increased N170 amplitudes for delayed (about 6-7 s) compared to immediate feedback (Arbel et al. 2017; Höltje & Mecklinger, 2020; Kim & Arbel, 2019; but see Albrecht et al., 2023 for opposite pattern). As findings from a functional magnet resonance imaging (fMRI) study show that activity in the hippocampus is increased when processing delayed compared to immediate feedback (Foerde & Shohamy, 2011), it has been hypothesized that the feedback-locked N170 represents hippocampal activity (Arbel et al., 2017; Kim & Arbel, 2019). Alternatively, as the origin of the N170 in response to visual stimuli has been shown to be in the fusiform gyrus, the N170 could indicate the recruitment of higher-order visual areas during feedback processing. Based on fMRI studies (Pleger et al., 2008, 2009; Schiffer et al., 2014), we hypothesize that when feedback is presented, the neural representation of a previously selected stimulus with which the feedback is associated is reactivated in sensory cortices, especially when feedback is delayed. The hippocampus might mediate this reactivation. The N170, reflecting the processing of visual stimuli, should then be more pronounced when feedback is associated with a visual stimulus compared to stimuli in other modalities. In this line, Röhlinger et al. (2025) tested whether the N170 is influenced by the modality of the stimulus the feedback refers to. They presented participants a probabilistic learning task in which they could choose between visual or auditory stimuli and differentiated between immediate and delayed visual feedback. As expected, they found that for delayed feedback, the N170 was more pronounced when participants learned associations between feedback and visual stimuli than between feedback and auditory stimuli. Moreover, a prediction error effect was visible for the N170 with larger amplitudes for unexpected positive compared to expected positive and for expected negative compared to unexpected negative feedback, but only for learned associations between feedback and visual stimuli in delayed feedback. Because feedback in the study by Röhlinger et al (2025) was visual, a potential confound is the modality switch between auditory learning material and visual feedback, which might have impeded learning processes and led to the reduced N170 for auditory material. In a follow-up study, we thus aim to explore whether the reported findings concerning the modality of the learning material can be replicated when feedback is given auditorily. Therefore, we will use the same procedure as in Röhlinger et al., 2025, but change the modality of the feedback from visual to auditory feedback. Kim & Arbel (2019) found the N170 modulated by feedback valence and timing when feedback was given auditorily. We hypothesize that also when feedback is presented auditorily, the N170 would be more pronounced if it refers to visual stimuli than auditory stimuli, especially when feedback is delayed. The planned study will shed further light on the question whether the amplitude of the feedback-locked N170 depends on the modality of the chosen stimulus, and whether this effect is modulated by feedback timing and feedback modality when feedback is auditory. For this purpose, we will record brain activity using electroencephalography (EEG) while participants learn to associate visual vs. auditory stimuli with immediate and delayed positive or negative auditory feedback.