The Influence of Visual Cues on the Sound Externalization of Virtual Loudspeaker Setups

Research has shown that visual cues influence the perceived sound externalization of binaural reproduction. However, it remains unclear whether providing a virtual visual representation of loudspeakers in addition to their binaural rendering influences their perceived externalization. Using augmented reality (AR), loudspeakers can be visualized within the listener’s environment, potentially improving the externalization of binaural loudspeaker renderings. This study presents a listening experiment using dynamic binaural rendering of a typical 7.0.4 immersive loudspeaker setup. Participants rated the externalization of noise bursts rendered as single loudspeaker sources, phantom sources, and moving sources, as well as immersive music, under three different visual conditions. Results show that the AR visualization significantly improves the externalization for noise bursts reproduced by single virtual loudspeakers and for immersive music excerpts reproduced by the entire virtual loudspeaker setup but not for phantom and moving sources. Thus, shifting a percept farther away from the head seems to work best when the acoustically perceived sound direction matches the visual source location. The effectiveness of AR visualization for immersive music, on the other hand, may be due to listeners’ familiarity with spatial music productions over loudspeakers.