Visual perception support of sound movement using peripheral displays in VR space

In recent years, research on sound visualization for people who are deaf and hard of hearing has gained attention. Previous studies have mainly focused on displaying the names, types, or onomatopoeic representations of sounds heard by people with no hearing impairments on a screen, thereby supporting the daily lives of people with hearing impairments. However, few studies have addressed the visualization of the sound movement from moving sound sources, aiming to enable people with hearing impairments to perceive changes in sound movement in a way similar to people with no hearing impairments, for example, in the context of hazard avoidance. This study aims to support deaf and hard-of-hearing individuals and older adults in visually perceiving the sound movement from moving sound sources. We propose a method in which off-the-effective-field-of-view displays are placed in front of the user in a virtual space using a head-mounted display, and visual representations corresponding to the pitch and loudness of sounds from sound sources moving out of the user’s effective field of view are presented. This allows the user to recognize the presence and movement of sounds without shifting their gaze.