The Science behind Virtual Reality

When referring to the science of virtual reality (VR), it distills down to the illusion of being human—the illusion of how we perceive the world through the senses. VR isn’t about creating 3D virtual environments (VEs) that the user can walk around in; it’s about creating immersion and presence ( Fuchs et al., 2011a , Fuchs et al., 2011b , 2011c , Fuchs et al., 2011a , Fuchs et al., 2011b , 2011d ) where the user truly believes they’re in a 3D VE. Immersion can be defined as the user’s degree of sensory engagement within the virtual simulation ( Kim & Biocca, 2018 ), while presence is the perceptual illusion of being there ( Slater, 2018 ). Slater identifies the correlation but difference between immersion and presence, where immersion is the objective property of the VR system, while presence is the subjective ( Slater, 2018 ). This immersive presence is not achieved through visuals alone. Although visuals are the most critical component, a cross-modal integration between other senses must be achieved to create a stronger presence for the user ( Malpica et al., 2020 ). A wooden door within a VE should contain visual attributes—scale, depth, and textures—that make it believable, but also auditory—it creaks as the user rotates it open and clicks when it shuts—and tactile—the user operates on its handle and is made aware of their body relative to the space as a result. This sixth sense, known as proprioception, is crucial to VR as the user perceives themself as the center of the VE ( Fuchs et al., 2011a , 2011b , 2011c , 2011d ), thus making them aware of their position, movement, and action. To understand the science of VR, this chapter will start with the fundamentals. It will describe the anatomical and neurological processes in which we perceive these senses. Research and technology of the visual, auditory, and haptic/proprioception senses for VR have greater development compared to other senses, such as taste and smell. The former will be the focus of this chapter. With a basic understanding of human sensory perception, this chapter will proceed to describe the three levels of VR: non-immersive, semi-immersive, and fully immersive, and provide examples for each category. Using these examples, this chapter will present the technological developments that correspond to the mechanisms of human perception—the science behind VR.