Hearing in VR

Pros and Cons for users

  • Pro: To date, virtual reality has focused mainly on the visual aspects of immersion
  • Pro: Dialog in virtual reality applications is normally pre-recorded, so there is no reason why it can't be converted into captions and sign language
  • Con: Some virtual experiences rely heavily on audible cues to guide users
  • Con: Users may have difficulty distinguishing ambient noises from interface elements such as clicks, bleeps and transitions
  • Con: Users may not be able to perceive the location of spatialized audio
  • Con: Bluetooth and Wi-Fi latency and packet loss impacts upon the ability of users to perceive visual and audio events synchronously
  • Con: Audio dialog requires captions
  • Con: Sign language needs to be recorded separately
  • Con: Users may not be able to communicate using voice chat
Sign language speaker being recorded on a tablet
Phoebe Kaplan films Maggie Bork as she signs an American Sign Language poem. The video was incorporated into a virtual reality mobile application. Photo courtesy of University of North Carolina Greensboro.

Audio Interface

Examples of Auditory Cues 1
AmbientObjectMovementInformational
City street noise Air-conditioning hum Footstep sounds Audio instructions

Hearing Use Cases

  • As a user with a hearing impairment, I need to be able to mute ambient sound scapes, so I can hear interface elements such as clicks, bleeps and transitions.
  • As a user who hears better in one ear, I need to be able to control the location of spatialized audio, so I can hear interface elements.
  • As a user with a hearing impairment, I need a visual indication of the speaker, so I identify who is speaking. 2
  • As a user with a hearing impairment, I need captions of dialog and significant background noises, so I can read what is happening.
  • As a user with a hearing impairment, I need visual or haptic equivalents of aural interfaces, so I can interact with interface elements.

References

Next : Cognition