When we talk about virtual reality's ability to transport us to whole new worlds, we often focus on the visual elements – the resolution of the displays, the quality of the head tracking, and so forth. But just as important is the audio work, as fresh research again cements.
A team from the Institute Jean Le Rond d'Alembert, Pierre and Marie Curie University, in Paris, are showcasing a new virtual reality demo that shows the importance of audio modelling in VR experiences.
The project uses a live show of the 19th century opera La Vierge, as recorded at the Cathedral of Notre Dame in Paris.
The Cathedral is famed for its unique acoustics, making music recorded there sound different to anywhere else in the world. And that's where the VR experiment gets interesting.
Acoustic modelling
Rather than just presenting a stereo sound field to accompany the VR visuals, the team instead reproduced the recordings as they would sound given the specific acoustics of the Cathedral.
A combination of computerized acoustical data, paired with 3D visualisations that model the interior architecture, allow the listener to hear the recording as it would have seemed in relation to different positions in the building.
"The importance of multimodal interactions, how visual and auditory cues balance in spatial perception, is key to VR and the sense of immersion, of being 'in' the VR world," said Brian FG Katz, lead investigator and CNRS Research.
It's research that will become increasingly important as virtual reality progresses – as we approach photo-real graphics and improved eye tracking techniques, audio that convincingly matches the specific dimensions, materials and atmospheric conditions of your digital surroundings will be vital to maintaining that all-important sense of "presence".
As well as using the information gathered from the project to better understand how visually impaired people navigate the world, the team will also be looking at how an individual's head and ear qualities can affect an audio experience.
- Via Phys.org
