iMMERSiVELY: VR Experience

14 IN DISCUSSION

THE WHATS AND WHYS OF VIRTUAL REALITY AUDIO - A CREATOR’S PERSPECTIVE

To create a perfect illusion, it’s not only visual content that has to be hyper-realistic, sound has to be, too. Daniel Soo and Lionel Chok from Singapore-based start-up iMMERSiVELY discuss how they develop spatial sound and the reasons why it is such an important part of the Virtual Reality experience.

Ambisonics and spatial audio have been a recent talk of the town and an important component of Virtual Reality (VR). As a team of creators specialising in creating content using immersive technologies like augmented, virtual and mixed reality, we knew that spatial audio would play an essential role in our work. So, let’s cover the basics: What is ambisonics? Ambisonics is a multichannel spatial audio technique that lets you spherically capture sound arriving from all directions, at a single point in space. For playback, this representation is rendered binaurally, allowing you to virtually rotate the orientation of the perspective in all directions, horizontally and vertically. This technique is supported by all of the major post-production and playback tools on the market today. This makes ambisonics the appropriate tool for VR and all other applications involving 3D sound. Some time back, our team was tasked to reimagine and recreate the environment of a few iconic locations in olden day Singapore. For this project, we employed spatial audio instead of mono or stereo audio. Why? Because spatial audio makes for a much more active and immersive experience for the viewer. With spatial audio, turning your head in the VR headset changes

• Small, elegant and full of impressive features: the AMBEO VR Mic by Sennheiser.

the orientation of the soundscape. These environments were created by superimposing digital paintings that depicted the past onto present-day scenes. With spatial audio, we could use positional sound to cue the viewer’s attention as parts of the scene revealed itself around them. In our case, the ring of cowbells or a shout was soon followed by a bullock cart or rickshaw man respectively. Although the digital paintings were still images, the use of spatial audio and image together gave viewers the illusion of the cart and the man coming towards them, as if they were actually inhabiting the reconstructed environment. We came to know about the Sennheiser AMBEO partnership programme and decided to get on board. The program has dramatically helped in supporting us with the audio hardware and software that top-notch VR content requires. Our relationship with Sennheiser has been immensely valuable to us. The program covers products and technologies for the entire audio signal chain, from capture to mixing, and processing to reproduction. Before the AMBEO technology came along, iMMERSiVELY had been using a portable four-channel recorder. One of the major challenges that we had with it was the sound quality - it was less impressive than that of the AMBEO VR Mic. While it was sufficient for simpler jobs,

our early setup would not have given us the control and quality we needed to create the soundscapes as convincingly as we now do. The AMBEO VR Mic packs impressive features, functionality, and quality into a small and elegant form factor; and the AMBEO plug-in makes conversion a breeze. It works very well with Reaper (our current digital audio workstation of choice) and the Facebook 360 Spatial Workstation - making the entire VR production and post-production process enjoyable. As VR content creators, we believe that spatial audio is and will continue to be an incredibly critical element when it comes to creating compelling VR content. Spatial audio applications and their value will only increase as more people begin to appreciate the unique affordances and challenges that VR as a medium pose. We are excited to experiment with projects that encourage spatial audio to take centre stage. Instead of relying mostly on detailed visual representations to tell the story, how can we start using minimal but evocative visuals in service of sound to build an intricate and compelling virtual environment? www.immersively.co

The opinions expressed in this article are the opinions of the author and not necessarily of mondo*dr magazine.

www.mondodr.com