Sound has always been all around us, but new technologies are enabling the creative industries to employ audio to immerse us more deeply into music, theatre, spoken word, movies and games.
In 1978 Lou Reed released an album called Street Hassle. It was a mix of studio and live recordings and, as usual for Lou Reed, a raw and uncompromising piece of work. But Street Hassle is perhaps most interesting now because much of it was recorded binaurally, with two small microphones located in the ears of a manakin head. Shut your eyes and listen on headphones and the binaural sound transports you back to where and when it all took place. So the roots of the “immersive” or “VR” audio that is these days becoming almost impossible to ignore go back quite a way. Actually, they go back much further even than Lou Reed in 1978. For example, in 1921, WPAJ Radio in Connecticut experimented with broadcasts in binaural stereo by employing two separate radio frequencies and instructing its listeners to employ two radios with an ear-piece connected to each one. It didn’t take off; not least because even one radio was financially out of reach for many 1920s Connecticut folk. But those two early examples of what might now be described as “immersive” perhaps show how long we’ve been fascinated with the trick that multi-channel audio can play; creating ghostly, virtual audio images where nothing physically exists. Audiophiles, myself included, have long been seduced by the nebulous concepts of stereo “image depth” and “focus”, and back in the 1970s when the BBC designed it’s own studio monitors, their designers would analyse stereo performance partly by measuring, with protractors and string, how sharply “focussed” were the images the monitors created.
So the concept of immersive audio, and audio as an element of virtual reality, is in some respects nothing new. Some things have changed however. Firstly, since the arrival of the iPod and smartphones everybody’s taken to wearing headphones, so the sound of binaural audio that previously only solitary audiophiles in darkened rooms were conscious of, is becoming, much more widely experienced. From Gareth Fry’s, sound design for ‘The Encounter’ that wowed Edinburgh, London and Broadway, to the BBC’s binaural Promenade Concert broadcasts, and to lesser known examples, such as sound designer Helen Skiera’s audio guide at the Maiden Castle iron age hill fort in Dorset, the power of binaural audio to shift a listener’s consciousness to a different time and place is becoming part of the cultural landscape.
Secondly, the technological ability to record, store, post-process and distribute multiple channels of audio data has developed to the point where it is no longer a limiting factor. Audio over IP technology such as Audinate’s Dante means that where once a multi-core cable of mooring hawser dimensions was required for anything beyond a few tens of audio channels, many channels more can now be streamed in real time via a single network cable. Sound designers working across fields as diverse as games, movies, theatre, music and VR, or projects that combine any or all of those elements can have practically as many audio channels as they can imagine.
Thirdly, the arrival of visual VR technologies to gaming has understandably brought with it a need for audio to keep pace, with sound that can track with the events occurring within the virtual environment. And lastly, the introduction of Dolby Atmos, adding height information to movie front, centre and surround channels now brings the experience of audio as a 3D, rather than simply a lateral phenomenon, to thousands.
So, suddenly, both the wider cultural world seems ready for audio to create immersive virtual realities, and the technology is available to those who aspire to create them, but where will it lead? There’s a sense that nobody really knows, and that partly adds to the buzz. Numerous artists and arts organisations, in a diverse range of cultural fields, are rediscovering the power of audio through immersive and VR technologies, but in no sense has anything settled down to a formulaic way of doing things. Exploration is still underway and the territory being uncovering is as diverse as it is inspiring.
One example is Moods is a jazz club in Zurich that plays host to gigs and artists across a wide range of genres, but rather than just limit its audience to those in the house, Moods gigs are available on a subscription basis as audio-visual streams – both live and on-demand. And a proportion of the streams include an optional binaural stereo feed generated via a Sennheiser AMBEO microphone. Sennheiser has been one of the more prominent technological players involved in immersive audio. For example, both the binaural “dummy-head” microphone and the audience headphones used in The Encounter were from Sennheiser. Its AMBEO VR microphone array, as employed at Moods, integrates four mic capsules in a tetrahedral arrangement to captures audio from 360° both vertically and horizontally. The signals from the microphones can be post processed to create everything from conventional stereo to faux -binaural stereo to the “B-format” encoded spherical sound field required for full-fat, immersive VR.
The Sennheiser AMBEO microphone actually appears to be a reinterpretation of an older and now patent-free technology; namely the Soundfield microphone developed by Michael Gerzon and Peter Craven in the late 1970s and as such is not entirely new. Another product in the AMBEO range however, the SMART HEADSET perhaps points further ahead. The AMBEO headset is a relatively conventional in-ear headphone designed use with a smartphone but it also incorporates microphones in each earpiece that enable binaural recording. With an AMBEO headset, a user can in effect, become his or her own binaural dummy head and record audio exactly as they experience it. How long will it be before the AMBEO headset triggers an Instagram style app for sharing snatches of binaural audio that can place a remote listener in the “seat” where the recording was made? And I use the term “seat” intentionally, because it could of course be a seat at a gig, and the recording, an instantly shared binaural bootleg. Perhaps gig promoters and artists need quite rapidly to short-circuit that prospect with “official” binaural bootlegs. Maybe download codes could be integrated with the smart ticketing technology of RFID bracelets? A organisation called Melody is however approaching the same immersive gig territory from a different direction. Melody VR plans to stream live performances using multiple 360° cameras and microphones to enable users wearing VR headsets to experience the performance with the audience of even, virtually to join the performers on the stage. There will be no need to go to the gig for real.
And speaking of gigs, the Moods jazz club model is of course not the only approach to integrating immersive audio with the live experience. A more radical example was Beck’s extraordinary 2013 “in-the-round” re-imagination and performance of David Bowie’s ‘Sound and Vision’. The online stream of the performance incorporated visual and audio VR technologies that enabled those lucky enough to experience the original to navigate their perspective through the audience and even the band and orchestra. The VR stream is unfortunately no longer available, but a conventional version that provides some idea of the ambition of the project can be found here.
Perhaps however the last words here should be from Gareth Fry, the sound designer behind The Encounter, and the audio creative behind numerous immersive and VR audio projects across multiple genres, “I think we’re still a way off from finding the best uses of the technology, just as it took us many years to come up with the developed conventions of recorded music, or cinema, or theatre, that we have today. There are going to be lots of failed attempts along the way, and every one of those will lead us closer to understanding the best way to use it, and will lead to us hearing stories in ways we can’t even imagine now.”