By Winifred Phillips | Contact | Follow

Hey, everyone!  I'm videogame composer Winifred Phillips, and my work has included the musical scores for top games on all sorts of popular gaming platforms, from handhelds and mobile, all the way up to the latest consoles and PCs.  Lately, I've been doing a lot of video game music composition for virtual reality.  I had the pleasure of presenting a lecture on Music in Virtual Reality (pictured left) at the most recent Game Developers Conference in San Francisco.  My experience as a composer for VR includes many VR games, including the Scraper: First Strike shooter (set to be released for the PSVR, Oculus Rift and HTC Vive in December 2018), and the recently released VR experience The Haunted Graveyard, which is now available on Steam and in VR Arcades around the world.  Since we're in the Halloween season, and this VR experience is designed specifically for your Halloween pleasure, here's a trailer that features my music from The Haunted Graveyard:

By virtue of all the experiences I've had recently creating music for VR, I've become keenly aware of the importance of sound fidelity in VR.  If the experience doesn't sound real, it loses the chance to actually feel like a fully-convincing, thoroughly awesome virtual reality experience.  With that in mind, I've been writing periodic articles about new technologies in connection with headphones for VR.

In the previous article from last month, we discussed developments with technologies that were introduced years ago and have made recent advances.  We also noted the disappearance of products that failed to live up to their promises and were never released to the public.  Now, let's turn to some brand new developments in the area of headphone audio for virtual reality projects.  In this article, we'll look at some newly announced products, and we'll also expand our discussion to encompass new technologies that promise to enhance existing stereo headphones for VR uses.  So, let's start first with some headphones that could come in handy not only for VR gamers, but for indie VR audio developers as well!

The Ambeo Smart Headset

Sennheiser is an expert manufacturer of high-end headphones and microphones for audio professionals and consumers.  Their catalog includes a product line designed for gamers, but nothing in the specs of these headphones makes them especially suitable for VR use (beyond their virtues as quality headphones). So instead, let's take a look at a new Sennheiser product design that takes headphones in a surprising direction with relevance to VR.

At the beginning of 2017, Sennheiser debuted an innovative new headphone product at the Consumer Electronics Show.  At the end of that year, their new product hit retail.  Usually, headphones live or die by their innate sound quality, but that isn't the primary selling-point of Sennheiser's new product.  With standard dynamic drivers and some noise-cancellation features, the headphones are capable of delivering an aural experience in VR that's perfectly acceptable - but not particularly exceptional.  Add to that the limitation imposed by the Apple Lightning connector (which is only compatible with iOS devices), and these headphones start to seem less than desirable. Admittedly, the Ambeo Smart Headset could serve as a satisfactory set of standard headphones for gamers who use their iPhones in combination with iOS compatible VR headsets such as Google Cardboard, Merge, VR One Plus, and other similar devices.  That said, there's nothing special on offer here... unless you're an indie VR developer.  In that case, you'll find a great additional feature that could make these headphones worth their $299.95 price tag.

What differentiates the Ambeo Smart Headset from any other prospective set of VR headphones is its capability to not only deliver convincing binaural sound for VR, but capture it as well.  The headset includes a pair of directional electret capsule microphones - one mounted on each earpiece, facing outward. When a user wears the in-ear headset, each microphone is positioned so that it captures the audio exactly as it reaches that ear, including all of its associated Head Related Transfer Functions.  In essence, the user has now become a living, breathing, full-featured binaural dummy head microphone.  Since the famous Neumann Binaural Dummy Head microphone retails for nearly eight thousand dollars, having a fully-functioning substitute that only costs $299.95 is a dramatic new development.

Sennheiser promises that a version of this headset with Android connectors will be released sometime in 2019, so we'll be keeping tabs on that.  In the meantime, here's a video produced by the B & H Photo Video site that demonstrates the Ambeo Smart Headset in action:

Giving your headphones a VR make-over

In several previous articles, we've looked at headphone models designed specifically to maximize the aural experience of a VR gamer.  Now that the initial wave of VR excitement has given way to the more steady growth necessary for long-term success, I've noticed that fewer headphones are entering the market touting VR-specific features.  Instead, we're seeing audio technology companies introducing ways to make the headphones we already own sound better in VR.  Since better-sounding headphones will mean that VR users will be able to better appreciate our video game music, let's take a look at a couple of these sound-enhancing technologies:

Cingo

As we know, any pair of headphones can deliver a fully spatialized, three-dimensional audio experience (as demonstrated memorably by the famous Virtual Barber Shop recording).  But the audio of a VR game doesn't consist of a single spatialized recording.  Instead, interactive VR audio can include numerous audio components that are combined by the game engine and delivered to the player's ears.  The resulting mix, when successful, will enfold the player in the game's aural universe... but getting that mix just right can be tricky.  This is where Cingo comes in.  It's a sound processing solution developed specifically for the challenges of optimizing mobile VR audio for headphones.  I got to try Cingo out during the Audio Engineering Society Convention in 2015 (pictured right).  

Cingo can be used by a VR developer as a complete audio solution for mobile platforms.  As an all-in-one solution, Cingo includes creation software (the Cingo Composer authoring plug-in for VST and AAX) and a post-processing engine that can be integrated either directly into the operating system of the VR device, or into the software of the VR game (wherein it would act essentially as audio middleware).  Cingo's audio enhancements for headphones in VR include loudness and equalization correction on-the-fly, and other aural adjustments based on head-tracking data.  

The Cingo technology is currently incorporated directly into the Google Nexus operating system for Android, the Google Pixel line of laptops and smartphones, the Samsung Gear VR (pictured), and the LG 360 VR device.  Let's see the technology demonstrated in a video produced by the Fraunhofer-Gesellschaft organization (creators of Cingo):

Creative Super X-Fi

We all probably remember the ill-fated OSSIC X headphones, which recently met an ignominious end when the manufacturer shut its doors due to lack of funds.  One of the promised features of these headphones was "individual anatomy calibration," which would have allowed the headphones to formulate a map of the wearer's unique ears.  This map would then have been used to construct a set of customized Head Related Transfer Functions that would purportedly have given the headphone-delivered sound a sense of unparalleled realism and spatial depth.  While we'll never know if the OSSIC X headphones would have delivered on this promise, we can now turn our attention to another device claiming to achieve the same feat.

Creative Technology made a big splash at the Consumer Electronics Show 2018 with their Super X-Fi device (pictured left), which earned enthusiastic reviews and a Best of CES Award.  The Super X-Fi resembles a slim black dongle, with input jacks for the sound source at one end, and an output jack to carry the sound from the Super X-Fi to a pair of standard headphones.  According to Creative Technology, users will be able to take photos of their ears and face, and then the Super X-Fi will use these photos to match the anatomy of the user's head and ears to one of a library of anatomical profiles. Once this match is made, the Super X-Fi will apply the associated Head Related Transfer Functions (HRTFs) from the profile to the sound passing through the dongle.  When the sound reaches the user, Creative Technology promises that the aural experience will be extraordinarily convincing and natural.  No need to purchase special headphones when a simple dongle can make any decent headphones sound like a million bucks, right?

While the product is receiving pretty widespread acclaim right now, it's only available in Singapore, while consumers from other countries can only add themselves to a reservation list.  Also, there's been no discussion of how this technology might interact with binaural sound sources.  Would the introduction of customized HRTFs along the signal path enhance the binaural content, or distort it?  This issue remains unclear, but it's an intriguing new technology, and one worth keeping an eye on.  Here's a video demonstration of the prototype model as it was shown at this year's Consumer Electronics Show:

And then there were none

Finally, let's take a look at a new technology that seems lifted right out of a high-tech spy movie, or some futuristic sci-fi flick.  It's called Sowlo, created by Noveto Systems.  Traditionally, when we want to enjoy audio privately, we've always needed either a set of headphones, or some privacy and a locked door.  But Sowlo requires none of these things. That's because Sowlo directs sound precisely into our ears.  No earbuds or headphones required.  The system tracks our eyes, ears and face to get a read on where we are in the room, and then sends the sound directly to us.  The audio signal proceeds in such an ultra-narrow, precisely-targeted beam that there can be other people in the room, hearing nothing at all, while we're getting a full-force bombastic aural soundscape directed with laser-focus right into our ears.

One of Sowlo's advantages for VR applications is its ability to free users from headphones.  With a VR headset already taking up valuable head-mounted real-estate, it could well be a welcome relief for some gamers to get rid of the dangling cables and weight of headphones.  Plus, there have certainly been many gamers who just detest headphones as a general rule, and wish that they could wander virtual worlds using a surround speaker system instead.  Up until now, that desire has been thwarted by the inability of a surround-sound system to track with the user's head movements and deliver faithful spatialized audio.  But the makers of Sowlo claim that their system can track head movements and convey accurate positional audio for VR.

The technology is certainly intriguing.  Noveto currently estimates that the Sowlo devices will be available to purchase by fall 2019.  It's worth keeping track of this thoroughly unique approach to audio delivery, and we'll check in with Sowlo periodically to see how things are progressing.  In the meantime, here's a video produced by VR Focus about Noveto's Sowlo technology.

Conclusion

So we've taken a look at some new technologies for VR audio.  Some promise to improve the experience of headphone wearers in VR by virtue of aural enhancement or user customization.  Another does away with headphones entirely in favor of sound delivered in a private beam directly to the user.  And one technology transforms headphones into a tool for not only experiencing VR audio, but recording it as well.  These are interesting times in what we hope will be a long life cycle for virtual reality gaming.  We'll continue to revisit this topic from year to year, to see what advances are being made in the field of VR headphone audio.  I hope you enjoyed the article, and please feel free to leave a comment in the space below!

Winifred Phillips is an award-winning video game music composer whose projects include the triple-A first person shooter Homefront: The Revolution. She has composed music for games in five of the most famous and popular franchises in gaming: Assassin’s Creed, LittleBigPlanet, Total War, God of War, and The Sims. Her latest video game credits include numerous Virtual Reality games, including The Haunted Graveyard, Audioshield, Scraper: First Strike, Bebylon: Battle Royale, Fail Factory, Dragon Front, and many more.  She is the author of the award-winning bestseller A COMPOSER'S GUIDE TO GAME MUSIC, published by the MIT Press. As a VR game music expert, she writes frequently on the future of music in virtual reality games.

Follow her on Twitter @winphillips.