6 Ways 3D Audio Can Expand Gaming Experiences

“Sound is that which does not simply add to, but multiplies the effect of the image.”  Akira Kurosawa. 

Intro:

Over the past few years, we’ve witnessed sound for cinema being transformed from 5.1/7.1 to the high channel-count and object-based sound formats available for today's film-makers to extend 'surround sound' to the next level of immersion for audiences (i.e. Dolby ATMOS, DTS Multi-Dimensional Audio, and Auro Technologies’ Auro 3D). These formats add speakers all around, including above the listener to emit 'elevation cues', allowing sound to pan around in all three dimensions.

Games can also use elevation cues to further expand audio in all three dimensions (without the need for additional speakers by using stereo headphones, for example); through the use of digital filters (HRTFs derived from dummy-head measurements or modeling of the auditory cortex), a convincing 3D 'effect' can be created by leveraging  a sound object's XYZ position in game space to simulate a sound being heard at a corresponding point in a 3D sound-field.  

Yes, 3D audio technologies and solutions have been around for quite some time but the majority of today's game developers do not implement true 3D positional audio into their games; instead developers tend to stick with standard stereo and 5.1 surround sound as the de facto standard for the vast majority of titles being produced today. Keep in mind that not all 3D audio solutions are alike and really compelling 3D audio typically requires quite a lot of processing resources — precious resources that aren't always made available for audio processing. With the explosion of multi-core processors for mobile gaming, powerful next-gen consoles arriving later this year, and dedicated audio DSP resources being offered on new GPUs (i.e. AMD's TrueAudio), the conditions are ripe for the mass adoption and standardized use of compelling and powerful 3D audio solutions by game developers for their game titles across many different platforms.

Here is what 3D sound can provide game developers:

Elevation: placement above and to some degree, below

Azimuth: placement in front, behind, and to the sides

Distance: using elevation and azimuth to define a position for a sound inside a 3D sound-field (i.e. a spherical sound-field wrapped around you) the sound can be pushed out in space with distance to allow improved depth perception (using distance cue processing, 3D room simulation, for example.)

These parameters expand the sound-field beyond what stereo, 5.1, and 7.1 can produce (which are 2D formats). For developers looking to offer a bit more realism and immersion for their players, 3D audio may be the 'lowest-hanging fruit' available to do so.

Here are 6 ways developers can use 3D audio to expand gaming experiences:

There are tons of fun games on the small screen (for your mobile phone or tablet) however, 3D audio can be used to create the illusion of a larger game world through the use of an immersive sound-field that envelops the player. While there may be a bit of a disconnect between a small screen and an enveloping, spherical sound-field, 3d audio for mobile gaming can heighten the feeling of being "in the game."

Ear Monsters, by Ear Games, is a forward-thinking iOS game that employs the use of 3D audio to drive gameplay, rather than visuals.  The use of 3D auditory cues to drive gameplay serves to extend the boundaries of the game world beyond the small screen of the mobile device, for example, the player can tap in the general direction of the sound being heard in 3d space (for example, tapping the top of the screen if an attack is heard coming from above.)

Using 3D audio, sound-centric games can:

This list barely scratches the surface with all the possibilities that sound-centric games can offer players.

3D audio can improve situational awareness in FPS games by relaying sound cues from certain directions correctly to the player. For example, in most modern combat FPS games that rely use stereo, 5.1, or 7.1 audio delivery, when a player on the ground level is being shot at by a sniper who's perched high in a tower, the sound of the sniper’s shot does is not heard from above. With 3D audio, the sound of the sniper’s shot would be heard from above, like it's supposed to. 

It is well known that using sound to substitute for UI (i.e. running out of ammo or player health indication using sound prompts) can help remove display clutter. 3D sound can be used to extend the sound cues by using space to indicate more information (i.e. using a sound cue with elevation to indicate a rainstorm is coming.)

By leveraging 3D audio, a '3D radio communications' of sorts can be created to hear team member communication based on actual position in the game (i.e. if a member of a squad is up on a hill, the radio communication from that player would be elevated.)

Conventional stereo, 5.1, or 7.1 audio playback limits the sound field to a two-dimensional plane creating a disconnect with the visual field offered by VR. 3D audio eliminates this problem by allowing a full 3D sound-field to perfectly compliment the 3D stereoscopic visual field; head-tracking (with the Oculus Rift, for example) coupled with 3D audio allows the player to move their head around and expect to hear sound all around them correctly, including sounds from above.

How can developers implement 3D audio in their games?

Games typically use audio middleware solutions (FMOD and Wwise, for example) for their run-time sound engines so developers can utilize 3D audio technologies made available from Dolby, DTS, GenAudio, Auro 3D, and Iosono (check with your audio middleware provider for available solutions.) Encourage your teams to explore using either of the various 3D audio solutions available today to enhance the aural experience for your title(s) beyond what stereo, 5.1, and 7.1 can offer.