Music That Leaves a Mark

Audio can be used to elicit emotion from our players, draw attention to something, emphasize an event, or set the pace for gameplay. Music is what draws us into the game world, it sells us on the fantasy of the world we’re entering when we pick up that controller.

In this article I want to explore how we can squeeze some extra juice out of those audio tracks. How we make our music and sound effects even more engaging.

I’ll never forget the first time I played Devil May Cry 5 where Dante enters Limbo for the first time. All of the visuals in the game shift alongside a heavy metal track. The entire environment bumped along with the music as if it was alive. Here’s a video of the scene if you haven’t seen it. The shift to limbo happens around 1:30, gameplay starts at 3:00 (warning, mature content) https://www.youtube.com/watch?v=LRmhjJTXdp4

That was the first time I realized how big of an impact audio can make when it’s seamlessly engrained in the environment. It made me think back to other games I’d played where visuals accompanying the music. Games like Audio Surf: https://www.youtube.com/watch?v=Js-lAXTixV4 and Luminess: https://www.youtube.com/watch?v=yGCrEmTo6h0. I’m sure you can think of others.  The point is, adding visuals to accompany your soundtrack can really leave an impact on your players.
AudioSurf

Luminess

I’m a Unity developer myself, and I found they have a couple methods that are really nice for working with audio:

AudioClip.GetData() http://docs.unity3d.com/ScriptReference/AudioClip.GetData.html

AudioSource.GetSpectrumData()

http://docs.unity3d.com/ScriptReference/AudioSource.GetSpectrumData.html

Basically they allow you to sample the audio every frame, what you do with that data is up to you. I encourage you to play with them and see what you can come up with. I spent the last few months exploring different ways to use this data. I realized you could scale objects to music, change their position, color, or material, adjust lighting, or cue any custom event based on the beat. However beat detection took a considerable amount of work. I compiled all these findings and created a tool on the asset store I call The Audio Visualizer: https://www.assetstore.unity3d.com/en/#!/content/47866

Here are a few screenshots from the tool: 

Audio Visualizer - Spherical Waveform

AudioVizualizer - Pad Waveform

AudioVizualizer - Line Waveform

AudioVizualizer - Panel Waveform

AudioVizualizer - Rainbow

If you’re looking to add visuals to the audio in your game I would love to hear your feedback on the tool or ideas about how it could be improved.

I'll be showing this tool off and some of my other stuff at GDC this year! If you'll be at the event and want to connect than hit me up on social media!

www.facebook.com/dogeatdoggames

@DogEatDogGames