informa
2 MIN READ
Blogs

Music That Leaves a Mark

Exploring how we can make our audio more engaging by accompanying it with visual effects.

Audio can be used to elicit emotion from our players, draw attention to something, emphasize an event, or set the pace for gameplay. Music is what draws us into the game world, it sells us on the fantasy of the world we’re entering when we pick up that controller.

In this article I want to explore how we can squeeze some extra juice out of those audio tracks. How we make our music and sound effects even more engaging.

I’ll never forget the first time I played Devil May Cry 5 where Dante enters Limbo for the first time. All of the visuals in the game shift alongside a heavy metal track. The entire environment bumped along with the music as if it was alive. Here’s a video of the scene if you haven’t seen it. The shift to limbo happens around 1:30, gameplay starts at 3:00 (warning, mature content) https://www.youtube.com/watch?v=LRmhjJTXdp4

That was the first time I realized how big of an impact audio can make when it’s seamlessly engrained in the environment. It made me think back to other games I’d played where visuals accompanying the music. Games like Audio Surf: https://www.youtube.com/watch?v=Js-lAXTixV4 and Luminess: https://www.youtube.com/watch?v=yGCrEmTo6h0. I’m sure you can think of others.  The point is, adding visuals to accompany your soundtrack can really leave an impact on your players. Audio Surf
AudioSurf

Luminess

Luminess

I’m a Unity developer myself, and I found they have a couple methods that are really nice for working with audio:

AudioClip.GetData() http://docs.unity3d.com/ScriptReference/AudioClip.GetData.html

AudioSource.GetSpectrumData()

http://docs.unity3d.com/ScriptReference/AudioSource.GetSpectrumData.html

Basically they allow you to sample the audio every frame, what you do with that data is up to you. I encourage you to play with them and see what you can come up with. I spent the last few months exploring different ways to use this data. I realized you could scale objects to music, change their position, color, or material, adjust lighting, or cue any custom event based on the beat. However beat detection took a considerable amount of work. I compiled all these findings and created a tool on the asset store I call The Audio Visualizer: https://www.assetstore.unity3d.com/en/#!/content/47866

Here are a few screenshots from the tool: 

Audio Visualizer - Spherical Waveform

AudioVizualizer - Pad Waveform

AudioVizualizer - Line Waveform

AudioVizualizer - Panel Waveform

AudioVizualizer - Rainbow

If you’re looking to add visuals to the audio in your game I would love to hear your feedback on the tool or ideas about how it could be improved.

I'll be showing this tool off and some of my other stuff at GDC this year! If you'll be at the event and want to connect than hit me up on social media!

www.facebook.com/dogeatdoggames

@DogEatDogGames

 

 

 

Latest Jobs

Xbox Game Studios

Redmond, Washington
10.5.22
Technical Lighting Artist

Innogames

Hamburg, Germany
10.5.22
Game Designer - Elvenar

Six Foot

Houston, TX
10.3.22
Six Foot Director, Player Relations

Hometopia Inc.

Remote
10.7.22
Lead Engineer
More Jobs   

CONNECT WITH US

Explore the
Subscribe to
Follow us

Game Developer Job Board

Game Developer Newsletter

@gamedevdotcom

Explore the

Game Developer Job Board

Browse open positions across the game industry or recruit new talent for your studio

Browse
Subscribe to

Game Developer Newsletter

Get daily Game Developer top stories every morning straight into your inbox

Subscribe
Follow us

@gamedevdotcom

Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more