Originally posted on the Team Junkfish blog.

This blog is based on an event that happened as part of Abertay’s Game Lab, loosely based on the topic of “story”, and how people from different disciplines tackled it in games. I was asked to provide an audio perspective, and given that I’m usually better at explaining things in written form than verbally I decided that I’d expand and clarify a few points I spoke about and why I think they’re important. There may be a few spoilers here and there, so heads up!

What is “Story” anyway?

The word “story” itself is a bit of a nebulous term. Most people would associate it with some form of narrative, such as the Hero’s Journey, that follows one main protagonist or group of people through their various trials and quests to some sort of resolution. But is that it? Is that everything that’s being told? Does the entire game world merely exist around the player character(s), with everything only being relevant them? What about everything else: the details, the history, even the context of the place and situations that you may be in. Going from A to B through C becomes much more engaging and involving if there are different forces, details and subtexts at play instead of “because that’s where you need to go”. 

Basically, how do you “build the world“?

Worldbuilding is something I try to push a lot when it comes to audio, and there are a few different elements that I consider when doing so. There’s the obvious, how things sound, to slightly more subtle stuff like communication and providing information, and not necessarily though speech and voice acting either.  Sound is one of those things that people will immediately proclaim “This is super important we must have it” but not really have a solid explanation at to why it is, often jumping on the cheapest/closest/first thing they get after their game is finished as they needed some random noises to finish up.

A game must “sound right”.  This doesn’t mean that it should be a one-for-one replication of real life sounds that are dropped into the game using real life simulations  to make sure that the game sounds SuperReal™. Realism isn’t always “right”, appropriate or even possible, as any designer will tell you. From a sound perspective: real guns sound nothing like guns from films and especially games, and I have some bad news for you if you’re into laser noises. But how do you determine what is right for your game? What sort of things do you need to consider when making those choices too?

Let’s start with how music is used first. If, again, we were going for “realism” then your player would probably need a set of headphones to hear all the music that’s going on. Alfred Hitchcock allegedly complained about the use of music for his film “Lifeboat” saying ‘Where, then, did the orchestra come from?’ The composer, Hugo Friedhofer, responded with ‘The same place the camera came from, Mr. Hitchcock.’.   This use of music that exists outside of the film/game world is called “non-diegetic”, and is not heard by any in game character. In terms of story, you can use this to provide context to a situation, like a lot of “Final Fantasy” games do. 

Would he have been even sadder if he could hear the orchestra in the background?

Similarly, it can also provide context to a place or person in the game world. Leitmotifs are nothing new, generally popularised by Wagner in the 19th Century, but chances are when you hear this theme (and variants) you know what sort of area you’re in, what gameplay elements may be involved and what types enemies lie ahead. Similarly if you hear this one then you know who is about to appear too.  Even main themes can say a lot about what is going on. “Papers, Please”‘s stark, military style anthem plays on old Soviet/Eastern Bloc tropes to set the scene of the game rather effectively before any explanation is given.

But games have the advantage of being non-linear, meaning that music and sound can provide additional context and information without any additional visual cues in a manner that films and cinematic scenes can’t. This can be a basic thing, like the music speeding up in Mario games once the timer hits a certain point. This allows the player to focus on what they are doing rather than a timer off in a corner away from the action. Here’s a great breakdown on the music systems in Capcom’s fighting games by Axel Speller that’s worth reading. It details how the music changes depending on a player’s health, the current stage, themes and even the effects of restarting the music for each round as opposed to letting it continue on.

Getting back on point, music is also key in building up the world. “NieR” is an excellent example of this, not just due to its sublime score, but the fact the use of vocals throughout is  related to an in-game character who has been keeping watch over the player characters throughout the game. While still non-diegetic, there is an over arching unity with the score, the world and events in the story that bind them together. Similarly “Grim Fandango”, with its Dia de Muertos theme combined with film-noir aspects, would just not be the same without the latin-jazz and big band score than runs through it, accenting and playing off of the visual and narrative themes. Appropriate genre choice is something that can bolster a game’s feel and hold on a player. Imagine playing “Far Cry: Blood Dragon” or “Hotline Miami“ without their 80’s influenced, synth heavy soundtracks, instead replaced with the standard orchestral or modern electronic OSTs  that ignore the source material and influences that these games are built upon?

But what if your character actually has headphones on or is listening to a radio? We can use “diegetic” music in these instances that give music a source within the game world. Instead of the background music being from nothing, the placement of an emitter like a radio gives it a reason to be there within the context of the game world. 

This is an advantage in world building too, as it allows you to give a game space its own musical culture. In the aforementioned “NieR“‘s first town you can enter a tavern with a musician, who will play songs about the world’s history in their own language, as well as composites and variations on other, real life languages. This implies that this world doesn’t exist around you, the player, but has a past and present that has and is happening all the same. On a side note, talking to her means that the singing stops as well, as it would in real life! “The Legend of Zelda: A Link Between Worlds” provides some fanservice in a similar manner, with the bards in the Milk Bar playing variations on classic themes from the series. 

After being introduced to Professor K at the start of the game, “Jet Set Radio” takes the headphones comment and runs with it with its funk/hip hop/pop soundtrack seemingly coming from Prof. K’s pirate radio station straight into the player character’s headset. Given the game’s focus on self expression and fight against authority, it’s these little touches that further solidify the player character’s place and ambitions in the city of Tokyo-to, and the culture surrounding the graffiti art and skating.

Beat from “Jet Set Radio”. Note the headphones!

On the topic of radio stations there’s also “Grand Theft Auto”, which goes all out in its use of diegetic music in cars. On the world building standpoint, the multiple stations and personalities imply that it’s not just your character in the city that is able to do anything, any other person in that world could be listening to one of the radio stations available. 

But the beauty of GTA’s system is how it’s implemented. The radio stations are played in cars, but unlike the above examples they are played in a space, as opposed to merely being in the background. This means that the sound you hear changes depending on what environment you are in, with it being more muffled outside of cars as one example, but in GTAV the radio signals themselves can drop out depending on where you are in the world. There is also some character interaction with radio stations in the game too. Should you be driving with Trevor in the car he’ll change stations if he doesn’t like what he hears. There is a lot more on how the game’s audio works in this GDC talk, including their music system, that is interesting viewing. 

These are only a few examples of what to consider and how to use music beyond merely window dressing or “because it’s needed”.  I’ve stuck a few links below that cover various aspects of music in games as well as some of the topics I discussed above. So hopefully it’ll be of some help!

Cheers,

Jaime

—

Further Reading

Karen Collins – Game Sound: A fairly detailed look at the history of game audio development

Michel Chion- Audio-Vision: One of the most important books on film audio theory, which is easily carried over to games.

Winifred Phillips – A Composer’s Guide to Game Music: A great starting point for composers looking for info on game music and what’s involved.

Dan Bruno – Dan Bruno is a developer over at Harmonix. He ran a blog called Cruise Elroy which went into great detail about the music theory aspects of a fair number of games. He only has his “Ocarina of Time” series on his new blog, but you can find some of the others that have been archived.

Alastair MacGregor – The Sound of Grand Theft Auto V (GDC Vault)

The Music of NieR (Wikipedia)

Alex Sheller – Fighting Game Interactive Music Study

The Hero’s Journey/The Monomyth (Wikipedia)