Author: by Caleb Bridge

"Immersion" has become cliché. It's often just another buzzword when talking about how great a game is, but it's all too infrequent that those discussing games will actually break down the finer details of what that immersion entails.

Like all pieces of the puzzle that is game design, audio must work in concert with graphics and game mechanics to help immerse the player into gameplay experiences of all shapes and sizes through its ability to convey vast amounts of the detail to the player, often without their knowing.

J White, Martin Stig Andersen, and Thom Kellar, of Visceral Games, Playdead, and Freshtone Games respectively, are three sound designers who have ample experience in creating such audio experiences.

It's Mental

"It can be easy for the media to reveal itself if you've got a sound loop or something that becomes annoying. As soon as the game reveals the mechanics, the media and the machinery behind these things is ruined and the player is thrown out of the experience," said Martin Stig Andersen, sound designer and composer on Limbo.

Limbo was a critical success and fan favorite in 2010, putting many end-of-year awards under its belt.

It was lauded for its striking visual aesthetic, but part of its success came from its ability to make the player feel a sense of isolation and foreboding that is largely the result of top quality sound design.

Andersen and Limbo's director, Arnt Jensen, teamed up after Andersen saw the game's initial trailer and felt his unique area of music (electroacoustic music, a non-commercial, almost entirely funding-based form) would add to the experience.

"I watched the trailer and I was really captivated by [the boy's] expressions," said Andersen. "It reminded me of the aesthetics of light and sound; you have something recognizable and realistic, but at the same time it's abstract.

"It's the same as what I love about how we use sound. We have all these slight references that focus on ambiguity, so it's more about what the listener imagines, rather than what I want to tell them."

Many can think back to sections of Limbo where they were struck by a certain feeling or sense for what the boy was doing in that place. But one thing Limbo never allowed the player to do was fully understand what was happening. Andersen achieved this by intentionally distorting the sounds of objects in an attempt to make the player think more about what's going on and how they're meant to react. This gave it a level of ambiguity that allows the player to "be there and make their own interpretation."

"The more identity the sounds had, the more I would distort them," Andersen said. "So I wouldn't include sounds that gave too strong associations. If we added something that had a strong identity like a voice or an animal, then it would almost destroy the atmosphere. So with that style, Limbo offered an audio and visual atmosphere that can really get into the player's mind, and make them feel scared, worried or on edge."

Limbo toys with horror themes throughout, but it is in that genre that exacting sound design is essential in eliciting player emotion and getting inside the player's head.

One series praised for its ability to do this is Dead Space. J White -- a LucasArts veteran who served as sound design lead at Visceral Games on Dead Space 2 -- took great pride in the team's ability to do just that.

"During the course of production, there were meetings I would have with people -- they'd come in and have never seen [the section] before. We'd be playing, and they would literally jump in their chair. It genuinely frightened people. It might be kind of mean on my part, but I took a lot of pleasure in that," said White, when discussing the effects of the sound in Dead Space 2.

Much effort was put into thinking about how they could use the game's audio elements to enter the player's mind, sometimes without them even noticing it.

"A fundamental thing that people cannot help but respond to is the sound of the human voice and, even more specifically, the sound of human suffering. It's just an unavoidable, deep reaction that people have. So one of the elements that we'll use as part of our soundscapes are the sounds of people in misery. It may be deeply buried, but the human ear and human mind are so attuned to human vocalizations that they'll respond to it even if it's just a sub-audible aspect of the overall sound design."

This sentiment applies to smaller games -- even iOS games -- in the same way it does to triple-A releases. One Single Life on iOS, which literally gives the player's character a single life to jump from a series of rooftops to another, is one such example. Thom Kellar, Freshtone Games' sound designer, wanted to give players the sense that they were actually on that roof, preparing to jump.

"I wanted to use a lot of sounds that would make you feel a bit on edge, so there was wind and a lot of noises you'd hear on a rooftop like birds fluttering and planes flying overhead," he said. "But it kept coming back to 'What did that make me feel?' and it might have been the best or coolest or most wonderful sound, but if it's not contributing to the emotion or atmosphere of the game, unfortunately it had to go into the basket."

Manipulation

"Manipulation" can be a dirty word, and is generally used to call out poor attempts by developers to force emotions or reactions from the player. But the reality is, that audio designers try to use their craft to manipulate players in new and different ways as an enhancement.

Many games will use a rousing score to heighten the emotional state of a moment, and while music certainly has its place, it doesn't have to be the crutch used for affecting sound design games.

"We experimented early on with putting dramatic music over the top, but we felt that emotionally, it was a lot more effective to only have that big score happening towards the end when players were getting to the very edge, and they'd established themselves in the world already," said Kellar -- despite one of his initial roles being to write music for the game.

As a result, One Single Life's efficient and selective use of music gives it an extra sense of tension. Likewise, Andersen went for a similar feel with Limbo.

"[Jensen and I] rarely like music as an instrument to manipulate the emotions of the player, or manipulate anything really. We both feel that everything should be open to interpretation, and people should be allowed to project their own feelings and emotions into the experience," he said. "When you allow for that space, and at the same time create something that's captivating and immerses the player, it lets them let go of those feelings and emotions. So if they're scared it will probably make them more scared when there's no music to take them by the hand and tell them how to feel."

Of course a game like Limbo is aesthetically such that this is possible. Conversely, Uncharted or Mass Effect would be less likely to convey emotional and energetic peaks without the use of music, as it's such an integral part of the style of the cinematic experiences they're trying to create.

Of course, that doesn't mean that there's no music at all in Limbo, it's just more abstract, like Andersen's own music. "It's the personal interpretation of this boy and his journey through limbo, and instead of playing the action and emphasizing what's already there, we're trying to add another dimension... For me, it's really melancholy that this boy has been subjected to all this violence. It's just the idea that he's been habituated to it, and there's a kind of forgiveness in the music," he said. But it's also used to help juxtapose actions and emotions, such as "when you come across the Gatling guns for the first time, there's this divine music in the background, but if you add something like action music, it becomes so one-dimensional."

Dead Space 2 featured a more traditional mix of music and effects. Throughout, the apparition of Isaac Clarke's ex-girlfriend, Nicole, haunts him. In the later chapters, that relationship changes and after they have a reconciliation, the audio relating to her presence transitions from being abrupt and uncomfortable as the player is unaware of what's coming, to one which indicates to the player that her presence is actually a helping one.

According to White, one of the ways this was portrayed was by incorporating sound effects that became characteristic of her apparitions, and using them differently in 'ambient one-shots' -- effects that elicit thoughts of a character or event - that would begin to play the closer you came to an encounter with her.

The mixture of music and effects ends up being an orchestrated experience. White gives the example of a room where the player has fought of a phalanx of Necromorphs, who were chasing off government forces, which was followed by another apparition of Nicole.

"We have these competing things we want to pay off on," he said. "We've got these ambient sounds devoted to giving you the sense that you're following this massive battle, but at the same time, you've got this helping presence that's Nicole coming in. It's really a matter of choreography to get all those kinds of elements to play together, where you've got this sense of the battle, the desertion, the quietness and the girlfriend."

Sound design can also manipulate the player's mind to give them a sense of place, for which game audio helps set the foundation of the game experience. Taking care with such base-level environmental effects is part of what can help make a game great.

"We had loud footsteps to tell the audience that the environment is really silent," said Andersen. "Of course you couldn't have loud footsteps through the whole game. Their level is very important, so we added different parameters like how long [they boy] has been running, and if he's been going for a certain amount of time it starts to change. It needs constant variation, otherwise you'd go crazy hearing those footsteps all the time."

Likewise, such meticulous care with environmental sound design can help give unique gameplay experiences. According to White, Dead Space 2 used its ambient sound to "foreshadow what's coming next. If the player backtracks to the same place after they've cleared the room or beat the boss, it sounds different. In the most flowery terms, the ambient sound in Dead Space 2 is almost a character in itself. It's such an intrinsic part of the player's experience, and the psychological experience of playing a game."

Immersion

"A good game mechanic will naturally immerse players. Pong has a great game mechanic with boops and beeps, and people are drawn into that because it's fun," said White. "These days, as audio designers, we still score that core mechanic in a satisfying way, but we have a lot more subtlety available to us."

White recognizes that audio is an important, additive element to the overall experience, and part of the role of sound designers is to help enhance those game mechanics and environments in order to immerse the player.

Andersen, who has worked on films in addition to games, gives some perspective to game audio's immersive nature.

"When I'm playing games it's so different from films, because there you only hear sounds when it's important to the viewer. Often in games, you can go up and down or around a sound, you approach it and you pass it, but when you pass it you still hear it as you progress."

Ultimately, for some sound designers, walking the tightrope of making or breaking immersion is a tough one, says Andersen. "If you create something that's engaging but at the same time keeps the illusion alive then you've got something that's very strong. But it's so difficult to do."