There and Back Again: The Circular Journey of Sound and Music in Video Games

From early arcade games all the way to modern PC and console games, sound has been an important element in the game maker’s arsenal. Initially, sound was used mostly as an added effect due to the limited technology for playing audio in video games, but in today’s more powerful console and PCs, video games often go beyond mere sound effects to incorporate full musical scores, which are essential for setting the desired mood of the game. Thus, as audio capabilities have proliferated over the last few decades, the importance of music to the success and popularity of many types of video games has also steadily increased to the point where background music rather than ambient sounds and effects may well be the key driver of emotion in a game. Now, with the advent of virtual reality (“VR”) and its focus on immersive play, the quality of the design of ambient sounds will once again move to the forefront, likely eclipsing music as the most critical audio component that can make or break the developer’s ability to elicit the desired atmosphere, mood, and emotions of the game. While still an important asset for developers, because of the immersive nature of VR, the successful use of music in future games will depend greatly on the placement of the music within the games and how well the music fits within the game’s environment.

History

In the beginning, there was nothing. Then, there was “the beep,” and “the beep” was good.

While today sound may seem an obvious and indispensable part of video games, the very first games had no sound at all. These games, which include Spacewar! (developed in 1962) and text adventures, were generally very simple (Brandom). They usually had little to no story and focused primarily on gameplay. Fairly soon after these games began to appear on systems such as the PDP-1 (a very expensive computer at the time), newly-formed and well-established companies alike began to develop arcade games. One of the most successful of these early games was also one of the first to make use of sound. That game was known as Pong (released on November 29, 1972). Pong made use of three different tones, quickly and indifferently decided upon by the principal engineer Al Alcorn, to represent the sound of the ball hitting the wall, hitting the paddle, and going past a paddle (meaning a player scored) (Caprani). His indifference towards which sounds were used in Pong is best summarized by his own words:

Alcorn’s attitude towards adding sound to Pong may seem strange now, but it accurately depicts the state of sound in video games at the time: it was simply not necessary. In fact, one of the first video games to feature background music was Space Invaders in 1978. This game “employ[ed] a four-note motive with fixed pitches, a fixed timbre, and fixed rhythms; all that change[d was] tempo, which [went] through a simple process of gradually increasing” (Neumeyer 330). However, while probably unanticipated by their designers, the sounds from these early games are now almost universally recognizable, such as the sound of Pac-Man eating pellets and fruit or the movement of the aliens in Space Invaders (Wall et al.).

Figure 1: A game of Pong displaying the score, paddles, ball, and center line (“Screenshot of PONG”)

After the North American video crash of 1983 and through the next two decades or so, Japanese companies like Nintendo and Sega became the largest home console makers. Both the improving technology and a greater level of control over third-party game developers resulted in an increase in the overall quality of video games during this period (“Nintendo Seal of Quality (Concept)”). Similarly, as audio technology improved, the quality of the music used in video games also significantly improved. Indeed, many of the soundtracks from this period, as well as their composers, are well-regarded even today (including, in some cases, by those not involved with the video game industry). For example, Koichi Sugiyama (Dragon Quest), Nobuo Uematsu (Final Fantasy), Koji Kondo (Super Mario Bros., The Legend of Zelda), and many others first began composing video game soundtracks in the early to mid 1980s. The music in these games, in contrast to earlier games, sought to either affect the emotions of players or provide a catchy melody that would entice players to keep playing. Towards the end of the Nintendo Famicom’s (essentially the NES) life, some video game soundtracks began to be published as albums (in original, remix, and live performance forms, among others). The albums often sold well in Japan, encouraging the growth of music composition in video games in addition to the live performance of video game music by orchestras such as the Tokyo Philharmonic Orchestra.

Figure 2: Music from Final Fantasy VIII being performed at the Los Angeles Dear Friends concert (Square Enix Rinoa Heartilly during the Dear Friends Concert)

With the advent of the Famicom’s successor in 1990, the Nintendo SNES, the technology for sound reproduction on home consoles had advanced to the point where creators of games such as Final Fantasy VI could experiment with (synthesized) vocal music and operatic music (“Aria di Mezzo Carattere”). The new musical frontiers emanating from an increase in experimentation encouraged composers to strike out and try new ideas, as exemplified by Yasunori Mitsuda’s Chrono Trigger soundtrack as well as Uematsu’s “One-Winged Angel” (from Final Fantasy VII), which discarded traditional notions of video game musicality to fit within the thematic elements of the game (specifically Sephiroth’s insanity),1 and “Dancing Mad” (from Final Fantasy VI), a boss theme that serves as the “last speech” of the game’s main villain Kefka, “outlining his rise to power, and lamenting his eventual defeat” (“Final Fantasy VI’s ‘Dancing Mad,’ a Critical Analysis”).

Probing deeper, the game Streets of Rage for the Sega Mega Drive (Genesis) was one of the first games to explore different genres in video game music. The game featured a soundtrack influenced by dance music, differentiating itself from other games of the time which usually took either an orchestral approach or sought to use and reuse synthesized melodies – although hopefully in ways that would not become annoying after significant periods of time (e.g. Super Mario Bros music) (Kondo). However, home consoles were not the only platforms upon which significant innovations were made to video game music. PC games in the 1990s such as Doom, with its metal-influenced soundtrack, and Quake, with its industrial soundtrack composed by Trent Reznor from Nine Inch Nails, helped establish a precedent for experimentation in genres for video game music.

Figure 3: Trent Rezor (Cultice)

Current Times

Today, and since the late 1990s, music has become one of the central elements of a video game. Due to its key role in game design, the video game and music industries have developed a mutually beneficial relationship. Many games feature music by famous musicians either specifically created for or licensed for the game. By creating and licensing music for video games, artists draw attention to themselves and their works, providing a boost in sales of their music and their name recognition. According to the music marketing agency Electric Artists, “40% of hard-core gamers bought the CD after hearing a song they liked in a video game” (qtd. in Collins 116). In turn, the overall quality of the music in video games is enhanced, sometimes attracting potential consumers who mainly choose to play the game because of the participating musicians.

Generally, the most popular video games today feature soundtracks composed by highly paid composers that sync well with the visual aspects of the games. In a sense, as games become more cinematic in their visual appeal, so does their music. Games such as Mass Effect, The Elder Scrolls: Skyrim, and Batman: Arkham City clearly demonstrate this trend. Further, as games based on movies are not uncommon, music by movie composers frequently appears in video game adaptations (for example, many Star Wars games feature pieces by John Williams) and, in some cases, movie composers are introduced to the video game industry in this way. Sometimes, television and movie composers decide to switch to video game composition as they feel it will be more interesting. Inon Zur, the composer for Dragon Age: Origins, Crysis, and many other games is one such composer. He switched to video games because he “was looking for something that would be more intriguing, more advanced, and basically a place that people really appreciate music more” (Zur).

Yet music is an essential component of modern video games for more than just the marketing impact and money a famous soundtrack can bring to a game. In fact, music has become important enough in video games that games with gameplay and graphics generally panned by critics and consumers alike can nevertheless enjoy fairly significant followings through the power of the soundtrack and story alone. Perhaps the best example of such a game would be NieR, a game released by Square Enix in 2010 for the Xbox 360 and PlayStation 3. NieR received lackluster reviews, but garnered widespread praise for its unique soundtrack (which features a vocalist singing in various languages after 1000 years of imagined linguistic drift) (Evans). The soundtrack actually received so much praise that it ended up on a variety of “The Greatest Video Game Soundtracks of All Time” lists (Agnello; Kain). While these lists are not necessarily definitive (given that such lists change each year and are inherently subjective), the fact that NieR’s soundtrack is consistently singled out is itself noteworthy. In fact, even today, years after its first release, the NieR soundtrack album still sells well (Seto).

Figure 4: The NieR soundtrack's success helped the game to gain a large cult following (Square Enix “Cover Art for NieR Gestalt & Replicant Original Soundtrack”).

Another example of the importance of music in modern games is the use of dissonance, minor keys, odd meters, whispering vocals, musical stabs, and reverb-drenched spaces, among others techniques, to convey a sense of dread and despair in horror games. While it is true that horror games could still be genuinely frightening without soundtracks, the tension built by the music contributes greatly to the anxiety of playing a horror game. In addition, the placement of music in horror games can make or break the game. After all, a jump scare simply is not as effective unless it is paired with a sudden, loud, and creepy chord. Similarly, successful game designers must also know the most appropriate times to use eerie silence, disconcerting noises, and slightly off-putting music to best frighten the player (Sweet 96).

Thus, the evolution of music in video games that began in the 1980s and truly exploded due to great experimentation in the 1990s has benefited and continued with the current generation of games. Adventure games and RPGs still make use of a wide variety of increasingly disparate types of music to influence player emotions, and the symbiotic relationship between the music industry and the video game industry is quite strong today.

Future and VR

As technology has improved, modern-day video games and their sounds have become far more complicated and intricate than video games from the 1990s and before. Music and ambient sound are now cornerstones of a modern video game, sometimes holding a position of importance to the success of a game that is equal to or greater than the quality of the graphics used, the story, or even the gameplay. Given that the importance of music to the success of a game has been increasing over the past couple of decades, one might presume that its importance will continue to increase in the future. However, I believe that this presumption is only partially correct. While it may be true that the importance of sound, and in particular ambient sound, will continue on its upward trajectory, whether or not music will continue to be an important factor of a game’s success is a very different question. The crux of this matter is VR. In VR, ambient sounds, such as the grass rustling in the wind and beneath your feet or the clanging of swords against shields, will be necessary to keep the player immersed in the game. In contrast, the introduction of random music while the player is running around fields or in caves would likely be immersion breaking.

Saying that music can easily be immersion breaking in VR might lead one to conclude that music should not be part of VR games. If so, does that mean that VR game developers will lose what has consistently proven to be one of the most useful tools in eliciting and conveying desired emotion? Fortunately, music can indeed be used in VR games. For example, one of the more obvious ways to make music fit in a VR game would be to make the music come from a diegetic source, or an “explainable in-game source, such as a radio, a telephone, or some other logical sound conveyance that exists in the virtual world” (Phillips). Some games, such as Fallout, already incorporate these types of elements as core parts of the game. In Fallout, the player can dial into radio stations using the “Pip-Boy,” a device worn on the arm of the player character. As the player travels farther and farther away from a given radio station, the signal from the station weakens until the point where he or she can no longer hear music from that station. Without the signals from these stations, all the player hears is ambient noise from the environment. The Fallout model of appropriate music placement and sources would translate easily to VR and provide a sense of realism to the game. Another solution, according to a variety of sources, would be to build up the music slowly over time so the player does not consciously notice, as “the time when music is most likely to draw [attention to itself] is the beginning and end” (Batchelor; Desiderio). This technique relies upon gradual changes drawing less attention to themselves than abrupt changes, allowing the player to acclimate to the music’s increasing presence.

Figure 5: The Pip-Boy from Fallout lets the player use the radio in-game (“Pip-Boy 3000 Mark IV from Fallout 4”).

Of course, because not every game in the near future will be designed for VR, music will continue to increase in importance for non-VR games as game designers experiment more and more with its capability to convey emotion. Games like Journey and Shadow of the Colossus will continue to appear on traditional platforms as not every game is suited for VR. For example, even games that are not often thought of as horror games such as Batman: Arkham VR become frightening when the player is placed in the position of the main character rather than being a passive observer in front of the television screen (Kohler). Given that some people already choose not to play horror games with the more impersonal devices we tend to use today, adding VR (and, in turn, a degree of horror) to certain types of games may actually cause the market for these games to shrink.2

It is interesting to note that the relative importance of sound and music to each other appears to be moving in a cycle. With early video games, sound played a greater role in the success of a game than music, as, for the most part, music did not exist past extremely simple melodies. Music gradually grew to overshadow the importance of ambient sound in games, but with the future of gaming seeming to be heading towards VR, ambient sound will likely reclaim its place as the most necessary element between sound and music in video games.3

Endnotes

1While it may be true that “One-Winged Angel” was not the first song to favor thematic elements over video game musicality, the extent to which it did so (along with its excellent composition) made it a landmark track among video game soundtracks (Uematsu).

2Expanding upon this thought, if one were to make a traditional RPG with dungeons, big boss monsters, and enemies everywhere, children would, in all likelihood, be too scared to finish the game (and the game could receive a higher rating from the rating boards, so children may not be allowed by their parents to play the game in the first place).

3Interestingly, this cycle is very similar to the path sound and music have followed in films. Movies progressed from silence (as in silent films), to early sound effects, to sweeping epics with massive musical scores, and now again towards fully-immersive experiences, focused heavily on ambient sound effects.

Works Cited

Agnello, Anthony John. "The 25 Greatest Video Game Soundtracks of All Time." GamesRadar+. Future, 29 June 2016. Web. 29 Nov. 2016.

Alcorn, Al. "Al Alcorn Interview." Interview by Cam Shea. IGN. Ziff Davis, 10 Mar. 2008. Web. 29 Nov. 2016.

"Aria di Mezzo Carattere." Aria di Mezzo Carattere - Videogame Music Preservation Foundation Wiki. N.p., 13 Nov. 2015. Web. 29 Nov. 2016.

Batchelor, James. "PlayStation VR Team's Five Tips for Using Music in Virtual Reality." Develop. NewBay, 15 Sept. 2015. Web. 29 Nov. 2016.

Brandom, Russell. "'Spacewar!' The Story of the World's First Digital Video Game." The Verge. Vox Media, 04 Feb. 2013. Web. 28 Nov. 2016.

Caprani, Ole. "The PONG Game." The Video Game Pong. Aarhus University, 27 Sept. 2014. Web. 29 Nov. 2016.

Collins, Karen. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge: MIT P, 2008. Ebrary [ebrary]. Web. 29 Nov. 2016.

Cultice, Joseph. Photograph of Trent Reznor. 1994. N.p.

Desiderio, Erik. "Composing Music for VR Games: 'Adventure Time' Case Study." GDC 2016. Moscone Conference Center, San Francisco. 14-18 Mar. 2016. UBM Tech. Web. 29 Nov. 2016.

Evans, Emi. "Deep into NieR: Interview With Vocalist and Lyricist Emi Evans."

Interview by Jayson Napolitano. Original Sound Version. Original Sound Version, 4 May 2010. Web. 29 Nov. 2016.

"Final Fantasy VI's 'Dancing Mad', a Critical Analysis." Destructoid. ModernMethod, 15 Dec. 2015. Web. 29 Nov. 2016.

Kain, Erik. "The Top 12 Video Game Soundtracks Of All Time." Forbes. Forbes, 8 Sept. 2012. Web. 29 Nov. 2016.

Kohler, Chris. "Batman: Arkham VR Is Smart, Scary, and (Very) Short." Wired. Conde Nast, 5 Oct. 2016. Web. 29 Nov. 2016.

Kondo, Koji. "Interview with Koji Kondo." Interview. Square Enix Music Online. EGM Media, n.d. Web. 29 Nov. 2016.

Neumeyer, David. The Oxford Handbook of Film Music Studies. Oxford: Oxford UP, 2015.

Google Books. Google, 12 Mar. 2015. Web. 29 Nov. 2016.

"Nintendo Seal of Quality (Concept)." Giant Bomb. CBS Interactive, 24 Apr. 2013. Web. 29 Nov. 2016.

Phillips, Winifred. "The Virtual Reality Game Music Composer." Blog post. Gamasutra. UBM, 30 Apr. 2015. Web. 29 Nov. 2016.

Pip-Boy 3000 Mark IV from Fallout 4. Digital image. GND-Tech. GND-Tech, 20 Dec. 2015. Web. 29 Nov. 2016.

Screenshot of PONG. Digital image. Wikimedia Commons. Wikimedia, 21 May 2006. Web. 29 Nov. 2016.

Seto, Dan. "The Full Story Behind Nier: Automata on PS4." Blog post. PlayStation.Blog.

Sony Interactive Entertainment, 29 Oct. 2015. Web. 29 Nov. 2016.

Square Enix. Cover Art for NieR Gestalt & Replicant Original Soundtrack. Digital image. Wikipedia. Wikimedia, 11 July 2011. Web. 29 Nov. 2016.

Square Enix. Rinoa Heartilly during the Dear Friends Concert. 2004. Los Angeles. Wikipedia. Web. 29 Nov. 2016.

Sweet, Michael. Writing Interactive Music for Video Games: A Composer's Guide. N.p.: Pearson Education, 2014. Google Books. Google, 11 Aug. 2014. Web. 29 Nov. 2016.

Uematsu, Nobuo. "A Day in the Life of Nobuo Uematsu." Interview by James Mielke. 1UP.com. IGN Entertainment, 15 Feb. 2008. Web. 29 Nov. 2016.

Wall, Jack, Tommy Tallarico, Joshua Bell, Kay Stern, Emil Miland, and Nina Flyer. "The Evolution of Video Game Music." Interview by Andrea Seabrook. NPR. NPR, 13 Apr. 2008. Web. 29 Nov. 2016.

Zur, Inon. "Inon Zur Interview." Interview by Spence D. IGN. Ziff Davis, 30 June 2004. Web. 29 Nov. 2016.