The State of Music Games

Ocarina Hero

In the meantime, on Nov 6 2008, another relatively unknown company called Smule released a quirky little iPhone app called Ocarina. Its premise was simple: blow into the phone’s microphone to make music, listen to Ocarina melodies from other people around the world and share your melodies with them. The app quickly became a hit with downloads in the millions, the first of Smule’s string of ‘interactive social music game experience’ successes. Smule followed up Ocarina with hit music apps / games based on the trombone, piano, fiddle, vocals / karaoke and sampling. One of their latest music games, Magic Guitar, shot to the top of the charts at launch in December 2011. This is not the dead genre Activision and Viacom walked away from.

So what happened here? Was Harmonix’s success with Guitar Hero and Rock Band just a fad, and Smule’s success just a temporary extension of the fad – a fad encore! – that will fade away soon? As someone who left a cushy corporate job to form his own development studio building music apps and games, my completely biased opinion is “no.” Music games are not a fad, because music is not a fad, and wanting to be a rock star is not a fad. Harmonix stumbled and was unable to innovate effectively beyond a certain point. Smule’s apps have taken over from Harmonix in moving the genre forward, and there is still tons of potential to innovate and expand the genre. To explore this thought, let’s think about what games do for us, and how they have evolved over the last few decades.

Not a Fad

First, the fad thing. By and large, video games serve as fantasy fulfillment engines: become a hero by defeating the enemy and saving the princess / country / galaxy. Feel great about yourself, rinse and repeat. The fantasies that they tap into are timeless, much older than the game industry itself. The best-selling video game franchises of any generation, from Mario to Madden to Call of Duty, have always directly targeted these classic fantasies. This is not a coincidence. Play-acting in backyards has turned into play-acting in virtual worlds and online communities.

Being a rock star is another classic fantasy, and a lot more relatable than being a wizard. Legions of air guitarists and shower singers across every generation can attest to the allure of rocking out live at arenas across the globe for millions of adoring fans. This fantasy is timeless, so the games genre which directly taps into this fantasy cannot be a fad. Harmonix knew this, and crafted every aspect of the Guitar Hero (and later Rock Band) experience around this fantasy. Where they have failed so far is where Smule is succeeding, in evolving the genre towards its natural destiny.

Evolution and Destiny

Modern video games, such as Halo and Burnout, are significantly evolved from their spiritual ancestors, Space Invaders and Outrun. Beyond the obvious improvements in audiovisual presentation, players of modern games are generally given a lot more control and freedom to engage in the game world (and simultaneously a lot more guidance regarding how best to engage). Where Space Invaders let you shoot in one direction and move along one dimension to hide behind a few fixed-position shields, Halo allows you to run and jump (and now jetpack) free-form around a fully-realized 3D space, shooting a wide arsenal of weapon types in any direction you choose. Where Outrun would only let you control gas/brake and strafe on the track, Burnout lets you navigate the city in any way you want, with a deep physics engine simulating handling and damage. Similar comparisons can be made for many other genres of games – the bottom line is that this increased level of control immerses players much deeper in the game. Each triumph or setback feels more personal and visceral because the moment-to-moment wide-ranging choices made by the player directly impact how events play out.

Guitar Hero and Rock Band may sport modern graphics and sound, but their controls are as primitive as Space Invaders. All of the music is pre-authored, so as a player I can’t control the note sequences, or the timing of the notes, or even the loudness of my strums. There are a few minor exceptions to this, but for the most part these games are the musical equivalent of Dragon’s Lair (which had a similar boom-and-bust cycle): follow a series of timed button presses to get through the game. How immersive can a music game be if the player never plays any music?

As it turns out, it can work well, for a while. These games became a phenomenon primarily because Harmonix DID nail the rock star fantasy via great presentation, narration and game design. Guitar Hero boasted an engaging rags-to-riches campaign storyline, superbly-curated and sequenced song list, pitch-perfect visual tone in presenting your virtual rocker’s ascent to fame, and finely-crafted note charts for each song that truly made you feel like you were playing that song, and playing it well. With Rock Band, Harmonix enabled multiple people to play together on different instruments, exponentially increasing the fun factor and making it a must-have game for parties. With Rock Band Network, Harmonix allowed indie musicians to submit their own songs to the game and make them available to Rock Band players around the world, significantly increasing the number of songs available to players. And with Rock Band 3′s Pro Mode, Harmonix took the bold step of expanding their game into the realm of music instruction, teaching advanced players how to play Rock Band songs on a real guitar, drum kit or keyboard.

For me, however, eventually the novelty wore off and I realized that rather than me playing a game, the game was playing me, micro-managing me, telling me every single button to press and exactly when to press it. Even in Pro Mode, playing a real guitar or keyboard into the game, I had no way to influence the pre-generated sounds coming out of the speaker, no way to improvise and add my own flourishes to the sound. I started wanting something more evolved, with more free-form control in a larger possibility space. I imagine millions felt the same way, and eventually stopped playing.

The New Hotness

Perhaps those are the same millions who, like myself, downloaded Ocarina and loved it. Or perhaps they were different millions. Either way, Ocarina sold well. The presentation is spartan – four blue circles on a black background. The audio is nice (a sweet, flute-like sound coming out of your tiny phone speaker) but not Guitar Hero-level awesome (a full rock band track with drums, bass, guitars and vocals blasting out of your home theater system). Luckily, it released on a platform where the bar for presentation was very low at the time, so the only thing that mattered was that it felt magical to play.

The controls are easy to use – blow into the the phone to play, blow harder or softer to control volume, change the notes with your fingers, and tilt the phone to add vibrato. What you hear is what you play, not what someone else pre-recorded for you. It might not always sound like a chart-topper, sometimes it might sound like a hot mess, but it’s YOUR hot mess. Most importantly, you can play it with your eyes closed and imagine yourself performing in a stadium for thousands of imaginary fans. The illusion is complete, the fantasy sold, no fancy 3D graphics needed.

As it turns out, you really ARE playing to an audience of thousands, and each of them is playing to you. On the “world” screen you see the earth as a globe, with thousands of tiny shimmering lights on the surface representing other Ocarina players just like you, doodling away on their new instrument. One at a time, the app tunes in to one of these players and lets you hear their performance. Heart it if you like it or tap ‘next’ to listen to something else. All of your performances are automatically and anonymously shared with this community too (you can opt out, but why would you?). Millions of people playing for each other, entertaining each other, inspiring each other. Real people, playing real music on a real instrument, giving each other real feedback. Not the fake NPCs and fake applause of the console games – after all, no modern AI is sophisticated enough to give meaningful feedback on a freestyle musical performance. Ocarina presents the beginnings of a significantly evolved way of fulfilling the rock star fantasy, and the direction these games need to go: a massively-multiplayer online game with music as the core mechanic.

Smule innovated further with subsequent apps, in significant and impactful ways. The “Play a Song” mode guides you in realtime to play a full song on a Smule instrument via an interface similar to Guitar Hero, surreptitiously teaching you how to play that real instrument via a fun game. User Generated Content (UGC) options allow advanced users to create and submit their own songs for others to learn and improve from. On the “World Stage” you can perform live for a panel of judges who provide realtime feedback, American Idol-style. “Join a Song” enables multiple users to contribute performances to the same track – the 3500-person rendition of “Lean On Me” in the Glee app is a must-hear. Officially-licensed backing tracks and sophisticated audio tech come together to make performances sound even better. Richer and more “full-sized” iPad instruments like Magic Piano and Magic Fiddle significantly expand the range of music that can be played on a Smule app, for those willing to learn.

The Future?

In theory Smule has assembled a fantastic mix of innovative features that enable non-musicians to easily learn to play a range of real (electronic, iOS-based) instruments and go as deep as they want into the rock star fantasy. However there are a few seams in the big picture. The ocarina, trombone and fiddle have limited long-term mainstream appeal as instruments, as they don’t really tap into the rock star fantasy given their place in popular culture. Unfortunately, Smule’s guitar and piano apps are not exactly easy-to-play real instruments. Magic Guitar has no freestyle mode and is thus purely a game, not an instrument. Magic Piano’s freestyle mode is very difficult to play and completely different from its “Play a Song” game mode, thus precluding any skill-building benefits of the latter towards the former.

On the other hand, over the years Harmonix has created very easy-to-play controllers inspired by guitars, drums and keyboards – the instruments most people care most about – and have nailed the narrative and presentation for selling the rock star fantasy. What we need now is a convergence of these two seemingly-opposed but ultimately complementary schools of thought. How awesome would it be if we had a visually and narratively-immersive game like Rock Band, with a simple and intuitive way to control the music as with Smule’s apps? The game could teach me how to play songs, introduce advanced techniques over time, and let me inject my own style into the sound. At some point I could just plug my controller into an amp or speaker and play that instrument for real!

There is one game that has attempted to bridge this gap somewhat. Ubisoft’s Rocksmith is essentially Guitar Hero played with a real guitar in which you hear yourself play, rather than a pre-authored guitar track. It teaches you to play songs via a rich, elaborate interface, and introduces advanced techniques using a very slick adaptive difficulty mechanism. However, Rocksmith has some major limitations, as well-covered by Kyle Carpenter’s excellent write-up. Ultimately, its “controller” effectively has 96 buttons, which is way too complex a proposition for a mass-market audience. Guitar Hero had 5 buttons, Ocarina had 4. This is the ballpark level of complexity needed.

Skeptics will say that there is no way to simplify the full breadth and depth of things you can do on a real guitar down to 4 or 5 buttons. And they’re absolutely right. But as Smule has proven many times, good design can solve this problem. Let’s bring together Smule’s sensibilities of simple instrument design with Harmonix’s instincts for tapping into the rock star fantasy, and let the players of that game jam together and collaborate, and perform live to a global virtual audience of real people providing real feedback. Easier said than done, but it can’t be impossible. The only question is who will do it – Smule, Harmonix, or someone else?

Further Reading: “Rocksmith and the Limits of Learning” (Medium Difficulty)

“Please Try Again” (Kill Screen)