Music is a critical aspect of game design players often overlook, and at XRDC in San Francisco today Owlchemy’s Daniel Perry shared some tips on how to use it well in virtual reality.

Drawing on his experiences designing and implementing soundscapes for VR experiences like Job Simulator and Rick and Morty: Virtual Rick-ality, Perry shared some advice for game designers on how to make the most of music in VR.

“I don’t want to talk to the audio people; I want to talk to the designers,” said Perry. “Just because these are things that need to be considered early on, and prototyped on, because they can make a significant difference in your experience.”

First off, Perry recommends you avoid non-diegetic music in VR, since it’s so well-suited to the medium. When possible, he says having diegetic music -- that is, music which appears to be emanating from specific objects in the game -- works best in VR game design.

Prioritize diegetic over non-diegetic

“We focus on diegetic music as our main thing, but that being said, we never take things for granted,” he said. “We have explored non-diegetic...and we do use both, so making a system that can accommodate both is very helpful.”

That said, he recommended that while devs should feel free to use mono, stereo, or surround sounds at their convenience, mono works well in VR -- and filling your world with diegetic, mono tracks affords players lots of opportunities to experience the sounds in different ways (by positioning themselves behind an in-game speaker or in-game walls, for example), which makes the VR experience feel more immersive.

“Meaning even if I have a stereo file, I basically want to separate that into two monos, and place them in the VR environment in a way that makes sense to my experience,” said Perry. “And since VR is a unique experience, we can definitely use weird setups...I can create as many mono audio sources as I want, and place them wherever I want.”

By way of example he provided a map of a Vacation Simulator level with the position of speakers marked out, illustrating how placing speakers in the virtual world made it feel more real and believable.

“Because we subscribe to the diegetic experience, we definitely wanted to have the speakers located somewhere for music to emanate from, but it has to be a place where the player can see visually where the sound is coming from, so the spatialization isn’t wasted,” said Perry.

Use the unique player data VR provides

He also encouraged game makers not to overlook all the extra data VR provides on where players’ heads and bodies are at. You can take advantage of your ability to see things like head rotation, gaze direction, hand position, and more to create unique triggers and effects with your game’s soundscape.

“There are a lot of opportunities there that can and should be used, when you’re prototyping, to consider how to do your music,” said Perry, showing an example of how a player’s head rotation during Vacation Simulator is used to dramatize a photography challenge for comedic effect. Look towards the natural beauty of the VR landscape, and soothing music plays; look towards the camera, and dramatic pulse-pounding bass kicks in. 

“Light-hearted humor, with the power of music, is very easily achieved and enhanced by using head rotation data,” explained Perry.

And of course, music is also a key tool you can use to tell players when they’re succeeding or failing, or whether they should be looking at something.

“Something you can not do in a conventional game...is to use your [player’s] head to create focal points,” said Perry. “For instance if you look at one bowl, the game clears out all the other sound elements, and you hear only the bowl...I think this lends itself to another cool game mechanic I’d definitely encourage you to explore, which is using a stethoscope or microphone to point at specific points in the world and ‘focus in’.”

As another example of what you can do with music in VR, Perry suggested you try and treat objects in your game like musical instruments, to help add life to your environment and make it feel more responsive. 

“VR, if you do it right, is a very responsive environment,” said Perry. “It should react to anything you do.”

By way of example, he showcased how a menu in Vacation Simulator is designed to play specific notes when players highlight specific options, creating an almost xylophone-like effect when the player scrolls through the options. It's a good trick.

With that in mind, Perry reminds game makers that you can use musical effects to differentiate and highlight anything in your game, including individual objects -- but you have to remember that whatever audio you tie to a specific object or event must harmonize with the background music and the rest of the game. Otherwise, you risk causing discordance and discomfort, a big no-no in VR.

And if you’re going to try and build something in VR which works like a musical instrument (be it a menu or a mandolin), Perry has some advice. First, he reminds that human hands are surprisingly small and powerful, capable of doing things to instruments (like managing the strings on the fret of a guitar) that VR can’t yet handle. So there’s a decent chance you’ll cause people to doubt themselves and your VR environment if you try to accurately duplicate something as complex as a six-string guitar.

Instead, Perry recommends you create VR musical instruments with an eye towards approachability and invention.

“Because in the real world we’re bound to the physics system, like with a trumpet it has the three valves at the top,” said Perry. “We don’t have to [in VR]; we are not bound to physics...meaning we can make instruments that are more intuitive.”

 He showcased how the Vacation Simulator team stuck to this design philosophy with the example of a guitar in Vacation Simulator, which is specfically designed to be played one-handed, with a wide “strum threshold” and a colorful design to make it feel approachable to players.

To sum things up, Perry closed out his talk with three big tips for game makers looking to improve the way they use music in VR:

Gamasutra, XRDC, and GDC are sibling organizations under parent company Informa