HoloLens dev shares advice on developing mixed-reality games

Ralph Barbagallo is the founder of Flarb, a studio working on the augmented-reality game Ether Drift for (among other things) Microsoft’s HoloLens headset.

HoloLens has yet to see a commercial release (though you can get yourself a devkit for a few thousand bucks) so Barbagallo’s HoloLens development learnings are highly intriguing -- though of course a good chunk of his takeaways may not be as relevant by the time HoloLens hits.

“A lot of the information I’m going to give you….is probably obsolete by the time I’m done,” Barbagallo said today in a talk at VRDC in San Francisco about developing for HoloLens. But some of the core lessons he shared likley won't be -- including some basic guidelines about making games for AR headsets.

For devs, know this: after spending a bunch of time developing, testing and debugging on HoloLens, Barbagello is keen to remind fellow devs that you need to spend a lot of time thinking about the whole room someone is playing your game in.

“It’s called mixed reality for a reason. When you’re planning your HoloLens content...you can use the floors, you can use the ceiling, you can use the real world,” says Barbagallo. “Makes it a little more annoying to develop for, but that’s unique to this platform.”

So when you're thinking about developing for HoloLens, you should be thinking about how you integrate your game with the real world: how you can make a game that works well within HoloLens' limited field of view ("if your game involves walking around and looking for stuff, not having any peripheral vision is going to be a problem,") and how you can take advantage of its unique strengths -- most notably, limited gesture control, player gaze and voice recognition.

“You have full access to Cortana on HoloLens; It’s a fully self-contained Windows 10 PC that you wear on your face,” says Barbagallo. So don't forget that “you can use Cortana’s own voice-recognition capabilities in your app.”

Another thing game developers coming to HoloLens should focus on, says Barbagallo, is spatial mapping -- it's not something you traditionally have to think about, but in AR game design it's key to ensuring your game assets don't appear out of sync with the real world, breaking player immersion.

“You really want to know about your environment, and where stuff is,” says Barbagallo. “Where’s a good empty space to place a game character, for example.”

And of course, the mixed-reality nature of HoloLens games means devs should expect to spend extra time not only learning how to make use of the environment around the player, but how their game will work (or not) in the many places people might choose to play it. 

"The build-test-debug loop is so much longer than VR," says Barbagallo. "You really have to test in a lot of different physical environments....it's a practical challenge, how to debug something that can be played in any practical space."