Virtual reality tends to grab the lion’s share of headlines when it comes to new gaming technology.

Mixed and augmented reality are often marginalized, only mentioned peripherally in relation to other stories like Pokémon Go’s massive success. VR has the advantage of a library of buzzwords instilled in pop culture through decades of science fiction; it’s easy to conceptualize its virtual worlds, its potential is visceral and obvious.

But it’s not just mindshare—AR is also comparatively fresh and untested territory. It’s tempting to think “everyone has a smartphone, AR has a huge install base,” but developing in AR often means wandering into unexplored territory and using (or building) nascent technologies, some of which are advanced enough to disqualify a big part of that install base. It also means playing in a field with really exciting potential for blending together real world and digital experiences.

“We wanted to bring the zombie apocalypse into the real world,” says Clayton Neuman, VP of games and entertainment at AMC, of The Walking Dead: Our World, a mobile AR game that lets players hunt walkers and undertake missions in the world of the cable network’s hit series. “[Walking Dead fans love] this hypothetical: ‘What would I do in a zombie apocalypse?’ The technologies enabled by AR allow our fans to explore that hypothetical in the most realistic way possible.”

AR lets developers play in the largest creative space available: the real world. On paper, it’s a tremendous idea, an almost unlimited space to explore to play in without many of constraints of traditional open world games. But bringing those ambitious ideas to life is much more challenging, especially when you’re among the first to attempt it. For Sulka Haro of Next Games, lead designer on The Walking Dead: Our World, not being able to rely on precedent was a huge challenge.

“As we started developing the game,” Haro says, “one of the things that hit us pretty quickly was that there are few existing location-based games that use a map that we could use as a reference to see how other developers have solved issues specific to what we wanted to achieve. We took a lot of inspiration from other games that use location mechanics on a map, but fundamentally we were building an MMO that has a single huge instance with every player in the same game.”

Building a massively multiplayer game atop the real world

Building an MMO game with the real world as your setting is an enormous challenge, fraught with a number of difficulties, one of the most pressing being the sheer diversity of areas the game would have to account for.

“The geography of each area in the world is wildly different, so if you want to provide a gameplay experience that's even remotely comparable across different players, your content distribution algorithms need to be pretty sophisticated and you need to do a lot of testing. We wanted the game to work not only in large cities (think of Manhattan) but also in rural areas ranging from Kansas with it's unique field grids, to the Finnish woods where our developers have their cottages.”

Of course, in a planet with a surface composed of more than 80 percent water, determining playable areas is a huge early step. “Water and other areas add another layer of complexity,” Haro explains, “where if you divide the planet into squares (which need to change shape depending where you are on the planet - we live on a ball) the amount of usable land for the gameplay varies hugely, so we needed to develop a system that looks at the available land area for each location and then distributes missions in proportion to the amount of land available."

Once you’ve established how much land is available to populate, you then have to compensate for the population density of those areas as much as possible.

“Going back to the cottage in Finnish woods, this might be in an area with just a single player per square kilometer, whereas in Manhattan, on a busy commute spot there could be a huge amount of players passing through. Given we wanted the game to feature a lot of social features, both location-based and those available at all times, this gave us some unique challenges on how to to scale the game. The game design needed to scale from ‘no players’ to ‘a huge amount of players’, even for social mechanics. Similarly, the backend technology for the game was designed so that each player dynamically subscribes to changes in the nearby areas based on how they move, so the total number of players who can ‘hear’ each other's social actions is somewhat limited, even in densely-populated areas.”

As they began prototyping early solutions, the team discovered that the game was often failing to feed the same content to players, even if they were together. It made their core idea of a social experience moot and crippled a lot of the interactivity they were trying to foster.

“This made us determine that it's critically important the players see the same content at all times,” Haro says, “allowing players to do things like hop into a car and go collect things together. The mission population itself is actually handled by a highly parametrizable deterministic algorithm that procedurally creates the same mission layout for all players, so the core experience of what players see in the game doesn't require any server communication to happen. The algorithm is seeded with data from a new Google API that provides game developers point data in the real world, with an importance score for each point, so the game knows how to place the most important mission content to spots on the map that are most likely to be visited by the players.”

When Haro and his team first began designing their game, it truly was the dark ages of AR on mobile; they were trying to build in-house solutions to some truly staggering problems that some of the biggest companies on the planet were having trouble grappling with.

“To set the context for this - we started developing the game a long time before ARKit and ARCore had been announced," he says. "Our initial implementation of the AR features just used the device sensors in a pretty raw form but were realizing the AR would be a huge battery drain, in addition to having technical issues like sensor drift with multiple devices.”

Luckily, while the game was still in the early phases of development Apple announced its ARKit AR solution, allowing multiple users to play together in persistent experiences, and tracking 2D and 3D objects. Shortly thereafter, Google released it’s own AR SDK, ARCore, a powerful tool that assesses and analyzes a devices environment and tracks its position in the world.

“It was a huge blessing,” Haro says. “We had an excellent technical solution on both the platforms we wanted to support.”

Real-world games are fraught with real-world expectations

Another big hurdle has been managing the expectations of players, who frequently don’t take into account just how raw and complex AR is.

“The first games that have featured gameplay pitched as AR have mostly used simple a camera feed on top of an object without any real world tracking features," says Haro. "This has made a lot of players expect that because their phone ran a game that was advertised as an AR experience, they can automatically run all other games with AR, even if their device is not compatible, and nobody likes being told they need a newer phone.”

Making a game that’s accessible to as many players as possible has meant some compromises, including building a version that eschew’s AR altogether.

“One of the early decisions we made was that players could choose to play any of the missions with or without AR,” Haro explains, “so all content was accessible to players regardless of what device they have. This in turn resulted in pretty tight requirements for what we could do with the AR mode in terms of performance and the type of game design that worked similarly in both graphics modes, as we didn't want to re-balance all the content separately for the modes.”

Safety was another important consideration: some of the initial mission designs inspired players sprinting around while staring at their phones, a recipe for disaster, so Haro’s team implemented a bounding box that paused missions when players stepped out of it.

Designing in AR has been a huge learning experience for Haro, who found that it required an even greater level of testing and iteration than traditional game design.

“A big lesson would be that you should reserve a lot more time for prototyping and design validation than usual,” he admits. “Using the real world as a design canvas introduces a lot of player expectations and limitations that are out of your control, so any changes to the game can have unexpected consequences in terms of player experience. To give you an example, one of our prototypes include an early version of the player-built safe houses, where the houses were only visible to the builder and the player had to be in close physical proximity to the building to collect resources. Everyone thought this was a great idea until we tried it and found everyone got terribly anxious, thinking they might not be able to collect resources in time due to physically not being at the right place.”

Of course, AR isn’t just a new concept for devs, it’s also brand new to many players. “Another surprising finding was about a third of our players automatically assume they have to stand next to missions to be able to access content and need a lot of guidance to understand some content can be accessed from a distance. Half of the players don't intuitively move at all when using AR and just assume they have to stand still.”

It’s the double-edged sword of working with any brand new technology; no one completely understands it, which can make for a lot of clumsiness, but it also means you get to help pioneer brand new standards.

“It feels like both the design language regarding AR is still being created and the player community's ability to read into mechanics around AR is still developing," concludes Haro. "The lack of conventions makes designing these games much harder, but also makes the work so much more interesting.”