Loot boxes have been the subject of tremendous controversy over the last few months, with some very high profile games blundering into this format of monetization in some extremely clumsy, extremely high profile ways.

Ignoring some of the basic precepts that developers have hashed out around loot boxes and real money purchases (rewards should be cosmetic and not influence game balance, contents should always feel appropriately valuable, etc.), many of these attempts have backfired in spectacular fashion.

But lost in the conversation around triple-A publishers seeking additional revenue through microtransactions or whether or not loot boxes are gambling is the damage these public blunders are doing to the free-to-play model, to games that have relied on loot boxes and executed them gracefully and successfully for years now.

As much as recent high-profile stumbles have been monopolizing coverage, there are a huge number of smaller developers that have been doing loot boxes elegantly for a long time, and they often rely on them to keep the lights on and to continue development/support.

With that in mind, Gamasutra reached out to a number of MMO developers to get a sense of what loot boxes mean in the free-to-play space, how they can be done tastefully and non-intrusively, and how they can be implemented without players feeling exploited or deceived. 

The allure of loot

Given how players often react to them, loot boxes represent a double-edged sword for developers; on one hand, they represent a potential revenue stream, but on the other there’s a risk of alienating users. As Matt Pettit, a producer on the free-to-play MMO shooter Defiance, is quick to point out, you make no money from players who walk away from your game.

“We think about the player-facing perspective as the priority, and the business need second,” Pettit says. “Without the first, the second doesn’t happen.  At our core, we’re focused on making the kinds of games that we want to play too, and making sure they’re sustainable as businesses. Creating a fair revenue stream for a free game is essential; the game wouldn’t be able to exist otherwise.”

Petitt says his team wanted to make sure that their loot boxes, called Supply Caches in Defiance, were part of the core gameplay loop, and attainable in-game as well as on from their store.

“Having Supply Caches on the store gives us that opportunity to provide players with two different ways to pursue their goals, and that choice is what attracted us to them in the first place.”

Another opportunity afforded by loot boxes is the ability to tuck odd, rare, or unique rewards into them, content that might exist on the fringe of a game’s universe. For Star Trek Online, which allows players to earn loot boxes through both real money and by trading in-game currency on an exchange with other players (and makes all items tradeable so that players have the opportunity to buy items that appear in boxes a la carte), this means loot boxes allow devs to control, to some extent, the proliferation of new items. It allows them to unleash new vessels that may seem out of place in larger numbers.

“Loot boxes provide the team with opportunities to deliver atypical starships in a way that keeps them rare, which helps maintain the in-universe believability of our fictional setting,” says executive producer Stephen Ricossa. “Lock boxes also allow us to experiment with concepts and designs for items because we know the player sample will be smaller than a standard release. We can then use the information we’ve gained based on back-end data, in-player auction pricing, and forums or social posts to determine the popularity of our experiment and potentially bring more of that type of item into the game.”

For World of Warships, loot boxes solve a completely different problem. For Philip Molodkovets, the head of Warships’ game analytics team, loot boxes (containers) were a way to solve a number of long-standing issues with daily missions.

“They were randomly assigned to you each day, with the ability to re-roll one of them in case you didn't like it. The rewards were fixed, known from the start, and typically were small amounts of in-game currency or consumables.”

While a fairly standard approach to an MMO staple, Molodkovets said it resulted in a number of issues. Players weren’t engaged when completing their dailies, and were often frustrated when they called for specific classes or weapons, ones they weren’t familiar with or didn’t enjoy using. Fixed rewards also meant that they’d never yield epic loot, and curtailed player choice.

“Funnily enough, our random loot containers offer more choice to the player. Before receiving a container, you may choose its type," he said. "This won't set your reward in stone, but guarantees it will partly contain the type of items you're interested in.”

And the container system, which awards a box after a certain amount of XP accrued instead of after players accomplished a specific task, means players can use whatever ships they like while chasing them, and are more engaged by the process of opening a container than with a static slate of goodies, especially knowing that a super rare item might be hidden inside. Containers also provided utility in a way that daily missions didn’t and have been incorporated into other reward systems and as part of holiday and special events.

What’s in the box?

While there’s a lot to recommend random loot, deciding which items to apportion to boxes and how to set the rarity on those items can be a tricky balancing act. There’s the ever-present threat of a misstep infuriating your community, and a huge number of factors influence contents and rarity. For the Defiance team, a few core principles shape the way their reward caches are designed.

“First, allow the player to obtain everything in-game through gameplay, and offer the store as an alternate way of accomplishing that goal,” Pettit told me.

“The balance is ensuring that what players can accomplish in-game does not take an unfair amount of time. Second, we make sure that no matter what one gets from a Supply Cache, if a person does choose to buy one, they’re always at least getting the value out that they put into it. Purchasing something from the store needs to always feel rewarding and speed up your progress. Even though the items you get from the store are the same as the items you get in the game, often a store-purchased Cache will have a higher chance for jackpot and rare items.”

Setting drop rates for the items in Defiance's caches comes down to calculating how long it takes for a non-paying player to obtain them, and balancing that with how they’re obtained, whether as a standard reward or through some random gameplay element. Figuring out those ratios is an ongoing task and one that Pettit says is core to ensuring that Defiance never feels pay-to-win.

It’s a challenge that Ricossa also readily acknowledges, one the Star Trek Online team approached through experimentation and careful iteration.

“We took our best guess with our initial lock box offerings and based the drop rate of items on what we felt would be cool or interesting from a player’s perspective," he said. "As we released more boxes, we adjusted drop rates based on general server-wide availability, player satisfaction, and interest in the items. Some of the items that we thought were cool didn’t resonate, so we either made them more common or eliminated them entirely.”

Ensuring an even playing field is much more difficult proposition than it often appears from the outside, a continuous process that can be difficult to read if there aren’t dramatic swings in one direction or the other.

“Unless something is catastrophically broken, it can be tough,” Ricossa said. “When we’re making lock box items, we talk extensively about what can go wrong, we playtest them internally in a variety of loadouts and try to think outside the box on the different combinations that could be made with existing items. At the end of the day, our goal is to provide people with things that give them more combat options and not necessarily make them more powerful.”

The result of all that research and iteration is a streamlined, developer-friendly system. “These days we have a refined menu of items that we can choose from and each of those item types has a set rarity.  When a designer builds the box they choose what they want to make, make very minor adjustments to the drop rates so everything adds up, and they’re set.”

But unless you’re restricting yourself to purely cosmetic loot, ensuring new loot doesn’t upend your game’s carefully calibrated parity is a huge undertaking, one Molodkovets admits the Warships devs have struggled with.

“The goal was, of course, to sustain the balance between appealing loot and not overflowing the game with free premium items," he said. "It was pure mathematics, for the most part. When we designed the drop rates we picked the most popular consumables and monitored the flow we generated by releasing containers with them.”

But despite their best efforts, Molodkovets found that some of the container items were disrupting the game, particularly consumable experience and performance boosters called signal flags.

“Distributing them with the containers contributed to their impact on game balance and their conceptual value too much. They had become a no-brainer choice instead of a valuable situational tool for customizing your ship," he said. "At the same time, we introduced special ship upgrades, and put them into epic rewards instead. Unfortunately these upgrades were not balanced enough, and some of them gave too little value for players.”

Molodkovets says the backlash was strong and immediate, but that the team responded quickly to redesign and improve the flags and upgrades. “Of course, not everybody is happy with the feature, and there are players who believe they should pull a premium ship each time they roll epic, but as a whole, we consider containers both appealing and safe for balance in their current state.”

The look and feel

Almost as important as the contents of a loot box are the aesthetics of it, the visual and aural flourishes and effects that make opening a box feel like an occasion, like the player is discovering some rare, incredible treasure. For the Warships team, the design of containers is drawn from the world of naval warfare they’re emulating.

“The containers have the austere design and animation of actual cargo containers. They are operated aboard the SS Red Oak Victory - a U.S. military cargo ship that can be found in any of the game ports. To animate the surroundings, we added a few crewmen engaged in various activities. The sound design followed the same strict concept.”

But Molodkovets recognizes that cracking open loot doesn’t have to be grim, serious business. “When we design special event containers, we let ourselves venture beyond classic naval style. For example, Christmas containers looked like gift boxes, and introduced Santa himself to the world of brutal naval warfare, and we had more fun designs in our Halloween event.”

The Star Trek Online team started with boxes that fit the slick futuristic aesthetic of the franchise’s 25th century. But they quickly realized something was missing.

“As for the opening mechanic, we actually launched without one! We decided to update our opening experience for players so it would feel more satisfying and have some visual flair, while not making the process of opening boxes take longer. We felt updating that interface was a critical part to improving the lock box experience.”

For Ricossa and the Star Trek Online team, the key isn’t so much the process of cracking open the box as much as the look and theme of what’s inside.

“The best way to ensure that the box itself is well received is that the items look great, are valuable to players and typically interact with the current story based events of the game,” he said. He claims the team wants players to be able to live out their Star Trek fantasies, and lock boxes are a way to flesh them out.

“For example, if someone wants to ‘be a Tzenkethi,’ we have a lock box that offers all of the ships, items, and cosmetic apparel they would need to look and fight like a Tzenkethi," said Ricossa. "Providing great items and interesting new gameplay opportunities at a value for the player community is the best way to ensure you can keep making your game while everyone playing it is having fun.”

Pettit agrees with this sentiment — for Defiance, the key is ensure players are happy with the result of their purchase first, and consider aesthetics later. While a zone-wide broadcast lets other players know anytime a rare Jackpot item has been obtained from a lock box, Pettit admits that the actual ritual of unlocking one could be better.

“The animation and sounds around opening a store Cache is one area where we really want to improve! Opening up a Cache should feel like opening a new pack of cards, or spilling candy out of a piñata," he said. "That’s something I’d like us to do better, but what I think we do right is make sure to provide is as much information as possible.

“We feel that details are incredibly important both when players are paying for a Supply Cache and when they purchase a Cache with in game currency," the Defiance dev continued. "They should know what rewards are hidden inside, along with what special drops they could receive. Fanfare when opening a box is a nice touch, but preparing you for what you will receive, and being fair in their designs, while also providing a sense of reward, is of the utmost importance to us.”

In speaking to these experienced F2P game devs it's clear that the question of how (and when, and why) to best implement "loot box" monetization schemes in your game is complex, even if you've been doing it for a while.