Daily news, dev blogs, and stories from Game Developer straight to your inbox
Sponsored By
Featured Blog | This community-written post highlights the best of what the game industry has to offer. Read more like it on the Game Developer Blogs.
Survivor Error
All humans can easily fall victim to survivorship bias. With the plethora of information and stats available to us today, it is more important than ever for designers to look past the success stories before the winning formula is revealed to them.
10 Min Read
As part of an academic class under Jesse Schell, I was exposed to the business aspect of games and how evaluating the playing field goes a long way in determining the road to success. One of the readings I studied was Emeric Thoa's post that provided data and trends related to the top apps in the mobile industry. If we look around, there are plenty of articles and reports evaluating the top companies and their products in every possible sector. And this is a huge thing today isn't it? Big Data. The power of recording every stat and more importantly, the power to analyze it in excruciating detail. But occasionally, we get bogged down by all these details. We end up looking at only the success stories. Actually, let me rephrase that: we are allowed to only look at the success stories and this leads to one of the most interesting logical fallacies: Survivorship Bias.
For those who are unfamiliar with Survivorship Bias, let me give you a military example to explain it:
During WWII, the chances of a bomber crew surviving a mission were almost 50-50. Even if they managed to make it back, the prolonged war meant they would fly out again soon until their luck ran out. It was crucial to find any kind of modification that would improve the odds of a bomber making it home. The engineers knew that the allied bombers needed more armor, but they couldn’t just cover the planes up like tanks, not if they wanted them to take off. They had to carefully choose the sections to be reinforced. They looked at the planes that had returned from enemy territory. They recorded where those planes had taken the most damage. Over and over again, they saw that the bullet holes tended to accumulate along the wings, around the tail gunner, and down the center of the body. Wings. Body. Tail gunner.
The commanders wanted to add the thicker protection where they could clearly see the most damage, where the holes clustered. But Abraham Wald who was advising them said no, that would be precisely the wrong decision. The holes showed where a bomber could be shot and still survive the flight home, Wald explained. After all, here they were, holes and all. It was the planes that weren’t there that needed extra protection, and they had needed it in places that these planes had not. The holes in the surviving planes actually revealed the locations that needed the least additional armor. Look at where the survivors are unharmed, he said, and that’s where these bombers are most vulnerable; that’s where the planes that didn’t make it back were hit.
A similar logical error is possible while evaluating video games when we consider only the ones that do well. There are definitely some insights within success stories but it's equally important to consider the titles that failed or others that you haven't even heard of before jumping to conclusions.
Let's take the MOBA genre as an example. League of Legends is probably one of the most successful games out there today and is helping bring e-sports into the limelight with a very professional approach. Having played the game since season one, I have seen it evolve from its rough and unproven origins. League was considered to be a cartoonish clone of DotA, meant for casual audiences and most people dismissed it in anticipation for the release of DotA 2. If you look at the success story of LoL, you'll find that its revenue model was unique. More importantly, Riot Games stuck to it throughout the early days even when there weren't many players invested in their game. This gave them an edge over competitors like Heroes of Newerth. But there was more to the game than just its business model.
Before we get into those details, let's take a look at some of the recent MOBAs that failed. These were games who had plenty of time to observe the successful titles, yet they still didn't make the cut:
1. Sins of a Dark Age
Status: Development ceased
Reason: More of a DotA clone that LoL, SoaDA hardly innovated anywhere and had no new features to boast about. To make it worse, their UI lacked the polish that their competitors had and thus it couldn't generate a strong player base.
2. Dethroned!
Status: Funding/Development ceased
Reason: This was a unique game where they tried to mix MOBA and RTS elements. The visual style was simple and so was the gameplay. A little too simple unfortunately. And that's where it lacked depth that players expect from this genre. In addition, there were lots of bugs that weren't ironed out so again, they couldn't retain enough players to continue development.
3. Panzar
Status: Near empty servers / close to extinction
Reason: Panzar was a game with pretty graphics riding on a strong engine. But they lost the plot with the grinding involved for gaining the gear and everyone recognized it as a pay to win game. This led to players abandoning the game over time.
4. Dawngate
Status: Development shut down
Reason: Dawngate was expected to be the next big thing especially since it had EA backing it. But that may be the reason why it never took wings. The game had a decent start and seemed to be in good shape for a beta version. However, the plug was pulled because it didn't attract a crowd large enough to take on LoL or DotA 2.
5. Infinite Crisis
Status: Servers shut down
Reason: IC was a title that tried too much without paying sufficient attention to detail. They had a strong backbone in the form of the DC universe, but they didn't build upon that. There were several complaints of negative interactions with the developers who were also notorious for not communicating at all. The game had balance and toxicity issues that were not addressed. And instead of the refining these features they were focused on e-sports and Twitch support. Their weak foundation didn't take long to crumble.
As you can see, there were several aspects apart from forced payment that led to player dissatisfaction. Riot made a lot of effort in different areas to stand out:
They embraced their design of being a casual MOBA. Elements like denying creeps and losing gold on death were removed. These simple changes made it easier for new players to get into the game and expanded the player base to include gamers who were too overwhelmed by DotA. They also allowed the losing side to have comeback opportunities which kept games interesting throughout.
The developers were quick to fix any major bugs and Riot has been fairly transparent when handling abusers. The patches were frequent and there was a constant effort to balance the game.
Throughout their initial years, Riot was constantly releasing champions and other content into the game to prevent it from getting stale.
The staff members spend time playing with other players to get an idea of the game and some decisions are based of these experiences. A lot of it is now decided by competitive play but they still have their ear to the ground.
Riot has constantly battled with toxicity and experimented with incentives like the Tribunal to address such behavior.
Within e-sports, they have also made continuous effort to improve the production value and they approach the broadcast very professionally - their quality was considered high enough to be featured on ESPN.
While monetization is definitely a key aspect, just utilizing a free-to-play model like LoL is not sufficient to create a successful MOBA. There were a lot of other factors which led to its success and these traits can only be revealed when you contrast it against the ones that stumbled after their initial steps. That's why it will take much more even from big games like Smite and Heroes of the Storm to scale those heights.
There is another example much closer to home. While designing for my current project, we had to create a tablet app that would make answering questions fun. Based on our target audience, I had to figure out the experiences that kids aged 7 - 11 years would like. My natural instinct was to look up the top rated apps on the store. It was difficult to find references for this age range, but I eventually stumbled onto a few good apps and it was easier to explore further by looking at related content. Based on this initial research as well as the observations from previous projects, it seemed like the key to an enjoyable experience was simple: Customization. Provide a ton of options for children to express themselves with and they will be satisfied. But customization was a feature that did not match our project goals. There needed to be some other hook within the experience to keep the children engaged. Thus, I started looking at mechanics.
Tap, swipe and drag were the most widely implemented gestures in games meant for a wide audience. The key learnings here were:
Keep the interface as clean as possible. Do not add complicated gestures or controls.
The tasks themselves should be discrete and easy to perform.
There were some age specific facts too -
Avoid keeping interactive buttons at the bottom of the screen as kids are likely to press them by accident.
Children are able to perceive when an experience is designed for an age level older or younger than themselves and may have strong opinions regarding it.
'Challenge and Reward' is the favored approach that most of these games take.
Now, if looking up the top apps was difficult, searching for the bad ones was close to impossible. There is no way to sort the apps by rating (possibly to prevent cheating) and looking at the bottom of charts meant that I couldn't filter for the required age range. Another hurdle was that I was looking for apps which were bad in terms of design and not because of lack of effort put in. There are one and two star rated games that are basically interactive memes but I was looking beyond that. I needed to check out games that had decent production value but still failed. After a lot of manual searching, I found some games which revealed other aspects of a tablet experience that I might have missed completely:
One of the major reasons for player discontent (apart from micro-transactions) is inconsistent performance. Games like Breakneck, a racing game with frame-rate issues and Disney Color and Play, which had unreliable page scanning both left their customers very frustrated. These issues were enough to tarnish the reputation of experiences that had the potential to be really successful. I realized that all the top games I had researched previously had one thing in common: No matter how complex the world was, there were no bugs or performance issues. The experience was seamless.
A game like Feed the Duck proved the importance of understanding the needs of your audience and the genre you are creating for. It should come as no surprise that a physics based game with bugs and bad world feedback was not very successful. While the art and concept were unique, the shallowness of the game prevented it from attracting puzzle game enthusiasts.
And finally, Final Fantasy: All the Bravest showed me how it was necessary to deliver on the audience's expectations. A player entering into a Final Fantasy game awaits the moment of battle when they can issue orders and watch their actions play out. Instead the combat in this game devolved into swiping the screen. That's it. Really! Go check out a gameplay video. Some of the animations are not even visible as your finger obstructs the view as you desperately try to swipe everywhere. Even big names are not immune to such decisions.
Well, this has been a long post. My objective was to bring attention to the fact that when we research or analyze something, it's vital to look at both ends of the spectrum to gain valuable insight. When the focus is just on the things that fared well, it's easy to overlook certain traits that we take for granted. Sometimes, these are the qualities that end up being crucial towards the success of the end product!
Credit to David McRaney for serving as the inspiration for this post.
Read more about:
Featured BlogsAbout the Author(s)
You May Also Like
