Indiepocalypse? More like INDIESCHMOCALYPSE!

 

If you haven't heard of it, the idea is (roughly) as follows: The barrier to entry for making and distributing video games has become super low. Consequently, the market is becoming supersaturated with indie games, and it is therefore increasingly unlikely for indie developers to find financial success in the industry even if they make a great game. The idea is that the pie (total games sold) isn't getting bigger, therefore the shares are getting smaller, and soon the market will be untenable for indie studios.

 

Figure 1 | AAAAAApocalypse!!!! (Data from Steamspy)

 

And that argument sounds legit. I mean, just look at Figure 1. On Steam only 10-20 indie games were released per month at the start of 2013, a mere 2.5 years later there were up to 200 indie games launched per month.

 

"SO WHY ARE YOU WRITING ABOUT IT NOW!?" you might be asking.

 

Well, we're closing in on the all-platform launch of our pride and joy, Crashlands. While planning launch strategies we came across a different piece of suspect industry wisdom. That is, that indie studios should not launch their games during the pre-holiday months, because that is when the huge developers release their games. Since a small indie game can't compete with a major AAA title, our launch would fail. Or so the logic goes.

 

That argument is well-reasoned, but its assumptions (that indie games even do compete with AAA titles) have no data defending them. And I couldn't find such data elsewhere on the interwebs. We really want to be able to launch soon, so I decided to test that claim. In the process, I realized that I also had the data to test the Indiepocalypse Hypothesis. So I did.

 

The result: There is no seasonal conflict when releasing indie games, and there is no Indiepocalypse.

The data

The Indiepocalypse Hypothesis

The loose hypothesis of the Indiepocalypse is this: As more indie developers enter the market, fewer indie developers will succeed. The first part of this hypothesis needs no further definition, but the latter part absolutely does (and the fact that its meaning isn't crystal clear is why I've been ignoring articles on this topic in the first place).

By "fewer indie developers," we could mean a smaller fraction of the overall total, or a smaller absolute number. Defining it as a smaller fraction is pointless because as huge numbers of developers enter the market, duh, of course a smaller fraction will succeed! This is necessarily the consequence of either (1) that player demand for games is lower than the rate of new game releases, or (2) that players do not want most new games (for any number of reasons: lower quality, less novelty, poor marketing, etc). The data I show later shows that demand for indie games is not increasing over time on Steam, so condition 1 is satisfied!

So the interesting case is when "fewer indie developers" means "a smaller absolute number." So our hypothesis becomes clarified as: As more indie developers enter the market, a smaller absolute number of them will succeed.

Great, so we've defined the first part of the hypothesis. But what does it mean to "succeed?" There are all kinds of success: critical acclaim, meeting a niche market, getting emotional fulfillment, etc. Here, though, I only mean it in the pragmatic sense: level of success is proportional to how many games you have sold. Success is relative in this sense, as any particular studio+game combination will require movement of a different number of that game in order to stay afloat.

Yeah, actual revenue and, even better, return-on-investment are even better metrics of success, but that data is not readily available. What is available, via Steamspy, is the number of people who own each game ("owners"), which I use here as a proxy for "success" - games with a higher number of owners are relatively more successful than those with fewer owners. So the specific, testable hypothesis becomes the following:

Indiepocalypse Hypothesis: As more indie games enter the market, each game will have fewer owners.

This makes logical sense at first. If the pie is getting sliced into more and more pieces, doesn't each piece have to get smaller? NOPE! That would only be true if each piece is equal. Spoiler alert: they aren't.

Testing the Indiepocalypse Hypothesis

If the market is saturated with games, the average success per game will be far lower than when the market isn't saturated. But this is just a boring fact of bad statistics. Yeah, sure, if you average all purchasing across a continuously larger number of games, that average goes down. But the average is the wrong metric. The relevant question is: does everyone do worse because there are more games?
 

Figure 2 | Beware the axes! Left, distribution of per-game ownership. Right, log10 transform of the same data, with fitted normal distributions. Top, data for games launched in 2013. The means correspond to ~98,000 (in 2013) , ~23,000 (in 2014), and  ~5,000 (in 2015) owned games.

 

Player purchases are not distributed equally across games. It's more of an exponential decay (pretty close to log-normal as the above plot shows). So, as more and more games enter the fray, is it the case that fewer total games succeed?

 

Well, Figure 2 seems to confirm the Indiepocalypse: the average ownership per indie game in 2014 is only 23% what it was in 2013, and it's reduced by 3/4 again in 2015! There are many more games, and it seems that everyone is losing. Game over. Pack up your laptops and find another job.

But hold up. There were way more indie games released in 2014 than 2013, and there were more games in the first half of 2015 than in all of 2014. So we should put all these on the same plot to compare (Figure 3).
 

Figure 3 | Many more indie titles were launched in 2014 than in 2013, and even more came out in the first half of 2015. While This data seems to show that indie games did significantly worse in 2014, and way worse in 2015.

 

Wow. That looks pretty frightening. Figure 3 shows the distribution of success moving significantly further to the left every year (and note the log scale, 1 unit left means 1/10th the success). But there is something interesting about the right tail of the 2014 distribution (Figure 3, blue) compared to the full 2013 distribution (red): it looks an awful lot like the 2014 distribution is a mixture of the 2013 distribution plus something else. This is the statistics concept of a "mixture model," where the distribution we see is actually composed of multiple distributions "mixed together."

That means this data is consistent with (at least) two hypotheses: (1) The Indiepocalypse is now, and everyone is going to lose even harder as time goes on; or (2) there are two kinds of indie games, successful and unsuccessful, and these games do not compete with each other. To find out if the second option makes sense, I re-plotted the Figure 3 data with only those games that received Steam user scores of at least 70% favorable (Figure 4).

Figure 4 | Restricting the data in Figure 3 to only "good" indie games (those with user scores >70%), we see that good indie games performed about as well in 2014 as in 2013. Good indie game seem to be performing worse in this first half of 2015, but this could easily be a consequence of time (see text for explanation).

 

Well, now, that looks a bit less terrifying. Figure 4 shows that "good" games in 2014 performed just as well as in 2013. It does seem to show that good games in 2015 are struggling; there appear to be fewer successful games (lower peak) with overall less success (peak shifted to the left). But this assumes that the distributions will not change over time. So, I wondered, is it likely that, given time, the 2015 distribution will shift right and get taller?

As for the question of getting taller, what needs to happen is that roughly 2x more good games come out. But guess what! Figure 4 only has 2015 data for the first half of the year, so we need to see if it is a reasonable guess that the second half will provide (roughly) as many good games. Plotting the release rates of only "good" games over time results in the right-hand plot of Figure 5. Note the y axis: roughly ten good indie games come out a month. The variation is high, but this is not surprising since the numbers are so small. From a statistical standpoint, then, the rate of "good" indie game releases has been constant over the timespan tested! And so, we can reasonably guess that the 2015 peak in Figure 4 will end up (roughly) twice as high as it is now.
 

Figure 5 | While the number of indie games released per month has increased dramatically over 2013-2014, the number of "good" games (having a Steam user rating of 70% or higher) released per month has been statistically constant.

 

But will the same curve shift to the right over time? About a year from now I can just test that idea directly. Unfortunately, for now we have to make a reasonable guess. A super-important point here is that success (measured by number of owners) can only increase over time as new players buy it. Meaning that, if the curve is going to shift, it is going to shift right, towards the 2013 and 2014 curves of Figure 4. Recently-released games have undoubtedly not yet maximized their potential success, and half of this year's "good" games haven't even been released yet.

If I plot the data over time (Figure 6), instead of throwing it all together as in Figure 4, it becomes even more obvious that "good' games (right plot) do not show the plummeting trend that indie games do overall (left plot). (I discuss this figure a bit more, below.) Still though, there is a downward trend for good games; only time will tell if this is because they haven't saturated their potential sales or if there is a downward sales trend for indie games. In any case, the effect is not nearly as dramatic as is the increased rate of new games, or the plummeting rate of success of those games.

Figure 6 | Release month and game ownership are indeed correlated (pc = Pearson Correlation Coefficient, P = p-value), though this correlation is much stronger for all games than for good games only (user rating of at least 70%). Gray dots are individual games. Red dots and crosses are the monthly median and quartiles. Blue squares are the sum of ownership across all games released that month.

 

(Note that, if you define the Indiepocalypse to mean that the fraction of successful games will plummet then, not only are you totally right, but the above plots show that this has already happened!)

In short, I find no evidence for the Indiepocalypse Hypothesis on Steam.
 

Does this "good games" thing make you nervous?

It totally should, so let's discuss it!

First, the caveat: games that aren't in the "good" category are not necessarily bad. Most of them are unrated, meaning they have so few ratings that the average is statistically meaningless.

When you saw Figure 5, with the dramatic rise in indie game releases (left plot) compared to the static rate of good game releases (right plot), you probably wondered what the hell was going on there. Surely, as more games enter the market there must be more good ones, right? What are the possible explanations for the stagnant release rate of well-rated indie games?

There are many possible reasons. Below are two, along with their consequences:

This could be due to marketing (e.g. no matter how many good games there are, there is a Steam bottleneck that allows only a handful to be successful), meaning that players only are aware of and/or rate a small number of newly released games. In the mobile markets it is well established that being on the front page of an app store, or having a huge advertising budget, are the only two ways to success. Indies typically only have access to the former, and the front page shows a very limited number of games. Built-in Steam discoverability may similarly limit indie success to this handful of games per month. In that case, the Indiepocalypse already happened years ago.

Another possibility is that the barrier to entry to the Steam marketplace wasn't a barrier to games that could be successful. As the barriers continued to fall, potentially-successful games came in just as they used to, but then so did more and more games that did not have potential for success (whether due to issues of quality, originality, marketing, or small-niche interest). In that case the Indiepocalypse, as an idea, simply doesn't make sense: of course you can't be successful if you aren't making a product people want! There is no evidence that success of "good" games is being impaced by the influx of all the other games.

If by "Indiepocalypse" you mean "a larger and larger fraction of new indie devs will fail," then of course you're right. But that has always been true, so it doesn't seem to warrant a freakout. In that case, things are exactly the same as they were: the market wants the same amount of indie games as it always did, and you can be successful by making games that players want.

Probability of success is not defined by how many other devs there are. It's defined by how good your game is and how well you market it. Most games fail. But most indie games are some combination of unpolished, unoriginal, poorly priced, and poorly marketed. So that isn't surprising in the least. As the barriers to making games go down, the barriers to successfully selling games stay right where they are.
 

As for the original question: how does release month affect indie success?