When is 'Pay-to-win' too much?

Currently the most popular model for mobile apps on the market is a Free-to-play model monetised with in-app purchases. However in-app purchases can be a tricky proposition. Your users are only going to continue playing the game if they find it fun or rewarding in some way, so over monetising your game can drive away players before they can contribute to the churn that keeps your power users happy. But at the same time; while mobile games are a creative endeavour, they are also a business. As such you want to get the optimum amount of revenue possible. This leaves us with the question: ‘When does it become too much?’

The First step to finding the answer is to look at a cross section of the mobile games market. These are all games that have in-app purchases in some form or another. They have also received varied receptions from their user bases. Using our tool, Player XP, I am able to find out how many reviews actually offer some kind of constructive feedback. I can then quickly find out how many of those express a negative sentiment. I then use that as a baseline to measure against the number of reviews that offer constructive feedback about the game’s monetisation, and how many of those also express a negative sentiment. Currently my sentiment is being derived using the review star system (negative sentiment being 1 or 2 star reviews). In the coming months, we will integrate a more sophisticated sentiment system into Player XP. This will provide even more information and granularity than the current star system.

Below is the tabulated overview of data I uncovered using Player XP:

It shows the number of constructive reviews, the number that are negative, the number that talk about monetisation, and the number that are both negative and talk about monetisation. For your convenience, there is also a column showing the total number of reviews we have processed. Analysing this data reveals some interesting points:

Dungeon Keeper

To put these figures in perspective: 43% of their reviews are constructive, Of those reviews, 67% are negative. Monetisation is talked about 52% of the time. 61% of negative reviews talk about the game’s monetisation.

This flags a large problem with the way the game handled monetisation, while Dungeon Keeper Mobile (DKM) is not very well received anyway (67% negative), negative sentiment increases substantially by 13% when looking at monetisation. There are a number of possible contributing factors that include: The price/value exchange of in-app purchases, negativity from the Dungeon Keeper Franchise’s “hardcore” fan base, and negative press coinciding with DKM’s release. If you then look at the reviews themselves, you begin to see a pattern emerging. Below are a few examples of what I mean:

Candy Crush Saga

To put this chart in perspective: 20% of their reviews are helpful, Of those reviews, 42% are negative. Monetisation is talked about 33% of the time. 38% of negative reviews are about the game’s monetisation.

Candy Crush Saga is a highly successful game for King. But 42% is quite a significant percentage of people who dislike the game. However upon further inspection 39% of reviews are also distinctly positive. This shows that Candy Crush is highly polarising. However of all the talking points discussed in player reviews, Monetisation is the most contentious with an 8% increase in negativity when compared the games overall sentiment. This is not insignificant, but can be partially attributed to people being more negative about monetisation in general. When looking at the review content you start to see that the users feel the game is too expensive. Below are some examples:

Clash of Clans

To put this chart in perspective: 30% of their reviews are helpful, of those reviews, 28% are negative. Monetisation is talked about 29% of the time. 34% of negative reviews are about the game’s monetisation.

We already know that Clash of Clans is a highly successful game for Supercell. Much like candy Crush Saga, monetisation negativity increases for Clash of Clans by 8%. What is interesting is that there are a lot of players complaining about the monetisation who feel frustrated with how the game feels like they’re being punished for not paying. Below are a few examples:

Real Racing

For Real Racing 2, 31% of their reviews are helpful. Of those reviews, 8% are negative. Monetisation is talked about 29% of the time. When talked about, it is referred to negatively 10% of the time. 35% of negative reviews are about the game’s monetisation.

For Real Racing 3, 36% of their reviews are helpful. Of those reviews, 35% are negative. Monetisation is talked about 32% of the time. When talked about, it is referred to negatively 50% of the time. 45% of negative reviews are about the game’s monetisation.

What is interesting here is that these two games are in the same franchise. This gives us a good look at what a change in tactics may do for your game. Real Racing 2 is game that you must purchase with a price tag of £3.99 (£4.99 on iPad), however it still contains in-app purchases. On the other hand, Real Racing 3 has a more conventional Freemium model with in-app purchases. And what we’ve found is that Real Racing 2 has a player base that appears to be smaller than Real Racing 3. However users are far less vocal about the way the game is monetised.

There are a number of possible explanations for this, the first being that Real Racing 2 has a smaller but more dedicated user base. As such, they are more willing to both buy the game and pay for in-app purchases. Another possible explanation is that with going completely freemium, changes to the monetisation model have made the game appear to be of less value per dollar or pound spent. If we begin to dive into the monetisation reviews themselves, you can see a significant difference between the ways the games are played. For Instance when looking at monetisation for Real Racing 2 a typical Review will look like something like this:

On the other hand when you begin to look at Real Racing 3 reviews that mention monetisation you begin to get a different picture:

To Conclude

So overall, Some of the most interesting insights are those related to the Real Racing Franchise. Real Racing 2 proves that if your pricing model is good, players will notice and mention it in their review of the app. These figures also go towards disproving the idea that you can only do in-app purchases if you make the game free. If you’re targeting a smaller but more dedicated group of players, this may in-fact be a superior model for you. Unsurprisingly there were complaints about the costs of both Clash of Clans and Candy Crush Saga. However, the games’ being otherwise well made and fun give the developers a lot of good faith with their users. On the other hand, Dungeon Keeper shows what happens if you take a beloved franchise and tool it towards what a large portion of it’s core audience considers a “money grab”. The game had already failed before anyone had ever gotten their hands on it because of the stigma of freemium amongst many of the “old guard”.

All this data I have collected and analysed has come from our tool Player XP. Player XP is a review analysis and review aggregation platform for iOS games that has been produced by Hertzian Ltd. As the CIO of Hertzian Ltd I was heavily involved in the production of Player XP, and I was still wowed by our tool when I began to use it for my report. I would urge anyone who is interested in what I have produced here to check out the platform over atwww.playerxp.io. This analysis and write-up was created over the span of two days, and I’d love to see what other people could produce with the tool.