informa
3 min read
article

Test your icon!

You probably heard this already - app icon is super important. I'll share a story on how we learned it the hard way and a piece of advice on how to A/B test completely for free.

Lately we released a Halloween update Kickerinho. It included 7 themed costumes: a zombie, mummy, skeleton etc. Not a very big update, but it added some new, fresh content. You can check out the promo trailer here to get a better idea.

We were quite surprised to see that the update had no positive effect on the download rate. Quite the opposite actually, we noted a substantial decline.

You need to know that we have a tradition of changing the game icon with content updates. It’s a cool strategy that many publishers incorporate. This helps our users notice that something new has arrived. It’s an impulse that could potentially retain churned players. Seems reasonable, right? What could go wrong?

We did some investigating and came to the conclusion that the icon we chose for the Halloween update was to blame. Why? Take a look at the downloads graph:

During the Halloween update we had 42% less downloads (compared to the corresponding period). Once the Halloween had ended we reverted the icon to the previous one (marked on the graph). As you can see, shortly after that, the download rate increased and went back to almost the same levels as before.

Those were the two icons that made such a difference:

Lesson learned? A/B  test your icon. Always. No excuses.

Happily - it's very easy to do so thanks to a feature introduced in Google Play Developer console: the experiments. Google lets you show a test icon to a portion of traffic visiting your store page and then compares the conversion rates. You can also test description, screenshots, promo graphics. Pretty cool, no?

We used it to test our winter icon. Those were the icons we wanted to test:

And the results:

Our "gut feeling" went towards the version A, but as it appeared all of them were worse than the "default" one. It was also clear that icons with sunglasses were better than those without them. So we made another test.

D was similar to B - which was the winner of previous test, but with a different scarf color. E on the other hand had the scarf removed, because the current (at that time) version was also without a scarf and it performed better than any of the three test icons.

The results?

Version E appeared to be the winner. Do note, that we used a smaller sample this time (hence the ranges are wider compared to previous test) but despite that it was clear which version should be chosen.

I encourage you to A/B test your icons. You can never be sure what your users will like and gut feelings can point you in the wrong directions. Ignore it, like we did with the Halloween update, and you could face some severe consequences. Since it's so easy to A/B test using the experiments feature, there is no excuse not to do so!

Latest Jobs

Treyarch

Playa Vista, California
6.20.22
Audio Engineer

Digital Extremes

London, Ontario, Canada
6.20.22
Communications Director

High Moon Studios

Carlsbad, California
6.20.22
Senior Producer

Build a Rocket Boy Games

Edinburgh, Scotland
6.20.22
Lead UI Programmer
More Jobs   

CONNECT WITH US

Register for a
Subscribe to
Follow us

Game Developer Account

Game Developer Newsletter

@gamedevdotcom

Register for a

Game Developer Account

Gain full access to resources (events, white paper, webinars, reports, etc)
Single sign-on to all Informa products

Register
Subscribe to

Game Developer Newsletter

Get daily Game Developer top stories every morning straight into your inbox

Subscribe
Follow us

@gamedevdotcom

Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more