How To Not Lose Money On External QA - And Have The Best Productivity

This blog post is a bit of an addition to a post by Anna Jenelius, and I suggest that you really should read it first. The post is also a bit of musing on topics that disturbed me when I was working as a QA tester. Obviously, no connections should be made between my personal opinions and my former employer's, as they are completely separate.

I have been working for almost 3 years in an environment which is hardly comparable to the one described by Anna. It was an external QA, located in Poland, which was a place for developers to outsource purely "bug-hunting" tasks. We had different sorts of clients, ranging from the biggest players to the small indie studios. All of them made common mistakes - some of which are easily avoided. So, if you want to outsource your QA, get the job done, and not leave some people with an impression that you are a terrible company, follow my advice. 

1. Do not be late with your builds.

I cannot stress this one enough. If you say that the build will be ready for Monday - it should be ready for Monday, 8AM in the morning. Because your testers will not test the build, when the next one is due in a few hours or a day. Nobody wants to reproduce all freshly found bugs a few hours later, to make sure that they will not go to trash immediately. Instead of that, people lose focus - they explore your game, wander around, but do nothing productive. This is losing time and money.

2. If you're in alpha - add things!

This is especially important when you are in the early phases of the development. Your game is boring and ugly, and people who test it are staring at the grey screens filled with geometry for 8 hours daily. And there are no features added properly. And there is barely anything to test, because the system is still so simple, that all interactions can be explored really fast. And what happens then is that testers are losing focus - and you lose productivity and money. The solution is adding new content to the game, bit by bit. Testers love new content. If you are in alpha - add anything, but add something. They will test it thoroughly and creatively. Leave them for a month with builds that are exactly the same, and you will lose them.

3. Fix the bugs. Really, just fix them.

I am not sure if this should not be the first. Imagine a situation, when you work for one week, producing reports. Then the second one. And the third. And you reported a hundred of bugs, including some of big importance. But they are stuck as "in progress", somewhere in the great, deep void of "that damn dev team". This usually happens when you use lab's bugtracker and your own - somebody just pastes the bugs into your tracker and they get processed there. But the testing team does not see that. And at some point, suddenly, nobody cares. Because if you, as a developer, do not care about testers' work - they won't care about yours. 

4. Maintain your bugtracker.

That’s another “obvious” one. If you have a properly configured bugtracker, it does everything by itself, right? But apparently that does not happen so easily. If you work with an external QA, please have some things prepared. Summary standard, what exactly should go into the description, proper version control present in the bugtracker. Also – react as fast as you can to the reports. They will have flaws, and if you do not get rid of them quickly, they will be reproduced in another reports. That’s because for testers every project is different, and sometimes they switch projects 4 times a week. Things can get confusing, and if you do not remember that, you will have a huge mess. And lots of duplicates.

5. Keep testers informed.

This is probably the most overlooked one. To have typical bugs found, you do not have to do this. Game design doc will not change anything in terms on finding flickering textures. And your internal QA will handle the weapon’s balance just fine. But external QA always has the drive to ask questions and post suggestions. And there is nothing raising morale in a team better than an e-mail addressing even the weirdest of concerns, or a thoughtful comment on a suggestion, even if it is rejected. These are the easiest ways to keep in touch with your remote team, and to connect with them in a way that will make them involved and caring. And if they care – you get a superb QA.

Conclusion

All in all - external QAs are the trenches, and we all know about it. But if you, as a developer, change some things, you can change your relationship with your testers dramatically. And believe me that they will work hard to test your game, given that you show that you want them to. Because testers are usually intelligent and passionate people - most of them could be somewhere else, doing other jobs, earning more cash in better conditions. But they love playing and testing, and exploring the boundaries of your systems. Just let them do that more easily.

@radsmyk