Rebranding Beta Testing with Baldur’s Gate: Enhanced Edition

Imagine the scenario: a massive repository of bug fixes and new features, a hungry community waiting for a patch, and limited time and resources for thorough testing. How do you make sure the patch is stable before releasing it?

That’s what we were facing with the 1.2 patch for Baldur’s Gate: Enhanced Edition. Well over six-hundred changes needed testing, and after a summer spent in development limbo our players were beginning to wonder if the game would ever see another update. We needed a way to get those people involved, we needed to do the kind of intensive testing that was necessary but beyond our limited resources as a small company, and we needed to make it fun for everyone.

We started by calling it an Open Beta about two weeks before we launched the event. We were surprised to see a great deal of negativity toward the term, which is understandable; if you’ve been waiting for a patch to fix a major bug, the last thing you want to see is a months-long testing process before it’s released, and that’s what people think of when they hear about an Open Beta. So okay, we went back and looked for a better way to present it.

In the end we settled on The Road to v1.2. We wanted to make it clear that this wasn't just any beta. We wanted to tell people, This is the light at the end of the tunnel, this is what we’re working toward, and this is your chance to be a part of it.

The community jumped on board immediately. We were amazed by the enthusiasm and precision of the feedback we received. On our forums, people talked about The Road to v1.2 as though it were an event, not just a beta test. It was great to see the community get involved so readily.

The Continue button at the bottom was recently added to dialogue selection for tablets, based on feedback from the community.

In the beginning we asked people to send us saved games to make sure they would still work with the new patch; as we moved into later stages of the event, we invited people to take a more active role. We opened up access to the beta build of the game to a small number of existing customers, and gradually expanded that pool of testers as we hit each milestone. By the end of the event, anyone who had bought the game through our direct channels could get involved if they wanted to.

When you open a game up for beta testing, if you’re not careful you might get inundated with feature requests that will only serve to delay the eventual release. So for us, it was important to narrow the focus of our testing. We created a series of phases to let us keep our testing parameters limited to what we needed to know. In the Hardware Compatibility phase, for instance, we weren’t interested as much in knowing that it would be cool to play as a drow elf, as much as we wanted to know whether the game was slowing down significantly at any point for people. By asking our players to focus on one or two areas of the game, we were able to get a lot of really helpful information without too much extraneous data.

What’s more, we were able to communicate our progress in a way that showed people how far we had come and how much work still needed to be done. A roadmap is made up of milestones, and we wanted to make sure we made those milestones public. For The Road to v1.2, our first phase of testing was “Saved Game Compatibility”; we wanted to make sure that our new patch wouldn’t break any of the saved games that people already had. As we tested them, we tallied our progress on a tracker, so that people could see that if we had tested two-hundred saves and our goal was a thousand, we needed to test eight-hundred more saved games in order to move on to the next phase. Our players found it to be a much more reliable method of estimating progress, instead of waiting for someone to post something on Twitter.

The results were overwhelmingly positive. As I said before, people referred to The Road to v1.2 as a kind of online event, lasting several weeks and ending with a patch that people were excited about. On our official forums, I still see people referring back to it occasionally, which isn’t something that happens very often for a beta test.

Right now we're wrapping up a second event, The Road to Android, which focuses on our upcoming Android release. It’s a little bit different releasing a new platform versus a large patch, but at the heart of the Road to ‘X’ is the search for strong feedback and clear milestones. It helps that our community is especially dedicated when it comes to providing that feedback; each time we start an event like this I know that the games are about to take a giant leap forward in quality.