UX & QA Testing: What's the Difference?

Hi there! My name is Lindsay. I'm the Director of The Research Centaur, the UX+QA divison of The Behemoth. We made Castle Crashers, BattleBlock Theater, and your favorite capsule machines at San Diego Comic-con.  

I get into situations like this a lot: I'm walking the show floor at E3 or PAX, or wherever, and I meet somebody awesome (woo!). We get to talking about what we do -- they make games, I make games and then help test them. I mention I have a background in both user experience (UX) and quality assurance (QA) testing, and they think for a minute.

Then they ask, "What's the difference?"

Well... I'm glad you asked.

TESTING IN VIDEO GAMES

There are so many things developers can do to make a great game more amazing, and testing is a critical piece of that process.

The goal of testing is two-fold, in my mind:

And I'd argue that you need both user experience and QA testing to help you get there.

USER EXPERIENCE TESTING

So what is user experience testing, exactly? If you ask a dozen user researchers that question, you'll get a dozen answers (and hear two dozen names for various types of tests).

Ultimately, though, UX testing is about examining real player reactions to and experiences with an in-development game.

At The Research Centaur, we do this in a variety of ways. Most commonly, we'll bring a group of gamers into our lab and ask them to play a specific part of a prototype game they've never seen before.

Then, we watch what they do and ask the right questions to help figure out whether the gamers are understanding the mechanics and game flow (or, failing that, at least enjoying themselves and reacting in a way the developers like).  If the developers are unpleasantly surprised by the player experience, the researchers running the test will also provide insight into and suggestions on how to change the game for the better.

This type of testing is great at finding out how real players will experience the game once it's released, and what they'll do. It's not so great at finding crashes, graphical problems, or other bugs -- which is where the quality assurance (QA) piece of the puzzle comes in.

QUALITY ASSURANCE TESTING

Quality Assurance is what most developers think about when they hear the word, "testing."

It's the process of systematically walking the game software through as many features and behaviors as possible, and seeing what it does. This helps catch critical bugs before the players ever see them.

The trick to good QA testing is doing it in a smart, risk-based way that doesn't cost too much -- but maybe I'll talk about that more in a future blog post.

HOW THEY WORK TOGETHER

So, to summarize! User experience testing:

And QA testing:

Together, these two test disciplines create a better game. Because both bugs and misunderstandings can keep a player from having the best possible experience with a game, I'd argue that they are both equally critical to a fantastic launch.

Do you have any questions about user experience or quality assurance testing? Please let me know in the comments!