Usability Evaluation For Video Games: Gulfs Of Execution, Evaluation

[In this reprinted #altdevblogaday in-depth piece, European University Cyprus assistant professor Georgios Christou begints to look at how to evaluate the usability of a video game.] A lot of the readers of #altdevblogaday are students, young developers and hobbyists, so I've decided to write down a few things about usability evaluation methods for games. I feel that very little is out there about how to evaluate the usability of a video game, and it seems that most people do this experientially, or not at all. It is also true that the interface is the only way that a game designer/developer has a real dialogue with players, so if the user interface sucks, players are not going to play a game for long. Of course, there are exceptions (take Skyrim, for example) where an atrocious interface is beat to submission by players because the game's other features (storyline, progression, etc.) are so good. But let's face it, that's rare! I'm going to break this topic into a series of posts because of the sheer amount of information that exists about it. I hope that you'll find the series interesting! Seven Stages of Action To understand how to evaluate a video game, first one needs to understand users. More specifically, to understand how users work with a user interface, what they expect, and how they react to the unexpected. So first, let me present Donald Norman's 'Seven Stages of Action' model, which is described in his excellent book 'The Design of Everyday Things' (Norman, 2002). I highly suggest this book to all my students, as well as to all people who are involved with designing anything! Norman proposes that when users interact with a user interface, they go through the following stages (Norman, 2001, p. 48):

• Forming the goal
• Forming the intention
• Specifying an action
• Executing the action
• Perceiving the state of the world
• Interpreting the state of the world
• Evaluating the outcome

Gulf of Execution The stages above describe the way that users conceive what they want to do, through forming a goal in their minds. Then they form their intentions as decisions to apply some actions that will bring about the completion of their goal. Here lies the first pitfall in designing user interfaces. The users know what actions they want to perform, but they do not know how to execute this. This is called the gulf of execution. A gulf that users must cross if they want to turn their intentions into a specific sequence of actions that are allowed by the user interface given to them so that they can reach their conceived goal. An example can be taken from text adventure games (for those who are "young" enough to have played them!), and the way these games handled interaction with their players. Once players entered the world of the game, they needed to know the commands with which to move around and execute actions. That's why manuals were so important. Otherwise, the players just read a textual description of the starting room of the game, and saw the prompt staring blankly back at them. Thus, they needed external help to turn their intentions into action sequences that the game understood. This, together with a very small vocabulary that text adventures understood, created problems of immersion. And that is why point-and-click adventure games later on had such an impact. They gave players a more natural way to interact with the game world, without requiring them to know exactly how to phrase an action so that the game would understand what the players wanted to do. Gulf of Evaluation This whole process talks about how users affect change on the game world. Once the change is affected, users require feedback to understand what they have actually done. Without feedback, there is no reason to go into the second part of the seven stages of action model. However, even the feedback that players receive presents problems. Once players have performed their action sequence, they perceive the new state of the world. And here is where the second pitfall lies. Between perception of the state of the world and the interpretation of this perception lies the gulf of evaluation. This gulf is the ease (or the difficulty) with which users can correctly surmise what has happened in the world. Again I'll use text adventures as the example. Players could type their commands, but until they actually told the game to "look around," they did not know whether their affected change in the game state was the one that achieved their goal. This was because text adventure games (sometimes) did not provide a description of the new room the players entered until the players actually looked at the new room and its contents. Once the players looked around, then they could evaluate the outcome of their actions and see how close they were to accomplishing their goal of moving to a target room in the game. Again, graphic adventures made the gulf of evaluation smaller by presenting graphical versions of the world, where the players could see their surroundings and realize that they had passed into a different state in the game world. Application The understanding of the gulfs of execution and evaluation arm the game designer and the game developer with some important tools. The problems that are created by a gulf of execution are mostly due to actions that don't map well to the game world. For example, World of Warcraft players were rewarded if they started playing Star Wars: The Old Republic through a very familiar user interface, and a control set that almost matches that of World of Warcraft out of the box. Thus, mastering the interface is not a challenge that needs to be overcome by the player, who would probably prefer to master the gameplay challenges instead. The concept of the gulf of execution meshes nicely with user interface standards. Although I am not a proponent of standardizing user interfaces , it is nice to have some knowledge transfer, at least of the controls in games that belong to the same genre (think about the 'WASD' control keys, for example). Remember here that one of the goals when designing a video game is to challenge the players as they gain further mastery of the game, not of the user interface of the game. And if your user interface is one that is entirely new, then you should allow players to acclimate themselves and breach the gulf of execution before creating serious gameplay challenges. On the gulf of evaluation side of things now, there are quite a few things that we can use to create feedback that is not obtrusive. While it's great to show players of, say, FPS games that they are getting shot in the back, there are subtler ways than printing it on the screen in front of them! Some games flash a red border to the players, and others provide not only visual but sound feedback as well. However, again, the primary goal should be to make this feedback understandable quickly. Think about how many mods have been created for World of Warcraft, designed to make the life of healers easier during raid battles. Why is that? Because one of the primary difficulties in WoW's interface was that it didn't allow immediate understanding of the health of all the raid members. As the developers saw and listened to their users however, the interface became more and more pliable, although not entirely to the point that healers are throwing away their Whammys and their HealBots. Conclusion In conclusion, the gulfs of execution and evaluation are probably two of the most widely used and widely applied concepts in usability evaluation. So next time you start your game design, maybe you'll think about how to get your players to cross those two gulfs in an easier manner! This blog post is the first in a series of posts about how to use usability evaluation principles for your games. If you like this, let me know so that I continue writing on the subject. You can also let me know if you are interested in a specific aspect of usability evaluation, and I'll do my best to either explain, or to study up and let you know what I find. References Norman, D. (2002). The Design of Everyday Things. Basic Books. [This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]