Torque's Eric Preisz and two Full Sail usability center PhDs offer four techniques to help make your game more accessible -- even if you don't have access to a giant lab and dozens of focus testers for feedback.
[Torque's Eric Preisz and two Full Sail usability center PhDs offer four techniques to help make your game more accessible -- even if you don't have access to a giant lab and dozens of focus testers for feedback.]
Usability is a crucial factor when attempting to reach large, diverse audiences with a game. A commonly stated rule-of-thumb is that if your player doesn't understand the basics of your interface in two minutes, they'll stop playing your game. Therefore, good usability is critical to game success.
When it comes to designing usable games, it is critical not only to understand how your target audience experiences your game, but also how other groups of individuals not in your target audience will react to a product (professionals call these "core" and "fringe" users).
For financial and quality reasons, increasingly usability work on games is being done by specialized user experience centers with broad access to large diverse groups of potential users and whose team members have extensive training and experience in usability assessment methods.
This is a rare combination of factors, however, and means that such user experience centers are typically located in and around universities, and at the few large game companies that can afford an in-house usability center.
Although this makes it harder for many game studios to get access to these groups, user research teams can pay for themselves quickly in several ways including: keeping a development team working together for a common achievable and prioritized set of goals, alleviating investor concerns about customer targeting and focus, improving the efficiency and effectiveness of development tools, and most importantly, improving user experience within a game environment.
An early focus on usability on game projects is financially critical -- research shows that the top four reasons for development going over budget are all related to unforeseen usability problems and 80 percent of the cost of patching and maintenance on software products is due to unmet or unforeseen user interface needs.
Unfortunately, cost, location, and personnel availability often make it difficult for small to midsize studios to set up or even get access to user experience centers. While not every game company can afford the services of a professional user research team, every game can benefit from the basic techniques these groups use. Many of these techniques are simple and cheap enough to be implemented by developers and artists directly.
In this article, we'll introduce four usability ideas, a simple methodology, and discuss pros and cons for each. These concepts and methods can be understood and implemented without specialized knowledge, and can pay dividends both during development and after a game ships.
Methodology. In the think aloud technique, the gamer sits down to play the video game while a user experience team member is seated nearby to listen and take notes. The user is given specific instructions that as they play the game, they are to say out loud the reason they took each action. This allows the team member to document both their actions, and what the user was thinking as they took them. The team repeats this with multiple gamers to get multiple perspectives and viewpoints.
Example. During a recent test, a player chose to try to load a gun with a water bottle by dragging and dropping the icon for the water bottle (a blue-gray tube with a narrow end) onto the icon for the weapon, saying "this looks like a bullet, so it's probably ammo for the gun." Their actions, and their spoken-aloud explanation, made it clear that the icon for the water bottle was confusing and needed to be redesigned.
Team members are able to understand what a player is thinking. This allows them to find and document areas of the game that caused problems, often with multiple players.
This technique works well in an iterative design and testing process. Research has shown that 75 percent of interface design problems can be discovered using a handful of participants (around five). With an outcome that fast, results can be generated in a single day and passed on to the development team for fixes.
Other team members (like developers, artists, and producers) can watch and immediately see firsthand when there are problems with the game. This saves time and helps them to understand why the fixes are important. Allowing developers to see a product in action with players is extremely powerful when the player makes a mistake or takes an action that developers never took into account.
Many players have no problem talking while they play, but some will have problems with this. Uncooperative players (intentional or not) can cause frustration and lost time.
"Thinking aloud" doesn't come naturally to players. In a natural setting, gamers rarely verbalize all of things they are doing in a game. It's also possible that in trying to explain what they're doing, they may make up reasons that are different from the actual cause of a behavior.
If it's hard for some people to walk and chew gum at the same time, you can imagine that it's hard for many people to play a game well and talk about it at the same time. Often they focus on one or the other, with less than ideal results. You can try to fix this by asking them what they're doing or thinking if they stop talking at some point.
Methodology. There are several published sets of "heuristics," or rules of thumb, which provide guidelines for what makes a good game. Jakob Nielsen has published a very solid and well-regarded set of heuristics for software interfaces in general (you can read more about them here) but some others have come up with their own published lists.
One set aimed specifically at adapting Nielsen's heuristics to games was developed by Melissa Federoff -- the full list can be found at the author's site. If you don't like any of the published lists, however, you can develop your own set specific to your game.
First, ask a set of game experts and game players (devs and beta-testers will work in a pinch) to identify what features are necessary for a game in your game's genre to be successful (in regards to story, game mechanics, game play, etc).
Then, while the expert or participant plays the game, have other experts observe and ask them to judge the intuitiveness and effectiveness of the game interface based on the list of important features you generated earlier.
Remember that these are specific to the kind of game you're making -- you would need to come up with a different list of features for first person shooters than for MMORPGs.
Example. Nielsen's heuristics state that if any process takes longer than 10 seconds (a loading screen, for instance) you should make sure that the user is getting some kind of feedback that the system is still working (an animation, or a loading bar, for instance). This helps players to understand not only how close the game is to being ready to be played, but also if something has gone wrong (a crash or lockup).
You can either rely on the existing work of experts, or identify your own experts to analyze similar games to the one you are creating and develop your own list of important features and design elements your players will expect. This can help prevent your user base from immediately going hostile when you launch because of obvious lapses in gameplay, mechanics, and story elements usually present in your genre.
Cost is relatively low because you are doing the evaluation yourself or with a few hired experts instead of having to bring in and test lots and lots of potential users.
This is very effective when used together with other, more expensive and time-consuming techniques -- the heuristic analysis identifies the big, easy to find problems so that later techniques can focus on harder to find, but still critical problems.
Heuristics themselves may be simple, but applying them well requires an expert's eye for detail and a lot of experience with interface assessment. If your team is relatively inexperienced, this can be a problem.
Using members of your team to evaluate your interface can be problematic -- after all, they already know how users are "supposed" to use the controls and interpret the HUD. Ideally, you want somebody who knows about games and usability, but isn't familiar with your game specifically.
Finding people who fit the bill can be a problem. This is doubly problematic when developing your own heuristics, as now you need more experts to generate the heuristics in the first place. When you don't have experts in-house, one way to deal with this in multiple-project studios is to have the dev team from one game work as the usability team on another game within the studio.
It can be hard to judge the heuristic statements that are created. For example, it may seem obvious that first person shooters should include a mini-map that allows the player to easily identify the orientation and placement of their avatar within the environment, but how do we assess this? Is this statement answered by a yes or no, a ten point sliding scale (called a Likert scale by professionals) or a subjective response from an expert? Worse, experts can disagree on how to answer the question or even on how to judge the question.
Sticking closely to your own heuristics can lead to limitations on game creativity. Do all gamers really want all MMORPGs to have the exact same game elements, mechanics, and story processes?
Methodology. Team members gather together a small group of potential game players to discuss their opinions on the design of the interface, as well as game mechanics and story. Team members develop a flexible script of a dozen or so important questions that need to be answered. A moderator then guides a discussion in which participants are asked to discuss features or aspects of the game in turn or as part of an open discussion. Focus groups are often used to review game concepts or prototypes before they are put into a full development phase.
Example. A group of six to 10 users can, over the course of a couple of hours of discussion, often identify most of the aspects of an interface that will actively annoy or upset most players. This not only improves the game's quality and reception, but may also provide valuable insight into how (and to whom) to market the game.
The basic principle here is that several minds are better than one. Focus groups allow for a sharing of ideas amongst participants. New ideas can also be generated from this sharing of ideas and opinions, as can design strategies leading to more efficient and targeted gameplay, mechanics, and story.
All the feedback from participants is collected all at once over a couple of hours. In an iterative design or decision making process this means you can collect a lot of data very fast, allowing for follow-on focus groups the next day or next week to assess new concepts or even new prototypes.
Designers, developers and producers can observe the focus group directly or through recorded video and audio feeds. Often the feedback gained by the team is very powerful in nature -- "I can't believe they hated that one feature -- we thought that was really cool!" Moreover, when designers, developers, and producers can see the reaction of their audience directly this tends to eliminate the need for time consuming analysis and report-writing.
This technique works best with an experienced moderator. In a pinch, a team member with good listening skills will work (producers are often good at this) but they must be very careful not to push the discussion toward ideas or answers that they prefer, or the focus group is useless. Leading questions from the researcher can manipulate the participants into answering questions a certain way, which reduces the usefulness of your findings.
Several minds can easily be lead by one ego. If one participant has strong opinions and expresses them loudly and confidently, other participants in the focus group may feel pressured to agree with the dominant member. When this happens, much of the value of the group is lost. Remember that the loudest or most talkative participant does not always have the best ideas.
It's important that all the participants feel that their opinions are equally valued. Very unequal levels of skill or experience can lead to participants deferring to the member who as logged the most hours in similar games. Often it is helpful to have groups made up entirely of beginners or experts. You may want to screen participants ahead of time to help with this.
Methodology. Team members observe users playing similar games in the environments where they are generally used. Game tournaments, game LAN centers, and internet cafés allow for this type of observation, as do some specialized user experience play center labs. Like biologists observing animals in the wild, team members try to observe gamers interacting with the game as unobtrusively as possible.
This technique is used in the scientific fields of Anthropology, Sociology, and Social Psychology, but it works very well in development settings as well, assuming you have the time and resources to do this yourself.
Example. While engaged in the assessment of the front-end menu for a new game, team members observed that users spent a lot of time clicking on a particularly detailed and cool-looking piece of game art that was used as background decoration. Team members quickly realized that players thought the decoration was a control for the menu system, and were confused when clicking on it did nothing. When the position and format of the decoration were changed, the problem went away.
During traditional playtesting, gamers often react to the sterile lab and testing environment and behave differently as a result, answering questions and playing the game differently from how they would in a more comfortable setting. The use of naturalistic observation techniques can help to overcome this effect.
With this technique, large amounts of information can be collected in a very short period of time.
By carefully observing how users interact with the game, team members can often identify unexpected and critical information about how users perceive a game, as in the example above.
Team members may not understand why a user does something because they cannot get inside their head to know what the user is thinking.
Sometimes, the environment where observation is occurring may have an impact on how users behave. If one of the goals of a game tournament is to win as fast as possible and rack up a large number of victories without worrying about losses this could cause team members to observe play behaviors that would not occur during normal gameplay.
It can be difficult to see and document exactly what users are doing, especially detail-oriented behaviors like choosing an icon or manipulating a slider bar.
Once you have usability information to share with the dev team, there are a few things that might help you to present it so it has maximum impact. Remember that every feature is somebody's baby, and nobody wants to hear that their baby is ugly. If you're too critical of a feature or mechanic of a game, you may find that the devs responsible for implementing fixes to the problems you found are uncooperative or unwilling to make the changes needed.
Instead of doing a big presentation, it's often better to write up a short report on what you've found, distribute it to the dev team, and then ask for questions. Criticizing a feature in front of the whole development team may cause those responsible for it to become defensive and uncooperative. Writing up a short report (or even a bulleted list with examples) allows them to look it over on their own without pressure and ask any questions they have without feeling that they're being criticized or "put on the spot."
When you report your findings, the difference between getting cooperation and opposition from a dev team member can be as simple as the words you use, or the way you explain the problem.
Using negative words like "broken," "problem," "unusable," "confusing," or "odd" can upset the devs responsible for those features and lead to a lack of cooperation or even long drawn-out arguments.
It's often better to talk about features that "need attention" or "could be improved." Silly as it sounds, poor word choices can lead to internal conflict on teams, and conflict can lead to delays in the development process, costing the team time and money.
Do yourself a favor and pick your words carefully when talking to developers or artists about usability issues -- after all, it isn't their fault that they're not the target audience. Every game has usability issues at first, and finding them is just another part of the development process.
When describing usability issues, it's almost always better to "show" the problem rather than "tell" about it. If you have video of users making a mistake, that can be the best possible way to get a developer to recognize a problem and get behind fixing it.
During a project involving a major appliance manufacturer, a confusing interface on a refrigerator often led to test users having problems using the ice maker. The engineers involved in the product were skeptical of the report and blamed the user experience team for exaggerating the problem -- until they saw a video of the reactions of several actual users dumping ice down their shirts by accident while using the device.
Even if you can't get video of a problem, a detailed description of each of the cases where the issue occurred is more compelling than a general statement that the problem exists.
Finally, if you can calculate some basic statistics on problem rates, that can also help emphasize the importance of an issue. Saying "40 percent of our users tried to load the water bottle into the rocket launcher when they were asked to find the ammo for it" has a lot more impact than "some users confuse the water bottle with the rocket launcher ammo."
This article is intended to help development teams without access to user experience professionals apply some basic usability assessment techniques to improve both their development process and the quality of their finished games. The techniques discussed in this article can be productively employed during preproduction, development, system test, after game release and beyond. In fact, typically the earlier they are applied, the better the result.
The difference in cost between identifying usability problems during preproduction versus during alpha or beta can be huge, with some estimates ranging as high as 100x the cost to fix usability problems identified post-release as during preproduction. Early and effective usability testing can not only improve the quality of the final project, but can also result in real and significant savings by the studio throughout the development process.
Deciding which usability techniques to use can be a challenge. As in most aspects of game development, the amount of available time and the cost of procedures are often the deciding factor in which (and how many) usability techniques are applied to the development process. In addition to the limited number of techniques presented here all usability techniques have strengths and weaknesses that should be evaluated before a single approach is decided on. Often a combination of techniques yields the best results.
Development team members (developers, artists, and designers) taking on the role of usability testers should remember a few important points. The goal of this process is to understand the system from the perspective of the player.
This means team members must be willing to step outside their usual role and take their ego out of the equation. Often users will have trouble using features that seem intuitive to developers, or may dislike features of the game that developers are personally invested in. This type of player reaction may be upsetting or irritating to development team members.
In these cases, it's tempting to blame the player or attribute their confusion to stupidity or laziness. Developers must try to remember that no matter how much they like playing and building games, unless they are building a game intended only be played by game developers, they are not the same type of people as the intended user of their product.
The knowledge and attitudes that make them good developers also make them different from the majority of game players, who typically do not think as developers think or know what developers know. Ignoring problems that players have with a game because "the users are stupid" only hurts the studio in the long run, when end-users choose not to buy the game or criticize it on the internet.
As studios begin to focus more on early and effective application of usability testing as a means of cutting costs and enhancing product quality, it may become harder for smaller studios to remain competitive in this area. Outsourcing usability to user experience groups or developing equivalent capabilities in-house is one solution, but in the absence of professional usability support, development teams can still benefit from usability testing methods through a user-centered mindset and the use of techniques such as those identified in this article.
Read more about:
FeaturesYou May Also Like
