[Starting a new series, former Ubisoft designer Luban looks at why regular and detailed playtests are vital to center a game's development around the player.]

There is nothing new about asking testers for their feedback on a game in development. However, the practice of managing playtests by following near-scientific protocols, and of integrating them very early in the development cycle, is a more recent trend.

The spread of real playtests in the game development cycle is probably part of this silent revolution; a revolution profoundly affecting the development environment.

How? Playtests force game development to center around the players instead of the hopes of the development team. Let's look at the effects of this shifted focus:

- Playtests allow the identification of gameplay or level design flaws that could elude the grasp of normal testers.

After all, testers are always seasoned gamers who are not necessarily representative of the target audience. Who better than a casual gamer to pinpoint issues related to the difficulty curve or the overall understanding of the game?

- Playtests fulfill a moderator role in situations of disagreement or controversy within the design team.

A series of playtests can quickly settle a contested issue by resolving almost any counter-argument or dispute, thereby preventing the disagreement from spiralling into an impasse. Playtesting is also a management tool.

- The partnership between playtesting and design can be very constructive. For example, it can be quite instructive for game and level designers to observe gameplay during playtesting, allowing them to immediately determine whether or not particular aspects of their design work as planned.

- Playtests executed on pre-prod mock-ups allow the anticipation of problems very early on, as well as timely corrections of said problems (the faster a problem is corrected in the development cycle, the less expensive it is). Game development can therefore become truly "player-centric".

- According to the playtest protocol and the selection of playtesters (hardcore, casual, etc.), playtests allow the examination of a specific aspect of the game with heightened acuity: game balance, navigation, understanding of the game objectives, etc.

We all have the opportunity to play games that display high production values but nonetheless suffer from obvious flaws: erratic difficulty curve early in the game, navigation issues, overly complex interface, and so on.

Such flaws could often have been easily avoided if they had been identified early enough.

Major names in the industry understand this quite well, such as Ubisoft, which possesses qualified teams and invest lot of resources in this aspect of game development.

What kind of problems might we fix or prevent with playtests? Some examples include:

- Accessibility and ease of use (interface, navigation within the game, etc.).

- Identification of sure-fire-wins, i.e. strategies allowing a player to easily overcome any challenge created by the designers and therefore remove any interest in the game or the current mission. This issue is especially sensitive for multiplayer maps.

- Fine-tuning of the game system: experience has shown me that the intensity of use of game features (weapons, equipment, actions, etc.) tends to vary considerably according to a number of factors.

These include player profiles, the time a given player spends on familiarizing himself with the game, and of course the game tuning itself.

Only through long-term playtests with relevant samples of players can we ensure that the game tuning maintains its balance and relevance even after long hours of gaming.

- Analysis of the early reactions of different categories of players during their first session. This will highlight their first impressions and initial frustrations. Some game demos have probably had a negative effect on the marketing of games they were meant to promote because of accessibility and tuning issues that could have easily been spotted during playtesting.

- For multiplayer games, the robustness of the game system and the potential of maps.

I have had several opportunities to delve deeply into playtest management. I built the playtest structure from scratch at the Ubisoft Annecy studio, where the successful multi-player "versus" modes of Splinter Cell: Pandora Tomorrow and Chaos Theory were developed.

I set up the recruiting methods, playtest protocols, and the debriefing methods employed in this program. I also set up a playtest cell at the Ubisoft Bucarest office and led playtests there myself. Playtests have changed the way I perceive my job as creative director, so I feel the need to share my experience with everyone.

Let us start with a definition. Playtests consist in analyzing the reactions of a representative pool of players toward gameplay in order to improve the final game and to make sure it matches their expectations.

Some will argue that game testing is nothing new. True, but real playtests have nothing to do with the debug testing executed at the end of the development cycle.

Traditionally, game designers ask testers for their opinions. Testers are often excellent players and are therefore not always representative of the targeted demographic which is often made up of mainstream gamers.

Moreover, testers generally get to know a game so deeply that their knowledge of it strengths and weaknesses profoundly influences the way they play. Therefore, they do not play as someone who discovers the game for the first time.

Well-executed playtests allow us to evaluate gameplay strengths and weaknesses with great accuracy since they rely on two solid principles:

- The careful selection of playtesters.

- The use of ad-hoc protocols.

The Selection of Playtesters

Just as a peasant needs fertile ground in order to ultimately obtain the best yields, good playtests require a group of carefully-selected playtesters. I could never insist hard enough on the importance of the recruitment and evaluation of the playtest candidates.

What are the recruiting criteria? This depends, of course, on what kind of playtests we are planning. We may need hardened gamers, beginners, console-only gamers, multiplayer fans, and so on.

The candidate's gaming proficiency and overall game culture represent the first criteria. The second is the candidate's ability for analyzing and drawing conclusions from their gaming experience.

Note, however, that it is not mandatory that a playtester should possess a high level of competence on both criteria. Again, the type of playtests will determine the requirements.

I have the utmost respect for the playtesters I have worked with. Their good will and enthusiasm are boundless. Many came to Annecy from distant cities like Lyon, Grenoble, or Belfort simply for an unpaid half-day session!

This generosity and enthusiasm are characteristics of our industry; let us nurture these characteristics by treating playtesters with the gratitude and respect that they deserve.

The Use of Ad-hoc Protocols

The protocol is the unifying thread of the playtest session, defining the objectives, allocation of resources, and especially the methods of collecting and parsing information for a given playtest. The playtest protocol needs to adapt to the specifics of the challenge at hand (game system tuning, navigation, map concept, etc.).

During the playtest campaigns that I led, I would prepare a different protocol for each session. Indeed, an important part of those playtests involved multiplayer maps under construction or game system tuning. Each session revealed specific problems to be analyzed in the subsequent session.

I shall conclude this first part by repeating that a playtest campaign must be directed with a true scientific rigor if it is to be of any use; one does not conduct playtests simply by bringing over one's buddies for a few hours of fun followed by a session of easygoing Q&As.

Each aspect of the session must be carefully tailored in order to best realize the objectives at hand.

Managing the session itself requires constant attention, not only because one can learn much by watching the playtesters in action, but also because things do not always go as planned!

I shall address concrete aspects of playtests in the second part of this article.

Previous Articles

The Megatrends of Game Design, part 1

The Megatrends of Game Design, part 2

The Megatrends of Game Design, part 3

The Megatrends of Game Design, part 4

Physics in Games: A New Frontier

Multiplayer level design, part 1

Multiplayer level design, part 2

Multiplayer level design, part 3