Playtests are always a valuable way to evaluate the state of your game, but it's important to observe the right signs and ask the right questions in order to get objective and worthwhile feedback.

I'm currently TAing a game design and development class that is playtesting analog games on a regular basis, and I noticed the same shallow questions appear on multiple questionnaires: "did you have fun?" "did you understand the rules?" "did you feel like the game was fair?" "did you feel like it was too hard or too easy?". These questions don't always yield honest, objective answers, and beyond that the feedback you receive isn't necessarily profitable to you. "Did you have fun?" "Yes I did have fun!" Ok, so now what? Your game is perfect and you can release? "No your game sucked!" Ok, so now make your game better. But how? In this situation the proactive decision would be to take this question out of your playtest and just assume the answer is no.

The objective in asking questions of your playtesters is to gain insights into the state of your game that you can't see from your viewpoint as an experienced developer. I decided to brainstorm a set of questions to stimulate insights and generate a new perspective of your game to redirect design and development onto a more aware path. Feel free to comment and share your own!

1. How much time did you feel like you were playing for?

This question is simply a replacement for the question "did you have fun?". If you're truly questioning whether or not your game at it's core mechanic is an entertaining system, this question will give you a more accurate answer than asking whether or not a player enjoyed themselves. Keep track of how long they were actually playing for, and if they feel like it went by quickly, they were enjoying themselves. If it felt like it took a while, they may have been frustrated or felt awkward or confused. Like Einstein apparently said, "When you are courting a nice girl for an hour it seems like a second. When you sit on a red-hot cinder for a second it seems like an hour. That's relativity."

2. Did you feel like you were making friends or enemies with the other players?

Obviously this only applies to multiplayer game, but the answer to this question will give you insights into the social atmosphere your game generates. If your game is highly competitive, for example, your players' answers should reflect a sense of competition. On the other hand, if your game is cooperative and your players feel like they are competing, your game mechanics do not provide for the social tone you aim to generate. If you don't have a targeted social atmosphere, players commenting that they felt like they are making friends is always a sign of positivity.

3. Could you play the game again without looking at the rules?

Sometimes this is impossible, as in the case with extensively complex tabletop RPGs where a dungeon manual is always necessary, but if your players understand repeated mechanics like turn steps (draw a card, roll the die, move the piece, play cards), then your rules are in a good place. Tailor this question to the context of your individual game, but it's always worth asking. If your players always need to look at the rules each time they take a turn, you might need to think about either simplifying the rules or designing a more clear, memorable way of conveying them to the players.

4. What was your strategy?

This is my favorite question to ask in a playtest. Having a player walk through their gameplay strategy breaks down their thought process into steps and allows you to identify where there are bumps in the road and where your design clarifications are working to facilitate an understanding of the game flow. Some playtesters will tell you they didn't have one, and the proper response to this is to recognize that at first glance your game does not present clear strategies to the player or facilitate their creating their own. If a player can step through their mental process, your game is successfully facilitating critical thinking, and you can evaluate their strategy in the context of your game design.

5. How far in advance could you predict your opponents' moves?

This question is similar to #4 in the sense that it gives you a perspective into how critically aware your players are of the strategy of the game, However, this question delves deeper into both the complexity of the strategies available to the players and the depth to which your players were invested in the game during the playtest. If opponents are entirely unpredictable, it can mean either that your game is too complex for its' players, or that your game is about adaptation and responsiveness. If opponents are too predictable, it means that your game is predictable and thus simple. This can cause the tic-tac-toe effect, where often the game begins with a predictable outcome or always ends in a tie.

6. To what extent did you react to your opponents' moves?

As an extension of question #5, this question gives you further insight into the pace of strategizing in your game. If your players can predict your opponents' moves, but don't feel any need to react to them by altering their strategy, then are your players really interacting in a meaningful way? How can you create a rock-paper-scissors scenario where players are forced to react to their opponents moves to generate meaningful interactions that redirect gameplay?

7. Can you explain why the victorious player won?

This question is somewhat a validation of #4-6 in the sense that it confirms an understanding of strategic thinking in your game. However, it also serves another purpose; if your players can answer this question successfully, it confirms dynamic strategic thinking but also reassures you that you are presenting the players with enough information about their opponents to make logical counter-moves. Sometimes your players are aware of when and how to make a counter-move, but don't have the proper information about their opponents (fog-of-war, hidden hands, too much randomness). If your players can recognize that they had the inferior strategy, or that they had the unlucky rolls and draws, you can be certain as to whether your players are making informed decisions or flipping coins.

8. To what extent did you feel like you were in control of the outcome of the game?

Depending on how controlling you want your players to feel, the outcome a well-balanced game always leans towards the players making the most optimal decisions. If a player makes a mistake, they should be punished, and vice-versa. If a player feels like too much of the game was left up to chance, or they felt like another player was more influential, you as a designer should examine the extent to which player decision-making impacts the outcome of the game. Is one player's power snowballing out of control to the point at which no other players can catch up? Is your game too reliant on the randomness of RNG, dice, or card-drawing? Or, in an edge case example, is your game not random enough?

9. Did anything hold you back from seeing your strategy or plans through?

Did an opponent make a move or counter-move that prevented a choice that would have furthered a player's strategy? Was a player looking forward to enacting a particular play, but was too resource-starved to make the proper moves? Did a player have all the right cards but one to make the perfect, game-decisive move? Answers should be interpreted subjectively within the context of your game, but hearing the same complaint multiple times should always be a flashing red light. Maybe your resource generation mechanics are inconsistent or too sparse. Maybe you need to provide more options for adapting a strategy to random mechanics like card drawing or dice-rolling.

10. Name the game you have played that is most similar to the game you playtested.

This is a simple one, and can produce negligible feedback from uninformed players, but sometimes you design a game focused on a particular audience only to realize it's far from the target. If players can draw parallels between your game and another, you can both confirm that your targeted market is spot-on by proxy, and you can identify the games that players are using to intuitively sink into control schemes, turn orders, and core mechanics, and develop a system to teach incoming players given what they likely already know.