Part 3 of 10: Feedback Handling [previous parts]

Audiences and creators have trouble talking to each other, because the former look at things from the outside, while the latter look at them from the inside.

In all projects I’ve worked on, each individual piece of work in progress would reach a certain point, where it was shown to someone who didn’t take part in its development. It could be a tester, the project lead, someone from the management, or a representative of the publisher. Sometimes they were laymen, sometimes not, but the point is they were not filled in on design and technical details. They approached the work in progress as players.

Usually, they would notice some problem. For instance, the game would seem too difficult to them.

Sometimes their next step would be to say something like “the game is too difficult”.

Most often, though, they would say something like “you need to reduce enemy hit points”.

In most projects, our own next step would be to take the feedback literally and implement it verbatim.

We would make enemies weaker, and then they would say the player character was too slow. So we would speed it up, only to be asked to increase player firepower. We would comply, and then they would tell us the game was boring now, so we needed to increase enemy hit points.

It's not an exaggeration: the design would walk in circles. This could last for months, and often ended in features getting scrapped, because we were running out of time. There was also  the side effect in form of a lot of cursing about those stupid testers, CEOs, publishers, etc.

Indeed, some of those people don’t understand their roles, but most of the time they are just trying to help you. They are, in fact, trying so hard, that they try to diagnose the problem for you. However, they cannot do that competently, and it has nothing to do with their expertise. It’s because they’re looking at the problem from the outside. The part of their feedback that matters most is the symptoms they experience. They are always right about those: when they feel the game is too difficult for them, then it really is.

Doctors face a similar dilemma. The patient has a real problem. But just because someone is asking for painkillers, doesn’t mean they need them. It only means they’re in pain. Maybe all they need is aspirin - or maybe a thorough medical checkup.

Likewise, when they say you need to reduce enemy hit points, it does not mean as much. It means the game is being too difficult for them, which in turn may be caused by a number of issues, from tough enemies to unresponsive controls.

In other words, their feedback is priceless, but it needs to be investigated.

This is, sadly, anything but obvious. I’ve seen hundreds of man-months go to waste due to this issue. I’ve seen good people quit their jobs, because they felt obligated to implement the feedback verbatim, and they still could not achieve good results. I’ve seen projects become such a mess that they would get cancelled or restarted. Don’t let any of this happen to you - even if it means saying "no" to your CEO sometimes.

Lesson learned: Stay on the lookout for symptoms. Ask players for their opinion, then read carefully between the lines.