In playtesting, minimize researcher-subject interaction

If you’re user testing your prototypes or playtesting your games, you must minimize researcher-subject interaction. Why?

The purpose of playtesting is to learn what would happen if this prototype were a real product in the hands of real users in the wild. What you don’t want is to find out what users would do under a bunch of contrived conditions that don’t resemble the wild. And what you really don’t want is to think you’re seeing what would happen in the wild, when really you’re seeing what would happen under contrived conditions, when the two are different.

Just the fact that you’re sitting down with the user and asking them to interact with your product already puts the user in a contrived situation. But that one is hard to avoid. It’s what happens after that that you can control.

In an ideal playtest, you’re totally hands off. You give the prototype to the user, and then simply sit back and watch. In a poorly-run playtest, you’re talking the user through it: “You can tap there to make the ball turn orange.” “You can open that menu if you want to build something.” Etc. Doing this will totally invalidate your results. Why?

Because humans are easy to influence and they really want to please one another. If you tell the user “you can open that menu to build something,” she’s also getting the sense that you want her to build something. Insofar as you wanted to know if she would have built something in the wild, your test is invalidated.

Simply hearing the way you describe your prototype can give the user the sense that you’re emotionally invested in its success (which you likely are), and then, with zero reflective awareness of how this just influenced her motivation, your user now interacts with your prototype with the overriding (but unconscious) motivation of finding a way to act such that you will feel good. Disaster.

One thing you can do to mitigate the risk is to minimize researcher-subject interaction. If the prototype needs explanation, write it down beforehand. If the user is stuck and you want to offer a tip, write it on a slip of paper and slide it across the table.

As much as you possibly can, keep your personality out of it.