Dealing with questions
Sometimes, your participant will ask you a question. It's natural to want to help, but this can confuse your findings, so you need to encourage them to find their own solutions.
One approach is to ask the question back. If they say, "What does this button do?", reply with "What do you think it does?". If they're persistently asking questions, politely explain that you're interested in seeing how they solve these problems themselves, but you'll be happy to answer any outstanding questions at the end of the session.
Similarly, there will undoubtedly be moments where your user gets completely stuck. Sometimes you can learn a lot by how people try to rescue seemingly lost causes, so don't intervene straight away, but don't leave them struggling for too long either. Ask their opinions if they've sunk deep into thought, and if they're still nowhere near the right path, make a note and then step in and help them.
Running your first test can be quite an experience. Your instinct will be to yell out in frustration when the user overlooks your beautifully crafted navigation and goes in completely the wrong direction. Resist this urge.
Just as fitness instructors will tell you that pain is weakness leaving the body, so usability experts will remind you that you're finding out how to make your website better. It can be a painful process, but testing will make a big difference to the people who use the site.
Some participants will be more 'useful' than most, uncovering dozens of issues, while others will breeze effortlessly through the test. Some you'll barely be able to get a word out of; others you won't be able to shut up. This is one reason why you generally want to test with a few people, but even the quietest participants will teach you something – probably hidden somewhere in the recordings.
After the test
You've got your users to come in, the tests went well – now what? Analysis is the most important phase of testing. Having the most well-executed tests with the most interesting and insightful users won't matter if you can't turn them into sensible findings and recommendations.
First, write down everything you remember from the tests while it's still fresh in your mind. If you have a number of tests booked in a row, gaps in between sessions are perfect.
At this stage, just try to capture the most obvious issues; the ones that made you think "Why didn't I see that sooner?". With any luck, there'll be a few of these. To do the detailed analysis you'll need to set aside some quiet time later, get those headphones on and run through your videos.
The main things to look for are moments when the user struggled to use the site productively. These could range from simply overlooking a checkbox to times of complete and utter confusion. Whenever you find one of these moments, pause, scribble it down and then resume playback.
As you review how your participant used the site, also pay attention to what they said. Listening is one of the best ways to uncover mental models, which may be surprisingly different from how the site actually works. This is usually an indication that your design isn't quite right, although for some larger sites a user may not need to understand all the intricacies. Just ascertain whether they understand enough to use the site well and mark down any issues with the others.
One word of caution. Although it's important to listen, there are times when you should also take users' comments with a pinch of salt. What they do is normally more important. Ignore any comments such as, "I understand this, but my mum wouldn't." You're not testing their mum.
