“What people say, what people do, and what they say they do are entirely different things”

– Margaret Meade

This quote perfectly sums up why usability testing should be at forefront of the UX design process.

But the reality often tells a different story.

We see companies placing far too much trust into what users say, even though memory is fallible. And we see this same trust placed in what users say they will do, even though we as humans are terrible at predicting the future.

The result is inevitable. Failure to understand users and thus failure of the product.

Yet this can all be avoided by using the best user research technique available to us. The one technique that allows us to really observe and thus understand our users: usability testing.

Usability testing is so powerful, in fact, that it is hard to imagine any project where user experience is important allowed to progress without it.

How a Usability Test Works

There seems to be this misunderstanding that usability tests are expensive, complicated and require a scientific setup to run. This goes some way in explaining why they aren’t utilized nearly as much as they should be.

But they are in fact so simple and inexpensive to run that almost anyone can do so.

Technical Setup

The first thing you need is a laptop or desktop.

You then need to install user testing software. Free tools, ones that have free trials and others that can be purchased are all suitable.

This software allows you to record a user’s activity on the screen as well as their face and facial expressions through the laptop or desktop’s built-in webcam. This is important because 90% of communication is non-verbal. So, if you really want to understand your users, you better be able to see their expressions to determine what they’re thinking and feel.

A built-in or external microphone is also used to record what a user is saying and what the moderator is asking.

Physical Setup

The physical setup is even simpler.

There is no need for a laboratory or one-way mirrors. All you need are two chairs, a desk, a user and a moderator.

The user and yourself don’t have to be the only ones in the room. It’s fine to have a colleague or two sit in and observe the session as well. But it’s best to run an external monitor into the laptop or desktop so they can also observe without crowding the user.

Define Your Test Objectives

Each usability test you conduct should have a clear purpose and set of research objectives.

Here are some guidelines to make sure you don’t waste your efforts.

Define clear goals: Before running a usability test, you should have clear goals. For example, say you know that your conversion rate is below the industry’s average or your users are dropping out along the purchase funnel. You can define the clear goal of finding out the reasons why this is the case.

A usability test is therefore a great way to find out the reasons behind a problem you want to solve. Other data, like analytics, can inform you of a problem but a usability test will tell you the reasons behind it.

Be specific: It’s tempting to want to cover everything in a single usability test. But this will just result in shallow insights into a number of different areas. So, get specific instead.

Gaining deep insights into a single problem you have is better than shallow insights into a dozen – and is ultimately more cost effective.

Test frequently: A usability test isn’t a one-off occurrence. Schedule them regularly to cover different aspects of the UX process particularly as it lessens the temptation to try and cover everything in one session.

Don’t confuse with functional testing: The goal of functional testing is to see if your software works. Usability testing is all about obtaining a greater understanding of how you can make your software work better instead.

Create a Test Script

A moderator of a usability test has a lot to consider. They’re trying to hold a conversation with the user while making sure they’re relaxed, observe what they’re doing and ask insightful and appropriate questions at the right times.

A usability test is therefore no time to wing it and prove your multitasking skills.

This is precisely why for a usability test to go as smoothly as possible a test script should be created.

A test script isn’t something to be mindlessly read as a robot, but instead acts as a guideline outlining the tasks you want a user to complete along with the questions you want to ask along the way.

If there’s something particularly important you want a user to focus on, you can make sure it is included in the script so you don’t forget.

The test script gives the usability test structure while still allowing you to deviate when users do something unexpected as they tend to do. When this happens you can get a user back on track.

A good test script would follow this outline:

  1. Introduction
    1. Goal is to relax the user
    2. Welcome and thanks
    3. Explain the purpose of the session
    4. Explain the room and technical setup
    5. Tell them there are four things to keep in mind:
      • We’re not testing you. We’re testing the product
      • Speak your mind
      • Think out loud
      • Feel free to ask questions
  1. Interview
    1. Simple background questions to start
    2. Ask them to describe the last time they did x….
    3. Understand their goals
    4. Understand the nature of the problem
  1. Tasks
    1. Start with a natural task, if appropriate (what they did the last time they used similar software to ours)
    2. Then pre-defined tasks (3-5)
    3. Ask lots of “what” and “why” questions
    4. Clearly explain the scenario
    5. Clearly state the tasks

How to Recruit Users

One of the roadblocks to companies conducting usability tests is not knowing where to find users. While this is true of established companies looking to launch a new product, it’s even more so of start-ups launching a new product.

There are four ways in which you can recruit users.

From worst to best:

Colleagues: Colleagues are the easiest source of users but also the worse to use. They are too close to the product. Even if not directly working on the product team they still know too much about the industry and business. In short, they are too bias to reflect an objective user’s view. Testing with colleagues should be disregarded from the get-go

Friends and family: Using friends and family is better in the sense that they are not insiders to the company. But they still know you, may know others within the company as well as some knowledge about the business through you. They are also unlikely to make up your target market

Do it yourself: Recruiting users yourself is certainly a viable option. If the company is already up and running, then there will be a database of users who can be contacted. An email or phone call explaining what you’re doing, when you’re doing it, and what incentive there is for them are all necessary steps. The only issue with this method is that it is time-consuming

Not only do you have to reach out to all these people, but you then need to find at least five of them who are willing to participate at a set date and time that works for all involved

Professional recruiters: Inevitably using a professional recruiter is the most expensive option. But it also saves the time and the headache associated with recruiting suitable users yourself

A recruitment screener that defines how many participants are necessary, the demographic criteria, as well as participants who have used your own or a similar product is an absolute must. You also need to state how long a session will last, the compensation for each participant and the location where the test will take place.

What to Test

What you want to test depends on whether you have an existing product or it doesn’t yet exist.

Existing product or company:

Current version: The current version of your product can be tested to find out where you need to improve the product

New version: If you ’re developing a new version of the product or adding new features, you can test this new version in the form of a prototype. Prototypes are invaluable to the design process because you can find out everything you need to know without spending time and money into developing a new version

Competition: Testing competitor’s products along with yours in a side-by-side usability test can be an eye-opening experience, as it tells you how well your product stacks up against the competition.

Product doesn’t yet exist:

Competition: Solely testing your competitor’s products can tell you about the three most important things you need to know about your users: goals, behaviors and context of use. You can find out about your users’ problems, how well they are being solved, and what you can do to improve upon them

Prototype: Whether you ’re looking to improve a current version of a product or build a new product, the same concept of building a prototype applies. This doesn’t necessarily need to be an interactive prototype either. You can still test the software flow, assess mental models and see how people go about completing a task

When to Use a Usability Test

Usability tests can and should be used throughout the product development process and beyond.

This includes during the:

Research stage: At the beginning, a usability test helps you understand the nature of the problem. You gain valuable insights into your users and what the problem is that needs to be solved

Design stage: At this stage, a usability test helps you to validate your solution. You find out if you are actually solving users’ problems, how well they are solved, and what you need to improve upon before committing time and money to actually building the product out

Final Testing: You can use a usability test one last time as a final assessment. Once you have built out your proposed solution, you want to test before you go live. While it may be too late to make any changes at this point, you can at least be aware of any minor issues, which makes them easier to deal with from a customer service perspective. These issues can also be prioritized to be fixed after launch.

The Benefits of Usability Testing

There are both high level and more specific benefits for every user test you run.

High Level Benefits

  • Influential: There is no better way to influence internal stakeholders of user requirements than showing them real users using the software
  • Unite stakeholders: Clarity about users’ goals and experience can help unite all stakeholders behind a common cause
  • Resolve assumptions: User testing lets you validate or challenge assumptions you might have about the product and its features
  • Additional Insights: If a usability test includes an interview (as it should) it gives you additional insight into the three most important aspects of your users: their goals, behaviors and the context in which they use a product
  • Avoid feature debates: An all too familiar scenario in every product team is members arguing about which features should and shouldn’t be included. These debates can be put to bed by seeing what real users think instead of what the team thinks
  • Cost effective: There is no user research method that has a higher ROI than usability tests. 5 tests can be conducted (and you only really need to do 5) within a day and for under $1,000

Specific Benefits

The best thing about usability tests is that we as designers can learn and understand our users so much better, which is of course vital to designing better products.

  • Goals: Users may not only have a primary overarching goal when using a product but also secondary and tertiary goals. You can gain insight into what these are so you can design excellent software
  • Behaviors: By observing your users you can learn about their behaviors, which gives you far greater insight than other user research methods might. For example, seeing a user struggle to find something, miss what was thought to be a clear call to action, or take a long time to complete a specific task would not necessarily become apparent just by talking to them or by looking at analytics data
  • Context:  When you learn more about the context in which a product is used, you better understand how it fits into the picture of your users’ lives. You can then design a product to better suit this context
  • Pain Points: The UX of a product should always flow as smoothly as possible. Usability testing enables you to identify pain points and roadblocks that interrupt this flow
  • The Competition: A side by side usability test where you ask a user to complete a task on your software and then on your competitor’s can tell you how well it stacks up against the competition.
  • Market Demand: 42% of start-ups fail due to there being no market need. As we mentioned in our An Introduction to User Research article, it’s no use outright asking users if they will use or buy your software – but you can gauge their reaction to see if you’re really solving a problem instead. Additionally, if you also do a side-by-side usability test, you can also see if you’re solving it in a better way than your competition.