Skip main navigation

When should we user test?

In this article, we take a close look at a user-testing framework you can use, and learn when would be the appropriate time to user test.

It is easy to assume that user testing should happen at the end of the design process – after all, you need a product for users to test. In reality, testing should be happening throughout your design process – from ideation through to launch.

User testing doesn’t have to involve sophisticated methods and technology to give you useful insights. Many successful companies started out with an idea that they developed by relying on simple user testing methods.

How Airbnb uses user testing

The following case study of the vacation rental company, Airbnb, shows how a conversation with your users early on can be the difference between success and failure for a business.

Read: Think user testing is a luxury? Ask Airbnb [1]

As you develop your concept and take it into the prototyping and development stage, the methods you use to test may change and need to become more sophisticated. You will go from gauging user responses to an idea or concept to testing whether a products’ overall experience continues to meet user goals.

It is useful to spend some time developing a structured framework for your user testing throughout the design process. This could either be: [2]

  • Using paper prototypes before you undertake your initial design, along with focus groups and surveys to
  • Using lo-fi and hi-fi clickable prototypes during the digital prototyping phase
  • Conducting extensive user testing at the end of the process to help you to refine what you’ve built.

Now, let’s take a closer look at a user testing framework you can use.

Framework for conducting user testing

Before you begin user testing, It is useful to spend some time developing a structured framework for your user testing throughout the design process [3].

1. Define your goal

Before you can begin your test, you need to have a clear idea about what you are trying to understand or know more about. This could be unpacking what your users want to achieve with a product, or gathering evidence to use for stakeholder buy-in. This goal will help you decide what needs to be tested and what methods you can use.

2. Prepare the test object

Next, prepare your test object, keeping your testing goal in mind. Perhaps you are still in the early phase of your concept development and can prepare a few sketches to gauge how users respond to it, or you might have hi-fi clickable prototypes to test with your users.

3. Select the test method

Decide how you are going to test based on the requirements and testing objective. Will the test be conducted in person or remotely? Will it be an unmoderated test or do participants need guidance? There are a number of factors that can influence this decision, from the scope of the test and the nature of the concept to be tested.

4. Write a test script

It’s important that your user tests are structured to keep them focused and aligned to your defined goal. A typical user test begins with a warm-up, a series of tasks and a cool-down or reflection.

5. Recruit test participants

When researching and selecting participants, it is crucial that you only recruit people that are representative of your real users. Consider the potential diversity of your user base and select an appropriate number of participants for the method of testing. Studies have shown that even testing with just five subjects can be sufficient to identify pain points in your concept [4]. However, this decision should be made with the goal in mind of obtaining useful and insightful data.

6. Prepare the site/infrastructure

Whether you are conducting a remote or in-person test, make sure that you prepare the test environment beforehand to account for any distractions or unexpected variables.

7. Carry out a test run

Conduct a pilot test to make sure that any potential stumbling blocks or issues are ironed out before the actual test. This can be conducted with members of your team in lieu of users to get a fresh perspective.

8. Evaluate and analyse the results

Once the test is complete, the results need to be analysed and a report compiled. This report can be shared with the broader team so that they can investigate and form solutions for any pain points or issues identified during testing.

References

1. Carroll, I. Think user testing is a luxury? ask Airbnb [Internet]. Progress Telerik; 2021 Jun 23, Available from: https://www.telerik.com/blogs/think-user-testing-is-a-luxury-ask-airbnb
2. Murphy, C. A Comprehensive Guide To User Testing [Internet]. Smashing Magazine.; 2018 Mar 17. Available from: https://www.smashingmagazine.com/2018/03/guide-user-testing/
3. What is…User testing [Internet]. Omniconvert; 2021 Jun 23. Available from: https://www.omniconvert.com/what-is/user-testing/
4. Nielsen, Jakob, Landauer, Thomas K, A mathematical model of the finding of usability problems, Proceedings of ACM INTERCHI’93 Conference; Amsterdam, The Netherlands, pp. 206-213.

This article is from the free online

UX Design Fundamentals: Creating business outcomes

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education