Research methods

In our introductory course, we learned about a number of different user research methods. In this step, we will consider some of the benefits and limitations of different methods and how it is not just about choosing the correct method for the correct task – it’s also about implementing it in an effective way.

In the previous step, we discussed research proposals or briefs. This is the starting point for designing research, as it gives a broad outline of the research questions and intended objective of the research – what we are trying to find out.

User research is structured around broad, high-level research questions and objectives, but there are different ways of designing and structuring research sessions to answer these questions and meet these objectives. We need to choose the most appropriate methods in order to do so, and also implement these in an effective way.

As a researcher, you think in broad questions, but your team may not – they may want to know specifics, such as “do users struggle with this element?” rather than thinking more broadly about “what do users struggle with in general?” You should not be looking to confirm a specific hypothesis when starting research, but knowing what information your team is looking for can be helpful when planning your research.

When a key stakeholder asks a question and wants to hear “yes”, you do sometimes have to tell them “no”, but this will land better if you have understood that it was a yes/no question and they were looking for a specific answer in the first place.

If we do not plan carefully, we may end up conducting research sessions that do not map clearly on to these questions and objectives. This can lead to incomplete or irrelevant data and wasted time and resources. We have learned that design decisions should never be based on assumptions about our users, but they should also never be based on incomplete data that we have had to manipulate to make fit.

Benefits and limitations

Good research comes from an understanding of the benefits and limitations of different research methods, but this is about more than just the method itself.

When planning research, you need to understand how to balance different methods with each other, as well as with external limitations – time, resources, organisational constraints, and your own capacity.

A clear benefit of using different research methods is that you gather different types of data and can gain a much broader and more in-depth understanding of your users. This also helps you to communicate with different team members and stakeholders. Some people respond better to quantitative data (facts and figures, such as KPIs) than qualitative insights (how people think and feel), so having both will allow you to communicate your findings to a wider audience.

However, when using different methods, you need to ensure that you are not duplicating efforts and essentially generating the same data. You need to make sure that you are taking advantage of the distinct features of each method and using them correctly.

Let’s take interviews and surveys as an example.

An interview is essentially a conversation in which the researcher asks questions and the participant responds, but there is much more to it. During an interview, the researcher observes the participant, looking at body language and emotion, which can provide useful insights into their thoughts and emotions. In an interview, the conversation can also go down different tangents and lead to unexpected and valuable insights.

A survey is also a list of questions that the participant answers, but in writing. Some questions will generate more quantitative, number-based data, such as “on a scale of 1-10, …?”, and some questions will allow the participant to respond more freely by writing in an answer to an open question. However, unlike the focus and detail of an interview, surveys are primarily designed to generate a large amount of high-level data from a wide range of participants.

When using these methods, you need to think about how to design your research and use each method correctly, thinking about practicality – whether something is actually within your capability – and scale, or how many people you involve as participants.

Interviews take time! You and your participants need to find a convenient time to meet, which can involve travel, and you need to allow enough time for the actual interviews to allow the participants to fully answer all of your questions, taking their individual needs into account. You also need to consider the time it will take to process and analyse all of your interview data.

It takes much less time to complete a survey than to participate in an interview and there is no need to arrange specific times or locations to meet to gather the data, so this can be done more quickly and at a much larger scale than interviews.

As the time available for research is often limited at each stage of the development process, you will not be able to interview a huge amount of participants. It’s good to be ambitious and want to conduct thorough research, but you need to also be practical – trying to interview 100 people by yourself in just a few weeks would be very difficult!

On the other hand, surveys are designed to reach a much larger group of participants to gain a broad, high-level understanding, but if you only send a survey to 10 people, you will probably not generate useful insights as you will not gather enough data from such a limited range of responses.

You also need to consider how you will manage your data after it has been gathered. As we learned in Week 1, new AI technologies can assist with the processing of substantial amounts of data, but you may not always have the opportunity to use them. This means that you will need to analyse your own data – and if you have hundreds of interview transcripts or survey responses to go through, this will take a significant amount of time and effort. You might also find that your attention to detail and the quality of your analysis starts to slip doing such a huge amount of work, which could mean that you miss important information.

We want our research to be effective and efficient, but let’s consider what those words actually mean.

Research can be done quickly, but that does not necessarily mean it was effective or efficient, if you do not end up with the proper data.

You could conduct several interviews or send out a survey and receive 100 responses in a week, but if your questions are not answered properly – or if you are asking the wrong things – then the data might not be that useful.

As we have learned throughout this course, research is not effective if it is not inclusive or accessible. This limits your insights and risks excluding people from your service.

Efficient user research is planned alongside development so findings arrive when they are more useful. This ensures that you can help the team prioritise and progress work, embedding improvements into the service without wasting time or resources.

Effective, efficient research is planned to balance different methods, account for available time and resources, remain inclusive, and deliver the right insights at the right time. This ensures informed decisions, wise use of resources, and a full understanding of user needs

Task

Think about research you have conducted or participated in. What methods have you used? Think about how different methods complement each other, or perhaps the limitations you have encountered

Share your thoughts and experiences with your fellow learners in the comments below.