We quickly realised that people have yet to understand the difference between reliable and unreliable data and the difference between factual evidence and opinions. Most businesses are making decisions based on users’ opinions which likely lead to a really frustrating product development journey. We’ll talk more about this later in this article.

Now, user research is not rocket science, but just like any other skill, you need to practise in order to get good at it. For some reason, most companies will send their least trained employees, or even interns, to go out and talk to customers. For them, if you can talk and ask questions, you can do user research. “This is the set of questions, now go wild!” is often the attitude, but really, there is an art to interviewing and doing user research that yields truly useful results.

This article is about avoiding these fundamental mistakes so that you get reliable, useful and usable data. We’ll first discuss common mistakes and then we will go through small steps you can take to improve your practice.

Mistakes

1. Collecting opinions, not facts
2. Imposing Quantitative methods on Qualitative data

Steps to improve

1. Using Themed questions
2. Insights: more than just findings

Collecting opinions, not facts

In user research, the people you are talking to and interviewing are naturally inclined to be helpful and will give a lot of feedback that they think is important. Be wary, though, as a lot of the “feedback” you get is actually opinions if you don’t ask the right questions or if you ask in the wrong way. One of the most common research mistakes that people make is to interpret opinions as facts. Opinions are dangerous because they change, are subjective and are inconclusive. If you build your products/services based on them, your product can become irrelevant pretty much anytime because opinions shift and are heavily influenced by how a user feels at the time. The focus should always be on how users use your product instead of what they think of it.

Facts, on the other hand, are concrete. You want to dig into the facts of users’ lives – pain points, behaviour and what solutions they deploy. The focus should be on understanding if there are any existing habits, like how they usually tackle a problem, or technology that you can ride on for users to easily adopt your solution as part of their routine. These all need to be taken into consideration as you design the solution. It is the holistic journey of how they will adopt your solution and not about whether they like it.

In order to collect facts from a user, you need to ask the right question. I usually divide my questions into three parts and types: context specific, product specific and opinion-based. Most of my focus will be context and product specific as useful insights will mostly come from these two areas. I often ask for examples so users will tell me stories which are facts. If I have extra time to kill, I’ll squeeze in some opinion based questions. Let’s run through an example:

Research Goal: Find out how might we improve Jira as a product for task management.

The first two types of questions will give you richer data about a user. It allows you to:

1. Imagine/understand how your users use your product as part of their daily routine
2. Decide on information hierarchy, i.e. prioritisation of what to show or provide your users with, as you now know what important information they need to complete their tasks
3. Decide on your non-negotiable features and “nice to have” features

Qualitative vs. Quantitative Data: More does not necessarily mean more

Most companies are familiar with quantitative rather than qualitative data which means that they are skeptical of research that has a small sample size. After all, when learning about research and statistics in school, we were always taught that a larger sample means more reliable results. As a result, and from my own experience, most teams resolve to increase the number of research participants to double, or even triple that of a qualitative study. The outcome is often remote, unmoderated testing or well-curated surveys where the focus is on getting a large number of responses rather than getting rich, quality data. What’s worse is when they quantify the results and turn them into graphs making things look more scientific than they are.

It is not that quantitative data is bad, but you shouldn’t try to use quantitative methods on qualitative data. You need to first understand the differences between the two and what they can be used for. Quantitative is used to segment and identify your target market and qualitative is used to gain an understanding of the underlying reasons, opinions and motivations of your target market.

Using quantitative methods on qualitative may mean testing the product on and talking with more people to get more results but this does not necessarily lead to more insights, just a lot more work. Be aware that there is a difference between the two types of data. If you want to understand the user beyond the surface, you should focus more on qualitative data. It is not just about what they say and think, but finding out what they feel and discovering their unmet needs. This is something that you almost certainly cannot achieve through a survey.

Doing better research

It’s not hard to change your research methodology a little to get better quality data. Here’s a couple of suggestions.

Using Themed questions

When you design your question guide, start with broad themes. Then test it.

Themes will help you create categories on different kinds of data you want to collect. Once this is done, flesh out a few questions under each theme and test it with people around you. From their answers, you will know the questions to ask to solicit relevant data. Let’s look at an example:

Research question: How might we help our youths get better at saving money?

Insights: more than just findings

Good researchers are great at spotting patterns and findings. Great researchers take the results, analyse them and can derive insights to inform breakthrough solutions. Deeper insights are truths that are unearthed by continuously asking “why” to get to the heart of the problem.

Identifying the difference between a finding and an insight is like a muscle that needs to be strengthened. The key is to always be curious about the deeper meaning of someone’s behaviour. “People watching” is one of the best exercises to train your eyes to identify interesting findings that can lead to deeper insights. When you are in a public space, observe what people around you are doing or carrying. Question why they’re doing this and if you are bold enough, casually chat them up to understand them better. If this is too much of a challenge, try observing and chat up your colleagues instead.

Example of Findings vs Insights

Small steps, quick wins

Overall, the good news is that user research is just like any other skill; it requires practice to get better. These are small, easy steps you can start adopting as part of your practice. If you take small steps to avoid these fundamental mistakes as a part of your current user research practice, we are sure that the time you invest into talking to your users will be more productive and will yield better results. Let’s move the needle on the user research scale from “awareness” to “nailing it”.

The post User Research: Common Mistakes and Best Practices appeared first on Stampede: the strategic design & technology company.

Were answers provided to our questions? Not quite, because we doubt the reviews but we’d love to watch it anyway. So we set up expectations, or metrics – that if something happens or does not happen in the movie, then it’s going to be good. So we go. It ends up making us feel good. Hypotheses proved, metrics met.

In this case, we are the end user. So it is easy to conduct research based on the persona we know best – ourselves. But what if we are taking someone on a date to see the movie, whose favourite genre is entirely different from ours – and impressing him/her is paramount?

The scenario is no different in the user experience (UX) when you are building products, services, applications and websites for people who are intended to use the website, and whose opinions and impressions matter.

In UX, research is important to ensure that we are designing it for the users, instead of ourselves. The formalised research is called UX research, which helps us identify, prove or disprove our assumptions, recognise their needs, mental models, their pain points, challenges as well as what unites them together towards the common goal of using the product. Essentially, research shall add information to our work, improves our understanding and validate our decisions.

How do we start?

Created by Erin Sanders in this article, the Research Learning Spiral provides five main steps for conducting UX research. The first two steps are about forming questions and hypotheses, and the last three steps are about gathering knowledge through selected UX research methods.

1. Objectives. What are the knowledge gaps we need to fill?
2. Hypotheses. What do we think we understand about our users?
3. Methods. Based on time and manpower, what methods should we select?
4. Conduct. Gather data through the selected methods.
5. Synthesise. Fill in the knowledge gaps, prove or disprove our hypotheses, and discover opportunities for our design efforts.

There are many ways to do UX research. The most important question is: how do we frame it within many other factors? These factors can include:

• The end goals of the product – both from business and users’ perspectives
• Time constraint
• Accessibility (to information, users, stakeholders, decision makers etc.)
• Other limitations

The first three are the most common factors we encounter in Stampede. In taking considerations of these factors, we work closely with clients to ensure that we get as much information as possible in the research stage before we begin in the UX designing phase.

The end goals of the product

When a UX research does not take place…

For example, after 3 months of launch, they’d like to see the sales increase 300%.

For strategists, this is an important thing to note – as we can reverse engineer our way in finding out how to increase the sales by 300% through a number of methods, which should be specified in research.

Secondly, the most important thing is of course, to consider the needs of the users. For example, by the end of the process, the client should be receiving testimonials as how easy it is to use the website now. Other ways to measure this expectation is also to install heat map analysis tools for your websites such as Hotjar, Crazy Egg or ClickTale.

The tools on UX shall be explained later in Part 3.

Time constraint

As much as we like to talk to as many users as possible and take much longer to conduct research, most of the time we do not have that luxury.

Typically, user research stage in Stampede takes place for the first two weeks after a UX project kicks off. At this point, as we work closely with our clients on securing the business and user end goals, we also narrow down the best methodologies as possible to fit in the timeframe.

For example, in the course of two weeks, it is quite impossible to conduct a physical workshop involving the main stakeholders who are scattered all over the country. It takes a lot of time by itself for the arrangement and logistics, let alone preparing and conducting it.

So what do we do? In the research stage, we ask the clients…

• … for the contacts of key primary users we can get hold of and ask them for time to do usability tests – depending on accessibility arrangements, if any (see next point)
• … if there are existing products they like, and we do comparison reviews
• … if there are existing data of users we could take a look – users statistics, testimonials, dev site links, wireframes etc. Every data should come of use.

We shall talk about knowing which UX methods to use later in Part 2.

Accessibility

Another factor to take into consideration before starting on the UX research journey is to find out the accessibility of the key stakeholders. These can be:

• The decision maker from the client side – most likely the product owner
• How accessible we are to users if we intend to interview or do usability testing?

Other limitations

When we talk about user experience we often forget about the most important thing: the user’s experience. Too often we are so obsessed with clever solutions and stunning visuals that they completely ignore the fact that their users won’t care about it as long as it works. This is why limitations are key to great UX. The odds of achieving a clear and understandable user experience are way higher if you limit yourself and the features of the product you’re designing depending on the users’ needs.

Other limitations that might surface before you start on UX research would be:

• Scope given by client. There are clients who are lenient about us experimenting and suggesting, but it is always good to check with them every single time.
• Client approval
• Other items that might have been mentioned in Non Disclosure Agreement (NDA) e.g. the limits to how accessible the current site data might be, their users or disclosure of current information etc.

All of these can help us to narrow down how to begin our UX research and determine which methodologies to use in the next stage.

What’s next?

Now that we have established the factors that will help us to move to the next stage, here’s one thing to keep in mind: the biggest mistake a UX strategist can make is excluding end users from the process. Start the design process by including the users from the start, listen to their disappointments, pain points as well as what makes them happy – not only while using the product, but also understanding their mental model in general.

In conclusion, UX research is a very interesting process in a sense that it helps us avoid our biases since we are required to figure out solutions for people who are different from us.

What factors have you considered before you embark on a user experience research? Share with us in the comments section!

There are a number of popular methods used in improving the user experience at all phases of research and design. We shall take a look at some popular methods and when you should use them in Part 2: Which Methodologies Should I Use?

The post UX Research Part 1: How Do We Start? appeared first on Stampede: the strategic design & technology company.