Stampede’s usual remote usability testing session, conducted with a user from the organisation. (Disclaimer: Actual screenshot of the session was altered to maintain the user’s anonymity)

Stampede offers various solution services to address client’s problems, across the world and Malaysia – one of them being service design, to address problems pertaining to workflow beyond the digital realm propose solutions for them.

At one point we were conducting research for service improvement of a client from the energy industry, and we investigated one of the touchpoints that potentially be where we could improve service efficiency — the request form, a product that serves as the single channel for requestors to assign designers and resources to their projects. The most logical way to proceed with this investigation would be conducting usability testing, to detect areas for improvement of the form and reduce frictions pertaining to the touchpoint. However, limiting ourselves to a singular research method will risk us missing the insights of a bigger scope that may correlate to the problem or even be the cause of it.

What happens if we are more open about the scope of the outcome, where we are still bounded by the limitation of problems we need to address at the moment, yet receptive to deeper underlying issues? That will enable us to solve the right problem, and we tried to achieve this by employing the hybrid research methodology, which in this case usability testing is combined with user interviews — where product testing is still prioritised, yet we open up to discovering bigger problems to be addressed.

Here are some of the learnings from the hybrid research method:

1. Getting the most out of the research session without spending additional time and resource

Our initial plan is to only conduct usability testing on the request form, which is merely one of the touchpoints in the whole service and was assumed to be the main point of friction by the client. But we believe that in order to really solve the issue, we need to see the bigger picture, get a clearer understanding, detect the areas needing attention and address the correct problems.

How do we do this, without additional time and resources allocation to investigate the matter?

Every research method can only cover a limited scope of results. Usability testing, the main method we need to use, can only provide insights within the test product’s confine. For broader discovery, we will need to use the user interview method before conducting the usability testing, but we thought that we could employ a more efficient approach.

From there we got the idea to combine them in the same session. In this project’s case, the product that we need to test does not contain too many flow, tasks and screens — enabling us to fit in the user interview components wherever relevant. This is a chance we saw where we could maximise the research outcome by saving the time and resources needed should we choose to do it the other way in order to cover different scopes of results.

2. Additional findings can be obtained organically by being more contextual in the research session

As explained earlier, we wanted to probe into the issues outside of the tested product to understand the service side of the matter and the correlation between them. However, the nature of questions for these two scopes will be quite different; asking them blatantly without consideration will affect the session and responses from users as the questions will be seen as coming out of nowhere and causing a mental load due to not anticipating them.

How do we organically ask the questions outside of (product) scope?

To make the session flow coherently, we strategised meticulously on where to ask the usability testing question and when to ask the questions outside of the product scope, with dependencies.

If we want to ask the users more about a certain topic that is out of the tested product, we ask them when we arrive at the relevant section in the prototype. For example, we want to investigate where the user gains awareness about the product (responses could vary outside of the tested product’s confine), we may ask them about it on the prototype’s landing page, not on the screens that is more specific to tasks.

Being organic in the question flow is important, to get the right answer at the right moment and make it feel less scripted. Putting more effort into strategising this will yield better responses from the users and obtain results efficiently.

3. Insights gained will be richer and open up bigger opportunities for consideration to be addressed by stakeholders

By opening up to a mixture of research methods, researchers also open up the avenue to explore beyond the supposed research outcome from a singular chosen method. This will enrich the insights gained further, providing more value to the stakeholders who placed the investment towards the research — be it in effort, time or monetary form.

Problems of varied scale and spectrum can be found out in one go for consideration, on which issue to prioritise at a given time. To take this case’s example, the usability testing findings about the form are actionable in the product domain and could be solved for a short-term remedy, but our discoveries beyond that (on the service level) have given the stakeholders a greater vantage point on issues in a bigger scale, to be solved in the long run.

But caution must be exercised on scoping the outcome to be investigated, not to dilute the research result and to make them viable to be addressed to all parties involved.

The result of probing outside of the tested product’s confine

Lessons Learnt

Any research methods — be it usability testing, user interviews, or surveys, they ultimately serve only as the means to get what we are targeting. By clearly outlining the desired outcome and being more open to how we approach the problem, we will be freed from the confine of usual practice on conducting the research and know better when or where to utilise the methods. One may even combine them, by not strictly married to the method’s outcome but by adjusting the method to get the desired outcome. This will enable us researchers to make full use of the scheduled sessions with the users, to get the most out of it and ensure the utmost efficiency is being met.

Efficiency is the crux of us researchers, as not everybody sees the value behind the activity as much as we do. Delivering them within the constraints of time and resources is one of the ways to maintain stakeholders’ confidence in research, supplemented by the apparent impact of the insights gained and the value brought upon through it.