This scenario plays out more often than you might think – products that work perfectly fine on paper, but somehow fail to truly resonate with users in practice.

Why teams skip live testing (and why that’s understandable)

Let’s be honest – when you’re managing a product team, live testing often feels like a luxury you can’t afford. You’re juggling feature requests, fixing bugs, meeting deadlines and keeping stakeholders happy. The product is live, metrics are “okay,” and testing seems like something that can wait.
We get it. We’ve worked with enough product teams to understand the reality:

• Your backlog is already overwhelming
• Resources are stretched thin
• Stakeholders want new features, not refinements
• Getting access to real users is challenging
• Testing could disrupt existing workflows

But what I’ve learned time and time again remains true: skipping proper testing now almost always costs more in the long run. When products evolve without deep user understanding, teams end up building features that don’t solve real problems, fixing symptoms instead of root causes, and missing opportunities for genuine improvement.

What live testing really reveals: three stories

Story 1: The Trust Deficit

A few years ago, while investigating an underused performance monitoring feature in an enterprise reporting application, we discovered something unexpected. Users had developed elaborate verification routines, double-checking every number in Excel. Why? A lack of trust in the data, stemming from inconsistent data input and unclear data lineage.

The solution wasn’t in the dashboard design – it was in fixing the upstream data collection process and making data sources transparent.

Impact: The improved data collection process and transparency transformed how teams used the system. Analysts stopped maintaining parallel Excel sheets and began trusting the platform for critical decisions. Team leads reported more confident decision-making in planning meetings and the platform became their single source of truth for performance discussions.

Story 2: The Presentation Paradox

We were brought in to find out why a reporting dashboard was failing to drive action in executive meetings, despite its sophisticated analytics capabilities. Live testing revealed the core issue: we were forcing a long-form analysis tool to serve as a presentation medium.

This led to a breakthrough solution – creating a separate presentation mode that prioritised narrative flow over deep analysis, allowing presenters to tell clear data stories while maintaining access to detailed insights.

Impact: The new presentation mode transformed how teams communicated data insights. What was once a source of confusion in executive meetings became a powerful storytelling tool. Leaders could now focus on strategic implications rather than questioning the numbers, leading to more decisive actions based on data insights.

Story 3: Hidden Opportunities

Late last year, I was part of the team at Stampede who were tasked to conduct a UX gap analysis for Malaysian government. We tested a startup ecosystem accelerator website with real users to identify the gap, but walked away with so much more. Not only did we identify immediate usability issues and recommended practical fixes, we also discovered unexpected ways users were finding value in the platform.

Impact: Beyond addressing the immediate usability issues, the insights led to the development of new services that better matched how startups were actually using the platform. The client was able to evolve their offering from a simple resource website to a more comprehensive support platform that better served their ecosystem’s needs.

The Stampede Approach: What’s the (whole) story?

While many testing approaches focus on isolated aspects like UI or functionality, our methodology is designed to give product teams a holistic, more complete picture. To us, real insight comes from understanding not just how users interact with your product, but how your product fits into their broader work life.

Our approach combines both quantitative and qualitative methods, because we’ve learned that neither tells the complete story alone. When we spot a pattern in the data, we investigate the “why” through careful observation. When we observe interesting behavior, we validate its significance through data. This dual-lens approach helps establish clear causation, not just correlation.

Making it work: our testing methodology

Live product testing isn’t just about watching users click through interfaces. It’s a carefully orchestrated process that begins long before we meet any users and continues well beyond the testing sessions themselves. The team has refined our methodology through years of testing products across different scales and industries, from startup platforms to enterprise systems.

Here’s what we do and why they are effective.

Setting the foundation well

We begin by understanding the product context deeply. Having focused conversations with stakeholders to understand business goals and constraints, reviewing analytics to spot patterns worth investigating and designing research that fits your specific situation are all necessary homework. We’re particularly careful about selecting participants who can provide relevant insights – people who use your product in meaningful ways as part of their daily work.

As UX designers, what we look for is striking the right balance between rigour and flexibility. While we have a structured approach, we stay adaptable to product team’s needs and constraints. This thoughtful preparation helps us maximise the value of every testing session while minimising disruption to their operations.

Testing approaches

1. Direct Live Testing

We observe users interacting with your actual live product in their natural work environment. This means watching real tasks being completed with real data and real stakes. It’s like shadowing a user through their workday, observing not just how they use your product, but how they integrate it into their broader workflow. While this approach requires careful planning to avoid disrupting production systems, it often reveals the most valuable insights about how your product is actually being used in the wild.

2. Staging Environment Testing

Think of this as a dress rehearsal with your product. We create a testing environment that mirrors your production system, complete with realistic data and workflows, but in a controlled space where we can safely observe user behaviour. This approach is particularly valuable for financial systems, healthcare platforms, or any product where testing in production isn’t feasible. The key is making the staging environment feel real enough that users interact with it naturally.

3. High-Fidelity Prototype Testing

Sometimes we need to recreate specific parts of your live product as an interactive prototype. This might be because we want to test a particular user journey in isolation or we need to simulate a specific state of your product that’s hard to replicate in the live environment. As we carefully rebuild these scenarios, we can focus user attention exactly where we need it and iterate quickly on potential solutions.

Multi-method integration

Sometimes the most powerful insights come from combining methods:

• Using analytics to identify patterns, then investigating through live testing
• Combining prototype testing with live system observation
• Validating qualitative insights with quantitative data
• Cross-referencing findings across different user groups and contexts

When to consider live testing

Every product team faces moments when they need deeper insights than analytics alone can provide. Perhaps your metrics look good but user feedback suggests otherwise. Or maybe you’re planning a major evolution of your product and need to ensure you’re moving in the right direction.

Live product testing delivers particular value when:

• User adoption isn’t meeting expectations despite good technical performance
• Users are creating workarounds to your carefully designed processes
• You’re seeing high task completion rates but low return usage
• Feature usage patterns don’t align with your product strategy
• You’re planning significant product evolution and need ground truth

Often, teams discover that live testing would have helped them avoid months of misdirected development effort. The insights gained typically go far beyond the initial scope of investigation, revealing opportunities for improvement that weren’t visible through other methods.

Ready to bridge the gap?

So what could look like a simple usability issue at first, could signals a deeper opportunity to make your product truly work for your users.

This is what fascinates me about live product testing: the way small, everyday user behaviours can point us toward significant opportunities. Time and again, I’ve seen how these insights help teams avoid months of building solutions for the wrong problems.

Your product likely has similar stories waiting to be uncovered. While you focus on keeping your product running and delivering new features, we can help you spot these patterns and understand what they’re telling you about your users’ real needs.

—

If you’re wondering about the hidden potential in your digital product, we’d love to hear your story. Reach out to studio@stampede-design.com

This post is part of our ongoing series about evidence-based product evolution. Follow us for more insights about building products that stand the test of time.

The post Live Product Testing: The gap between working and working well appeared first on Stampede: the strategic design & technology company.

—

Well, we shared a little about the event orchestrations in the previous blog. Now, let’s focus on the next part.

We attended the workshops on Day 2 of the Focus Days and also talks on the latter Conference Days. Following are the breakdowns of our learnings there, in separate POV formats to cover our different experiences!

Day 1: The Workshop

Day 2 of the Focus Days portion, was our Day 1 — the workshop day.

All the workshops were very useful and interesting and we were keen to join all of them. The topic ranges from soft skills development like negotiation strategies and UX influence, to the craft-oriented ones like eye tracking and service design. But alas, our time and resources are limited so we need to choose really well which one to attend!

We needed to make full use of this opportunity, so we deliberated in-depth on which workshop to be attended by each of us. Factoring in the relevance of topic with our domains, strength, future project and function considerations, finally we settled with:

• Me (Sarah): Soft skills for UX professionals by Prof. Clemens Lutsch, Claudia Bruckschwaiger & Hanna Köhler
• Azim: Service design and UX – a conversation by Eric Reiss

On to the learnings we obtained from these two workshops!

In Sarah’s perspectives:

In this workshop, the speakers, led by Prof. Clemens Lutsch, divided us into several groups consisting of multiple hats to allow us to see different perspectives from different roles.

We were given some activities to increase our self-awareness about our own capabilities. To top it up, the speakers then provided some input about how these capabilities have been useful to our lines of work. We were then given an assignment to role-play based on several scenarios. These scenarios were to mimic the real life situations on how we adapt to different kinds of challenges during communication.

The outcome of this workshop emphasised that our technical skills are just medium to execute, but in order to be successful in design, there are many other soft skills that we need to master, and they are not easy and that practice is all it takes to succeed. There were 3 categories to these soft skills, being communicative, social and personal soft skills.

Communicative soft skills revolves around the way we designers communicate with the people we interact with. This includes our business or product team, our developers, and our designer peers. Examples would be the negotiation skills, active listening, ability to abstract, linguistic fluency, or ability to enthusiasm.

Social soft skills on the other hand would be on how we are dealing with the interactions with our team, like resiliency, emotional intelligence, conflict resolutions, diversity competence, or even ability to cooperate. Do you see the difference from communicative skills?

Personal soft skills is all about the core supporting your work executing. These skills revolve around the how behind you are executing your tasks or job, like analytical skills, decision making, creative thinking, strategic thinking, self reflection and willingness to learn. Whether or not you realise it, we pretty much rely heavily on these skills in your day to day work. It makes our executions more human and user-centric.

My take was around the communication area (that includes conflict resolution, influencing, negotiation skills, etc). We heavily practice this in Stampede, and it does not get any easier because you definitely don’t experience the same situation every day. We work with people from different industries, priorities and personalities. It is all about us versus the problem!

Azim’s Point of View:

Relevant to my designation as a UX designer, I enrolled into service design workshop by Eric Reiss of FatDUX. I had no prior deep knowledge about the details of service design works at this point of time, which makes this workshop such a pivotal moment for me to realise that services are everywhere – and they can be designed.

The workshop was filled with lessons that Eric himself has gained throughout his life and related them to the UX design or service design domain. One simple example, Eric told his personal story of cat food catering to “customers” (himself as cat owner who buys the food) rather than the “real user” (his cat, who didn’t appreciate green peas in the cat food) and the story of great service nuances provided by the hotels he stayed in during vacations, each of them provided the audiences with clear examples on how even the minuscule details can affect user experiences.

Those were the personal touches that Eric had throughout the session, but being a workshop that is geared towards service design, naturally there are technical lessons being passed down.

Some of the discussed methods and tools have already been incorporated into our practice in Stampede — like the how-tos of user interviews, building user persona, or ways to design effective surveys. Strengthening the fundamentals is always great as a starting point and further validates that what we have been practicing is on the right track.

However, plenty of methods and tools presented were new to me (or even to anyone in Stampede) that brought massive value out of this workshop.

The Service Star provides a clear visual on areas to be focused for the service provided, either for the company we are evaluating or our own service. A star-shaped spider chart would be ideal, but that would be an extremely rare (or even impossible) case where the service provided is perfect. On the service provider side, they would be able to gauge which areas to be improved and maintain the performance.

The ergonomics of need can outline users’ perspectives in clear metrics to detect which part of the service could be focused upon. The outcome will also prepare the service provider for what kind of situations they will face when dealing with customers (eg: when the attitude and expectation are already negative to undergo the task, how can the service provider handle the situation in the best way?)

Aside from the typically-used customer journey map and touchpoint matrix/service blueprint (which often end up being too convoluted and needing massive effort to be understood by people who don’t follow the process), FatDUX have their own framework of communicating the outcome better to stakeholders — the service-experience scope sheet. By filling in this sheet with the details and touchpoints involved, it will communicate the design clearly, outlining the factors and actors involved in the service, with a clear demarcation of the period in service.

There were many learnings to be unpacked and relayed to the team for us to improve our service design practice. This avenue is waiting for us to explore structurally as some of our recommendations when tackling digital UX problems were in the form of service design, and we are eager to flesh out the solutions using methods gained here to maximise the value that our clients can receive.

Day 2 and 3: The Talks

Finally, the main event starts — the conference.

The day started with the official launching of the event, as this is the day where most attendees registered for. We got to see the fantastic opening ritual where there were drums, gongs and an opening act by the Founder of the event itself, Hannes before all the talks started.

Once again, we faced the same dilemma — all talks were exciting for all of us. We had a problem here! The 2-day conference was filled with talks given by speakers that came from many countries and backgrounds. Yet it was set to be multiple tracks segregated by theme ranging from leadership to design management to UX research, it would be wasteful if we focus on just one or two, and miss the talks that would benefit us.

We had four team members, so why not split? To maximise what we could gain out of this, we made use of our numbers and specialisations, strategised on how to split up and went to the workshops that would give us the highest impact on our growth opportunities. Shaza and Wan attended the ones that were relevant to the business side, I (Sarah) attended the ones about design advocacy, and Azim attended the ones that were heavy on the UX research side.

We saw that the strategy deemed effective after several talk sessions, and we continued to do so till the end of the conference.

Sarah’s Point of View:

As a product designer, the central theme of the talks I went to was design advocacy. It emphasises that designers educate and collaborate with non-design teams and stakeholders about solving problems using design. The outcome of design advocacy is the value of design being recognised and practised in not only business, but also the technical team.

One of the things I learned was that when it comes to advocating design, the key is opportunity. However, designers ourselves are the ones who should create the opportunities. Scott Parker, who talked about “Functional Process to Strategic Contributor – Realising the Value of Design”, explicitly covered the topic where designers are problem solvers and how we are responsible for realising where and how we can create opportunities for ourselves when it comes to making the value of design visible across the organisation; we should not just wait for those opportunities to come our way.

On top of that, I learned from Ashnee Gounden of Discover Limited who spoke about “Scaling Design with Limited Budget and Time in a Sustainable Way”, that there are considerations when it comes to advocating design. You need to look into the UX maturity of the particular organisation. Evaluate it before you can start navigating around it; this is because as designers, we need to understand before giving solutions. Find out what matters to the organisation and work your way to improve the UX maturity level. Trust me, the “scaling” portion is most exciting part.

Scaling here would mean how you would navigate to increase that maturity itself. How do you plan that incremental journey and what your considerations are. Of course, there are many ways to improve it. It really depends on what matters to the organisation and what their challenges are. Empathise their situation before coming in and introduce the process. Really look into it and strategies based on all the resources you have. It was John Bowie who emphasised “Influence is Not Enough: Inspiring the Unconverted”. He talked about how people respond to inspiration more than influence, so what can you do to inspire them? And what’s next?

When it comes to design debt, of course, a designer like me would be excited to manage it. “What are my considerations?” “How much are my resources and capacity?” “What can I do with everything that I have currently?” These are the questions that you need answers before executing. According to Alicja Suska of Outlaw Design, who talked about “The ROI of Managing Design Debt – Getting the Leadership Buy-in”, communication of the return from managing the design debt is also crucial as it definitely helps the overall organisation to be more effective and profitable in the long run, so create those opportunities! Leverage the people around you as this helps us understand other people’s perspectives and priorities. This was also covered by Manuel Schnell of Kelag and Juergen Hillber of Red Bull. They emphasised gathering and understanding what the other team is doing so that the whole team could see things eye to eye. This way, our solutions will be richer with inputs from multiple disciplines.

As a plus point, Alejandro Carbejo shared “Context and Empathy as Pillars for Collaborations in DesignOps”. This is where I learned how the DesignOps team at the back end has been supporting the structure of the design team to collaborate as support to other teams in an organisation.

The interesting part is that Stampede has already been practising these for years and attending the talks diversifies my view on navigating situations.

Azim’s Point of View:

I was assigned to attend the UX research & management track, which is very relevant to my line of work in Stampede — UX design. These talks have expanded my horizon on how research can be done from an international perspective, with some familiarity about how we have always done things and issues faced in projects (many things are similar!). But of course, there are many new discoveries made here, which opened my eyes regarding the level of international design thinking. Wish I could cover them all but here are some of the selected highlights we could all learn from!

There were talks discussing the process. One of the notable one I have attended was by Javier-Bargas Avilla from Google, where he shared his experience of improving YouTube’s localisation feature, where they started with the complex German language as the base model (German translation were atrocious, at least according to native speakers). He outlined the metrics used as base to be gauged, and emphasised the importance of collaborative effort with the deserved party (locals in this case) which in the end lead to measurable success.

And there were talks discussing the methods. Dr. Panagiotis Zaharias from UX Prodigy discussed long-term UX, where he made the stance that the UX research methods we have adopted so far are actually measuring momentary experience only, not considering the long-term experience yet. Sourced from academic studies, 4 methods were outlined that have yet to be used in real practice; Day Reconstruction Method, Experience Sampling Method, AttrakDiff, and UX Curve — with their pros and cons to be considered before use. All these to capture the user sentiments of long term usage, in order to leverage on hedonic quality (related to user’s intrinsic motivation) of a product which will result in memorable experience and loyalty.

Then there were talks about the tools. Bill Albert of Mach49 shared about the advanced tools that can be used to measure user’s emotion to gauge the experience by breaking down the existing spectrums when classifying emotions and prioritising the relevant ones to UX. Plethora of scientific tools were demonstrated, like Galvanic Skin Response, Electrodermal Activity, eye tracker, pressure sensitive mouse or chair, and facial expression analysis, to name some — but ultimately, knowing the why behind all of the readings obtained is the most important aspect to assess the real user experience.

Another notable talk that attracted my attention was the talk attended by all four of us — about Systemic Design by Peter Horvath of Whitespace. The presentation was very smooth (too smooth even, for me who loves to snap every slide and failed to capture some of the slides this time) and eye-opening, where Peter described the components of Systemic Design and the relations between them. It took me quite some time to absorb the topic of systemic design fully, and how it plays into the bigger picture of UX design.

Overall, the conference was a treasure trove for those who love the research aspect of UX design and have curious minds open to new ideas and concepts — especially for the ones who are fairly new in the field like me.

Closing

So, as we look back at all these moments and learnings from the workshop and the World Usability Congress itself, we realise that design is so much more than just looking good or being functional. It’s about creating connections, understanding each other, and constantly evolving towards the greater good.

At the end of the day, design isn’t just a job for us, it’s a lens through which we view and interact with the world, imagining all the ways it could be better with every new thing we learn, every fresh perspective we take in. It just brings us one step closer to a future that’s not only smart but also kind and inclusive.

So, let’s embrace the complexity, celebrate the diversity, and continue to design not just for the eye, but for the heart and mind. After all, we are designers, aren’t we?

The post Lessons and practical applications we brought back from World Usability Congress appeared first on Stampede: the strategic design & technology company.

A few months ago, us from the design team were buzzing with excitement when Shaza announced that anyone could potentially attend the World Usability Congress 2023 in Graz! About a week later, Azim, the UX Designer, Wan, from business development team and I, the product designer, got the news — the three of us were selected to join Shaza herself for the trip!

We strategised ways to gain as much knowledge, as this conference is a treasure trove of UX practices in global level presenting many learnings that we could adapt and adopt. Even though the four of us have different roles in Stampede, we came to Graz with one mission in mind. We all want to find opportunities to elevate the design capability standard in Malaysia! On the 8th of October 2023, the four of us packed our bags and flew to Austria. We landed in Vienna International Airport and straight away took the train ride to Graz, while soaking in the sights and learning about Austria’s train system along the way.

World Usability Congress is an international conference organised by Youspi (an Austrian UX, CX and Usability agency) focusing on topics revolving around usability and user experience design practices. Held annually since 2013 in Graz, Austria, the conference brings together professionals, researchers, and practitioners from various industries to discuss the latest trends, share insights, and showcase innovative approaches to continuously improve user experience. This event covers the wide spectrum of usability and user experience design practice; ranging from UX design, user research, product development, strategy, Ops and more, making it a highly relevant event for anyone in the tech and design field.

The event lasted for four days from 9th to 12th of October 2023. The first two days were Focus Days — containing industry-specific talks and in-depth workshops, and then the final two days were the Conference Days — the main event containing multi-track talks, masterclasses and keynote (maximum 4 tracks at one time). Azim and I attended workshops relevant to our respective practices, and then on Conference Days all four of us dispersed to maximise the learnings from the multi-track talks and connections made. Check out our learnings in another post with more detailed breakdowns!

Our main objective in Graz was to gather knowledge, but we didn’t miss the chance to connect with as many people as we can as the UX community in this side of the world is very friendly and engaging, like ours back home. We used this chance to build genuine connections with fellow UX-ers or virtually anybody we can interact with and built our presence in the region.

Even before the conference started, the interaction with the UX community already started by meeting Johannes of UX Graz — one of Shaza’s comrades in arms, having organised together the 24 Hours of UX in previous years remotely from across the world (Making this occasion their very first time meeting in person, which we couldn’t really tell from their sense of camaraderie)! He toured us around the beautiful city of Graz and introduced us to several of his colleagues along the way. Very memorable introduction to the event and hosting city. We also met Ahmed Al-Huwwari of UX Jordan and Peter Horvath — both committee members 24 Hours of UX, making this event such a special in-person meetup for them!

The following day, after completing our workshops, we attended the UX Graz meetup dinner that was open to any WUC attendees. As our usual practice in any mass gatherings, Stampede members scattered as we make connections with as many new faces as possible because we love connecting with a diverse cast of attendees, boosting the chance of getting many new and different perspectives. That very night was very exciting as everyone we met were very genuine and friendly, we also gained some insights into how the tech and design scene looks like in Europe through those valuable conversations. We all had the same mission going into the conference. The actual conference is on the next day, but we already got a glimpse of UX practice through the lens of Europeans and pumped up to go through with the rest of the event!

Real conference days came next. Naturally, we made many connections there as the WUC event was designed to give plenty of opportunities to network and mingle around — during breakfast, in-between talks (20 minutes gap between talks), lunchtime, and the night of picturesque hilltop at Schlossberg during the Networking Dinner. Networking there was so magical that we made so many new precious connections throughout the journey, and we long to see them again anytime in the future. Even after the event ended on our way back to Vienna via train, we connected by pure chance with Susanne, one of the WUC volunteers, that our hearts were filled with joy till the very end and beyond!

And I’d like to share some tips in connecting with event attendees of any large prestigious events, that might not be applicable to everybody but have worked for us;

Focusing too much on design & professional conversation topics every time with everybody can feel transactional and takes a toll on our energy. Just remember that everybody is human no matter how professional they are, and we just want normal conversations to know each other better before opening up more.

So, we tested a different approach with a Danish designer named Anton. Surprisingly, we found that genuine connections were made easily when we are invested in that person’s story. Our conversations that night started to change, as they revolved around our respective country, life, career journey and virtually anything that came across our minds. The conversation eventually led to career & professional topics at some point, but it felt more natural. It was less tiring, and we could maintain our social energy for a long duration. And more importantly, it didn’t felt

We carried this approach to make more connections throughout the event and we managed to learn more about people, common issues and values, and finally not forgetting; to talk about the design cultures abroad to learn from each other. We even scored an office visit to Eventim in Vienna (Thank you for your hospitality, Marcus and Katharina!), attributed to the small talk and genuine connections made with the people we met there. All in all, connections should be pure and have minimal underlying agenda, as we all appreciate the natural relationships between people — as we always do in design field.

The highlight of the World Usability Conference for us aside from the important profession-related learnings, was the connections made with the people, thanks to the numerous networking opportunities designed within the conference itself. We cherish all of them, and this bond will go a long way in the future in whichever means that we have no way of predicting.

The post World Usability Congress: A pilgrimage every UX practitioners should go to appeared first on Stampede: the strategic design & technology company.

Stampede offers various solution services to address client’s problems, across the world and Malaysia – one of them being service design, to address problems pertaining to workflow beyond the digital realm.

At one point we were conducting research for service improvement of a client from the energy industry, and we investigated one of the touchpoints where potentially lies area of service efficiency improvement: the “request form”, a product that serves as the single channel for requestors to assign designers and resources to their projects. The most logical way to proceed with this investigation would be conducting usability testing, to detect areas for improvement of the form itself and reduce user frictions within it. However, limiting ourselves to a singular research method will risk us missing the insights of a bigger scope that may correlate to the problem or even be the cause of user’s woes.

What happens if we are more open about the scope of the outcome, where we are still bounded by the limitation of problems we need to address at the moment, yet receptive to deeper underlying issues including ones that are outside of assigned scope? That will enable us to solve the right problem, and we tried to achieve this by employing the hybrid research methodology, which in this case usability testing is combined with user interviews — where product testing is still prioritised, yet we open up to discoveries of bigger problems to be addressed.

Here are some of the learnings from the hybrid research method:

1. Getting the most out of the research session without spending additional time and resource

Our initial plan is to only conduct usability testing on the request form, which is merely one of the touchpoints in the whole service and was assumed to be the main point of friction by the client. But we believe that in order to really solve the issue, we need to see the bigger picture, get a clearer understanding, detect the areas needing attention and address the correct problems.

How do we do this, without additional time and resources allocation to investigate the matter?

Every research method can only cover a limited scope of results. Usability testing, the main method we need to use, can only provide insights within the test product’s confine. For broader discovery, we will need to use the user interview method before conducting the usability testing, but we thought that we could employ a more efficient approach.

From there we got the idea to combine them in the same session. In this project’s case, the product that we need to test does not contain too many flow, tasks and screens — enabling us to fit in the user interview components wherever relevant. This is a chance we saw where we could maximise the research outcome by saving the time and resources needed should we choose to do it the other way in order to cover different scopes of results.

2. Additional findings can be obtained organically by being more contextual in the research session

As explained earlier, we wanted to probe into the issues outside of the tested product to understand the service side of the matter and the correlation between them. However, the nature of questions for these two scopes will be quite different; asking them blatantly without consideration will affect the session and responses from users as the questions will be seen as coming out of nowhere and causing a mental load due to not anticipating them.

How do we organically ask the questions outside of (product) scope?

To make the session flow coherently, we strategised meticulously on where to ask the usability testing question and when to ask the questions outside of the product scope, with dependencies.

If we want to ask the users more about a certain topic that is out of the tested product, we ask them when we arrive at the relevant section in the prototype. For example, we want to investigate where the user gains awareness about the product (responses could vary outside of the tested product’s confine), we may ask them about it on the prototype’s landing page, not on the screens that is more specific to tasks.

Being organic in the question flow is important, to get the right answer at the right moment and make it feel less scripted. Putting more effort into strategising this will yield better responses from the users and obtain results efficiently.

3. Insights gained will be richer and open up bigger opportunities for consideration to be addressed by stakeholders

By opening up to a mixture of research methods, researchers also open up the avenue to explore beyond the supposed research outcome from a singular chosen method. This will enrich the insights gained further, providing more value to the stakeholders who placed the investment towards the research — be it in effort, time or monetary form.

Problems of varied scale and spectrum can be found out in one go for consideration, on which issue to prioritise at a given time. To take this case’s example, the usability testing findings about the form are actionable in the product domain and could be solved for a short-term remedy, but our discoveries beyond that (on the service level) have given the stakeholders a greater vantage point on issues in a bigger scale, to be solved in the long run.

But caution must be exercised on scoping the outcome to be investigated, be wary to not dilute the research result and to make them viable to be addressed for all parties involved.

Lessons Learnt

Any research methods — be it usability testing, user interviews, or surveys, they ultimately serve only as the means to get what we are targeting. By clearly outlining the desired outcome and being more open to how we approach the problem, we will be freed from the confine of usual practice on conducting the research and know better when or where to utilise the methods. One may even combine them, by not strictly married to the method’s outcome but by adjusting the method to get the desired outcome. This will enable us researchers to make full use of the scheduled sessions with the users, to get the most out of it and ensure the utmost efficiency is being met.

Efficiency is the crux of us researchers, as not everybody sees the value behind the activity as much as we do. Delivering them within the constraints of time and resources is one of the ways to maintain stakeholders’ confidence in research, supplemented by the apparent impact of the insights gained and the value brought upon through it.

The post Application of Hybrid Research Method to Uncover the Right problem appeared first on Stampede: the strategic design & technology company.