This scenario plays out more often than you might think – products that work perfectly fine on paper, but somehow fail to truly resonate with users in practice.

Why teams skip live testing (and why that’s understandable)

Let’s be honest – when you’re managing a product team, live testing often feels like a luxury you can’t afford. You’re juggling feature requests, fixing bugs, meeting deadlines and keeping stakeholders happy. The product is live, metrics are “okay,” and testing seems like something that can wait.
We get it. We’ve worked with enough product teams to understand the reality:

• Your backlog is already overwhelming
• Resources are stretched thin
• Stakeholders want new features, not refinements
• Getting access to real users is challenging
• Testing could disrupt existing workflows

But what I’ve learned time and time again remains true: skipping proper testing now almost always costs more in the long run. When products evolve without deep user understanding, teams end up building features that don’t solve real problems, fixing symptoms instead of root causes, and missing opportunities for genuine improvement.

What live testing really reveals: three stories

Story 1: The Trust Deficit

A few years ago, while investigating an underused performance monitoring feature in an enterprise reporting application, we discovered something unexpected. Users had developed elaborate verification routines, double-checking every number in Excel. Why? A lack of trust in the data, stemming from inconsistent data input and unclear data lineage.

The solution wasn’t in the dashboard design – it was in fixing the upstream data collection process and making data sources transparent.

Impact: The improved data collection process and transparency transformed how teams used the system. Analysts stopped maintaining parallel Excel sheets and began trusting the platform for critical decisions. Team leads reported more confident decision-making in planning meetings and the platform became their single source of truth for performance discussions.

Story 2: The Presentation Paradox

We were brought in to find out why a reporting dashboard was failing to drive action in executive meetings, despite its sophisticated analytics capabilities. Live testing revealed the core issue: we were forcing a long-form analysis tool to serve as a presentation medium.

This led to a breakthrough solution – creating a separate presentation mode that prioritised narrative flow over deep analysis, allowing presenters to tell clear data stories while maintaining access to detailed insights.

Impact: The new presentation mode transformed how teams communicated data insights. What was once a source of confusion in executive meetings became a powerful storytelling tool. Leaders could now focus on strategic implications rather than questioning the numbers, leading to more decisive actions based on data insights.

Story 3: Hidden Opportunities

Late last year, I was part of the team at Stampede who were tasked to conduct a UX gap analysis for Malaysian government. We tested a startup ecosystem accelerator website with real users to identify the gap, but walked away with so much more. Not only did we identify immediate usability issues and recommended practical fixes, we also discovered unexpected ways users were finding value in the platform.

Impact: Beyond addressing the immediate usability issues, the insights led to the development of new services that better matched how startups were actually using the platform. The client was able to evolve their offering from a simple resource website to a more comprehensive support platform that better served their ecosystem’s needs.

The Stampede Approach: What’s the (whole) story?

While many testing approaches focus on isolated aspects like UI or functionality, our methodology is designed to give product teams a holistic, more complete picture. To us, real insight comes from understanding not just how users interact with your product, but how your product fits into their broader work life.

Our approach combines both quantitative and qualitative methods, because we’ve learned that neither tells the complete story alone. When we spot a pattern in the data, we investigate the “why” through careful observation. When we observe interesting behavior, we validate its significance through data. This dual-lens approach helps establish clear causation, not just correlation.

Making it work: our testing methodology

Live product testing isn’t just about watching users click through interfaces. It’s a carefully orchestrated process that begins long before we meet any users and continues well beyond the testing sessions themselves. The team has refined our methodology through years of testing products across different scales and industries, from startup platforms to enterprise systems.

Here’s what we do and why they are effective.

Setting the foundation well

We begin by understanding the product context deeply. Having focused conversations with stakeholders to understand business goals and constraints, reviewing analytics to spot patterns worth investigating and designing research that fits your specific situation are all necessary homework. We’re particularly careful about selecting participants who can provide relevant insights – people who use your product in meaningful ways as part of their daily work.

As UX designers, what we look for is striking the right balance between rigour and flexibility. While we have a structured approach, we stay adaptable to product team’s needs and constraints. This thoughtful preparation helps us maximise the value of every testing session while minimising disruption to their operations.

Testing approaches

1. Direct Live Testing

We observe users interacting with your actual live product in their natural work environment. This means watching real tasks being completed with real data and real stakes. It’s like shadowing a user through their workday, observing not just how they use your product, but how they integrate it into their broader workflow. While this approach requires careful planning to avoid disrupting production systems, it often reveals the most valuable insights about how your product is actually being used in the wild.

2. Staging Environment Testing

Think of this as a dress rehearsal with your product. We create a testing environment that mirrors your production system, complete with realistic data and workflows, but in a controlled space where we can safely observe user behaviour. This approach is particularly valuable for financial systems, healthcare platforms, or any product where testing in production isn’t feasible. The key is making the staging environment feel real enough that users interact with it naturally.

3. High-Fidelity Prototype Testing

Sometimes we need to recreate specific parts of your live product as an interactive prototype. This might be because we want to test a particular user journey in isolation or we need to simulate a specific state of your product that’s hard to replicate in the live environment. As we carefully rebuild these scenarios, we can focus user attention exactly where we need it and iterate quickly on potential solutions.

Multi-method integration

Sometimes the most powerful insights come from combining methods:

• Using analytics to identify patterns, then investigating through live testing
• Combining prototype testing with live system observation
• Validating qualitative insights with quantitative data
• Cross-referencing findings across different user groups and contexts

When to consider live testing

Every product team faces moments when they need deeper insights than analytics alone can provide. Perhaps your metrics look good but user feedback suggests otherwise. Or maybe you’re planning a major evolution of your product and need to ensure you’re moving in the right direction.

Live product testing delivers particular value when:

• User adoption isn’t meeting expectations despite good technical performance
• Users are creating workarounds to your carefully designed processes
• You’re seeing high task completion rates but low return usage
• Feature usage patterns don’t align with your product strategy
• You’re planning significant product evolution and need ground truth

Often, teams discover that live testing would have helped them avoid months of misdirected development effort. The insights gained typically go far beyond the initial scope of investigation, revealing opportunities for improvement that weren’t visible through other methods.

Ready to bridge the gap?

So what could look like a simple usability issue at first, could signals a deeper opportunity to make your product truly work for your users.

This is what fascinates me about live product testing: the way small, everyday user behaviours can point us toward significant opportunities. Time and again, I’ve seen how these insights help teams avoid months of building solutions for the wrong problems.

Your product likely has similar stories waiting to be uncovered. While you focus on keeping your product running and delivering new features, we can help you spot these patterns and understand what they’re telling you about your users’ real needs.

—

If you’re wondering about the hidden potential in your digital product, we’d love to hear your story. Reach out to studio@stampede-design.com

This post is part of our ongoing series about evidence-based product evolution. Follow us for more insights about building products that stand the test of time.

The post Live Product Testing: The gap between working and working well appeared first on Stampede: the strategic design & technology company.

When I first joined Toro, which is a product team in Stampede (torotimer.com), the team was in the midst of a sprint. For those who are not familiar with sprint, it is a set period of time during which specific work has to be completed and made ready for review, usually involving the product owner, the design and development teams. The outcome is usually a prototype or solution to a problem, alongside valuable insights about customer needs and preferences.

In my case, the sprint focused on a freelancer product, which we internally called as Toro Freelancer. My colleague Luqman and I set out to build a new prototype for Toro, as well as being responsible for the testing research plan and participant recruitment.

Lesson 1: It’s okay to take a step back to solve the problem

From the ideas we had, the product owner selected a few viable ones, but also realised that they were skewed towards a known solution space. There are many solutions out there to help freelancers get paid easier, and beating this dead horse won’t really help our users in a meaningful way.

What we missed were strong ideas for helping our freelancer improve their productivity in a way that is sustainable for them. Realising this, we took a step back to shift our focus on the right problem.

Lesson 2: Standing out from the competition

Initially, I thought it was merely a minor issue to miss the step of having solid ideas for the right problem. But after the testing later on I realised one more lesson, which is that in making a product, we need to differentiate ourselves from the competitors, not just create another similar tool in the market.

Focusing on user’s problem in being productive sustainably really tested our product market fit. If we did not take a step back to focus on that opportunity, we might just waste our time creating another common quotation/invoicing tool, getting further from our goal of making freelancers subscribing for Toro.

Lesson 3: Be selective about what we want to test

One lesson that I learned from prototyping is that we need to be selective in what we want to test. Not everything has to be interactive in a product test. Sometimes we may want to keep a non-interactive item like a button that cannot be clicked, so we can ask the user to guess where it might lead. Test time is limited and this will help us to be more efficient and focused on the actual hypothesis we are testing.

Lesson 4: Be intentional in design

Another lesson is to be intentional in prototype design, to the small details. I learned this from a situation where I only randomised the variation height of bars in the chart, without any deep thought. However, during the testing, the users interpreted the bar chart in a certain way, like the chart was designed intentionally to show something. At that moment, it exposed my bias that some users actually do think and care about the tiny details.

Being intentional with the details is also another reason why we need to design the prototype with a smooth continuity so that users are not confused, surprised or distracted. For example: add a confirmation modal first to smoothen the flow, remove an existing CTA which can distract the user from the tested path etc.

Lesson 5: Ask the right questions to screen the right users

As I was taking the lead for user recruitment, initially I was thinking of an easy path: Edit an existing screening survey, spread it anywhere, pick the users and schedule the slots.

I did not expect it would be hard and time-consuming to find the right users. It was a moment of clarity for me as to why a certain UX agency has dedicated researchers only for user recruitment.

The screening survey attracted many scammers because of the monetary incentives, plus it was posted not in the right channels. We mitigated this by verifying through their LinkedIn profile or portfolio link.

Asking the right questions also means only asking the essential questions. I learned the hard way that sensitive questions, such as gender and marital status can be intrusive, especially when they do not carry any weight in finding the right users. Essential questions could make the screening form shorter, which would mean less friction for the users while providing enough info for designers to filter them.

Lesson 6: Be pragmatic, not dogmatic

Finding the right channel to funnel quality users was also important. What worked in the past might not work this time. After getting so many wrong users in the list, I started to think “If I were an established freelancer, where would I be to connect with like-minded people?”. I eventually hit a jackpot when I found a few Slack channels that vet the users to ensure a sustained professional community.

The benefits of getting the right users would later be manifested in the quality of the testing. The users could really relate well with the prototype especially when it was able to solve their pain points. I was super relieved when the right users stress-tested the prototype. Seeing what worked or did not work for them gave us some ideas on how we were gonna tackle the next step of the product.

Lesson 7: Making mistakes is a part of growth

Understanding different accents, handling internet connectivity issues, users being very prescriptive or just providing short simple answers, and managing time limits can be challenging to the facilitator. Having diverse users tested my ability to manage these sessions. As it was my first time handling a testing, having a more experienced designer observing me and providing detailed feedback improved my facilitation skills after each session.

I made the mistake of asking too many questions early on about what the user understood from the prototype. This consumed a lot of time and did not provide insight into whether the feature was useful (and how) or not (and why not). The reason we had testing in the first place was to test the prototype by understanding users’ thoughts on how the prototype would or would not solve their problem.

After conducting the testing, I feel more confident to do user interviews/testing. At the same time, I want to keep training those muscle memories until my interviewing skill becomes natural.

Another lesson that I learned is we sort of established rapport with the users. They could be a great resource if we have questions related to our product research, such as the reason they are using separate tools for invoicing and time tracking. Some of the users were so willing to help us, and they were just an email away 🙂

Lesson 8: On deciding what to ship

As a product designer, I had a sense of ownership to carry the product to success. However, I had a lot of uncertainty about how to nudge the team in the right direction when prioritising features for the minimum payable product (MPP).

During the testing, we tested a lot of features based on our research. However, in reality, achieving the ideal state of a finished product requires going through multiple release cycles, which can take months or even years.

So how do we prioritise certain features to ensure we are on the right path for users to start paying for Toro? What if our MYP is similar to all the other tools in the market? Will users be willing to pay for it?

The testing showed that we have a Unique Selling Proposition (USP) module that can differentiate us in the market, as it solves major pain points for users. However, shipping that module requires a lot of effort and other foundational features to be shipped first.

Since we do not want to release a half-baked product, should we build all those features, including our USP module, before releasing the subscription plan?

All of these questions were on our minds. After further reading and asking a few designers in the industry, we initially thought that a prioritisation workshop would provide more clarity on the path forward due to the inputs from the team. However, in our case, Shaza as the product owner, is well-versed in the feasibility, viability, and desirability, and this part is the product owner’s call to make, not the team’s. We learned that we should have discussed with the product owner first the best way this should be done.

Lesson 9: Learn from history

Later, I learned that Figma did not have the collaboration feature when it was first launched. There are many examples of successful startups that launched their products without their best-known features yet. What is more important is to ship, measure, and learn, rather than dwell in the uncertainty of which features to ship first.

Nobody can predict the future, but we can try to make the best decision at the time with the info that we have. If we fail, we need to fail fast and iterate. If the minimum viable product does not go the way we wish, at least it saves us from wasting our time and effort on the queued features. At least we learn what is working and what is not.

Lesson 10: Size the development effort carefully

I also learned that a feature might carry more complexities than I initially thought. If we overlook certain details, it might surprise us in the development phase. For example, can the user edit the invoice that has been sent? What if we allow the invoice to be edited if the client hasn’t seen the invoice on Toro platform? How sure are we that the client hasn’t seen the invoice?

Thus, it is important to collaborate with the developer to understand what the considerations and technical constraints are to build the feature and to be specific (to a certain extent) so that developers won’t undermine the actual efforts needed.

That concludes our 10 lessons. If you have any tips or best practices for designing valuable products, please share them in the comment section 👇

The post 10 Lessons from Designing A Product appeared first on Stampede: the strategic design & technology company.

Design

This goes without saying, having a great design attracts users to come visit the website. Its design is essentially your store front and if your users do not like how it looks, chances are they’ll be hitting the back button and going elsewhere. Web users also tend to be also very trendy, with all the exposure they have with other pretty looking websites. Because of this, your customers will likely prefer to see a modern-looking website than let’s say, a website that looks like it was from the 90s.

Speed & Reliability

Depending on the size of your online store, load speed and down times will mean a lot to your users. E-commerce sites such as Amazon has a lot of products, even more is the amount of traffic the website gets. If you’re online store has a fair number of things to sell and have many users accessing it, then you’ll want to make sure that the website loads quickly. To your customer, waiting for a page to load can be a daunting endeavor. Your website needs to be in a state where the user can get the content they want, at the time they request it.

What’s worse than a slow website? Downtime. Nobody likes getting their journey cut short due to unforeseen circumstances. To your users, down time is a big no-no and can leave a very strong negative impression on your brand. Avoid this from happening at all costs.

Product Catalog

You could be selling a handful of products or a warehouse’s worth, the most important thing is if your customer can actually find what they are looking for. Like how real brick-and-mortar stores have shelves to store wares and aisles to categorize types of products, online stores need to be able to sort products based on understandable categories. This will enable the user to be able to browse and search what they are looking for. To your customer, this part of their user journey is important because if they become frustrated by not being able to find what they want, chances are they’ll end up leaving your website discontented.

Product Details

They have nothing else to rely on other than the details that you are providing them.

One of the biggest differences in going to a store to buy something versus shopping online is that when you’re on the web you can’t actually see what you’re buying, at least not physically. This is why product details are very important to the online shopper. They have nothing else to rely on other than the details that you are providing them. If abundant and reliable information is provided to the customer, chances are they will feel more confident in what they’re looking at and will likely lead to a sale.

Clear product details consists of:

• Product images from various angles
• Description of the product
• Size & dimensions

Shipping Charges

For anyone who is familiar with buying things online, shipping charges become a norm. Your customers know that there will be shipping charges but the most important thing is to clearly show how much the shipping costs are. No one likes to see surprise costs at the tail end of their buying journey. For whatever your customer is paying, the total price should be broken down and itemized clearly on what they are actually paying for, this also includes the shipping costs.

Ease of Payment

Payment is the final step for your customer before they can begin waiting for what they’ve bought to arrive on their doorstep. The payment step is also one of the phases where a large number of drop-offs can happen and it all falls back to your e-business payment model. Two common types of payment methods for online purchases are credit card payment or using payment gateways. Your payment process should be fairly straight-forward using secured methods of payment. This is actual money that users are prepared to pay, the minute they notice that it seems not secure, the likely outcome will be the user dropping-off from the transaction.

Security

Users expect their data to be protected and encrypted to prevent identity theft…

To your customer, security is another important factor on your website. Users expect their data to be protected and encrypted to prevent identity theft, and this is where SSL certificates come in. SSL allows sensitive information such as credit card numbers, social security numbers, and login credentials to be transmitted securely. If your e-commerce website has an SSL certificate that is active, users will be able to tell by seeing a green padlock at the address bar of their browsers. This tells users that they are protected and can go ahead with shopping securely.

After Sales

When a customer has completed a prior transaction in the past, sending them an online newsletter can sometimes be a treat. Based on what they’ve purchased, you can promote items that the customer may like. This will act as a reminder to the user that your website is still around and attract them back for the next shopping spree.

By understanding what the customers wants, you can tailor a personal experience to said customers which will in turn make them want to revisit your online store.

The post What Do Customers Look For In Your Online Store? appeared first on Stampede: the strategic design & technology company.

From “me, me, me” to “I care about you”

There used to be a time when brands used to be implementing “me, me, me” approach in the email marketing campaigns. Some email marketing mistakes as ran through by Don Sadler, which rang true as early as 2004:

“It’s all about me.”

Many companies still don’t get the fact that it’s not all about them. E-newsletters that get bogged down in how great the company is, how many products they have, how long they’ve been in business, etc., etc., are doomed to failure, because people just don’t care about that. Readers want to know what’s in it for them.

No clear target audience.

You simply can’t be everything to everybody. As noted above, the tighter and more narrow your niche and content, the better.

Way too long.

With e-newsletters, people want to quickly get to the information that’s going to help them, digest this and then move on. If they know they can read your e-newsletter and get something of value in a couple of minutes, rather than having to slog through page after page, they’re much more likely to add you to their “must read” list.

The “me, me, me” approach sure  works after a while, but as according to Dale Carnegie:

To be interesting, be interested.

Today’s email marketing campaigns seem to move more than just a ‘campaign’. Most of the products and brands send weekly email newsletters detailing on their opinions, their work culture and things they find interesting enough to share – something very personalised and with a more human touch.

This gets customers to warm up more to them, knowing that there are real people behind these products working hard to give the best for them, and in return, they would appreciate it more.

Email newsletters for everyone

Recently I have been noticing an increase of individuals creating personal, curated email newsletters. To be honest, some of them are really, really good. Again, with easy-to-use platforms such as Mailchimp and Tinyletter (I have been contemplating on creating my own, but could not find great topics to share). It is really interesting to see this evolution, of how some methods that were once accessible to companies are now open to interesting individuals ready to share their valued opinions.

I love the Internet.

Lists of list

Here are some of the e-newsletters I have been subscribing and they are all my favourites. Feel free to add your recommendations in the comments!

UI/UX

• UX Design Weekly
• UX Thought of the Day
• Mailchimp UX
• Interaction Design Foundation

Web Design and Development

• WDRL

Companies and Brands

• Asana
• BigCommerce
• Buffer
• Basecamp – Signal v. Noise
• Invision
• The Great Discontent

Technology, Culture, Music etc.

• Real Future
• Meanwhile
• Experience Innovation
• JK Glei
• Good Reads

Miscellaneous (Writing, Ideas, Productivity, Cats!)

• A Working Letter
• The Modern Desk
• The Thread
• Weightshifting
• Buzzfeed
• Think Clearly

In the meantime, be sure to also subscribe to our Stampede Weekly to receive latest and curated web topics, from best practices and UX ideas to design inspirations & productivity tips!

(Thank you Wired for the photo.)

The post The Evolution of E-Newsletters appeared first on Stampede: the strategic design & technology company.

The issue occurs when you’ve reached the bottom of a very tall panel, and you clicked on the next panel switch to open the respective panel (which happens to be a very tall panel too). The scrollbar will stay at the same position while this new panel opens and the last panel closes and ends up putting your view at the bottom of the content of the new panel.

I have created a quick demo of this issue here. I’m using the awesome jQueryUI library to produce the accordion by the way. Click on the panel switch for Section 2 to expand it. Then scroll down until you see the panel switch for Section 3 and click on it. You will have to scroll up to get to the beginning of the content of Section 3. Not very nice.

To improve the navigation, a simple auto-scroll feature can be added to the accordion that brings the browser viewport to the top of the opened panel. In jQueryUI, we can take advantage of the activate event hook to trigger the auto-scroll. As for the scrolling itself, we can simply use jQuery’s animate function to add a smooth scrolling effect for a nice touch.

The final accordion code is as follows.

$( "#accordion" ).accordion({
	heightStyle: "content",
	collapsible: true,
	active: false,
	activate: function( event, ui ) {
		if(!$.isEmptyObject(ui.newHeader.offset())) {
			$('html:not(:animated), body:not(:animated)').animate({ scrollTop: ui.newHeader.offset().top }, 'slow');
		}
	}
});

Check out the updated demo here.

Let me know what you think of this workaround. Have a nice day!

The post Improving navigation on large accordion content panels appeared first on Stampede: the strategic design & technology company.