TL;DR—integrating continuous user testing and feedback into a development workflow means faster and more focused incremental iteration towards getting the most value from a product or service.

User testing and feedback means quicker validation—or rejection—of decisions, helping to ensure your project is moving in the right direction. The data gathered from real users can be applied programmatically to help guide discussions and resolve internal disputes on design and architecture decisions.

Mis-quotes and Missed Opportunities

"If I had asked people what they wanted, they would have said faster horses."

PROBABLY NOT HENRY FORD

While it's a useful soundbite of what seems like an astute observation, it doesn't really stand up to closer scrutiny. Scratch the surface and a far more likely response would've been "better transport". Either way, Ford's engineers would have had a starting point. What that quote really does is exemplify the science of creating useful testing frameworks, the art of interpreting user feedback.

Users understand their problems. They are certainly capable of communicating enough information (given the right framework) and demonstrating, through their behaviour, how you can innovate to drive growth and increase the value of your offering.

The Secret to Valuable Feedback? Ask the Right Questions...

We'll say it again: users understand their problems. They shouldn't be expected to lead design conversations but they will know what they want. The challenge is articulating and interpreting it in a constructive manner. When designing mechanisms for testing and feedback, ask questions that allow you to extract specific insights for your product or service.

You should also enquire about their challenges. Not through specific, feature-based questions—most SME customers aren't concerned with, or have sufficient knowledge of, how to implement features. Their challenges are more universal—increase turnover, reduce inefficiencies, save money, meet legislative requirements, and so on. Instead, gather feedback through established frameworks—multiple choice questionnaires or Likert scales—and gain insight on user behaviour using automated analytics tools like Fullstory or Hotjar.

Using multiple-choice questions, with the space for "other" and the opportunity to write a response, is a great method. But beware of giving too many choices or worse, allowing only free text. This will result in poor-quality, unstructured feedback, but there's good value in providing the option to bring up something you didn't anticipate. Just be sure to be able to quantify feedback, especially from multiple sources.

So we've established that users are a valuable source of information. Now, let's dive in and explore how testing and feedback can boost agility, avoid costly rework, and help you deploy with confidence...

Why Do You Need Continuous User Testing?

Continuous user testing allows you to:

• Validate and improve design and development in real time so valuable resources aren't wasted on building something nobody wants.
• Make decisions based on data from actual users, not opinions or abstract concepts.
• Get useful feedback at every step of the process by inviting relevant stakeholders to participate in user testing and feedback sessions which can be recorded to review later.

What Are The Benefits Of Continuous User Testing?

Continuous user testing means we can:

• Make better product decisions which directly impact business metrics like user acquisition, retention, conversion rates, etc.
• Quickly resolve internal disputes by determining what works best instead of debating it. This allows teams to focus on execution and collaboration, rather than disagreements.
• Speed up incremental iteration towards getting the most value from a product or service.
• Reduce internal politics by having solid data to justify decisions and serve as a tie-breaker for contentious issues.
• Ensure you are building something users want, and gaining more granular insights into specific design problems, how to fix them, and who's responsible for doing so.
• Better the user experience because the correct things get built in the first place and then fine-tuned on an ongoing basis post-launch, via a backlog, based on feedback captured through continuous testing.
• Create higher team morale since there is consensus, rather than confusion, about what works best at any given time. This has the added bonus of reducing employee turnover.

Now we know what user testing is and why we do it, let's dig a little deeper into methods and highlight some best practices.

What is A/B Testing?

A/B Testing is a technique for experimentally comparing two versions of a feature of a product or service.

It's most often used in digital products and services, such as web applications and websites. An example could be to determine which version of a landing page is more effective at converting visitors into customers, or achieving other goals that benefit the business, such as which metrics should be presented on a dashboard to give the best insights for business intelligence (BI).

An A/B test can be performed by randomly assigning users to either experience A or B (called random assignment) or by giving users alternative options and tracking their behaviour over time with each option (called an intentional design).

The goal of an A/B test is to make improvements to your product or service based on data gathered from actual user behaviour.

A Faster Feedback Cycle

For many projects, A/B Testing is the easiest way to track incremental changes in your product or service for immediate feedback—or to test new hypotheses about what people are likely to do next. Developers can write A/B tests with minimal effort, because it's usually based on percentages—small tweaks to the user experience (UX)—rather than wholesale user interface (UI) changes which can be resource-intensive and difficult to implement.

Thank you again XKCD. Always keeping us on our toes. Source: https://xkcd.com/1770/

Continuous A/B testing gives developers and designers more confidence in their decisions by giving them rapid feedback on how well they're doing—a failure means go back and change code; a success means move forward and release code. Having this kind of data allows decision makers to have more experiments running concurrently for faster learning, and A/B Testing can also help stakeholders better identify their most valuable users.

Structuring Feedback for A/B Testing

When thinking about A/B testing, it’s important to consider that A/B tests can be run on many different types of metrics. A metric that is valuable to the business might not be valuable to users, and vice versa. With this in mind, it's useful to gather user feedback on A/B tests (e.g., on which feature performs better) so you can make more informed decisions on how to structure future tests.

User feedback on A/B tests helps prioritise your backlog for future development by identifying what problems are most urgent or consequential to solve first—and by giving developers insight into how people will use new features when they're ready.

Continuous A/B testing has changed how we approach user support and feedback, as we can take advantage of the data we collect from tests to see whether there are problems with new features.

Creating a User Feedback Backlog from Continuous User Testing

Machine learning algorithms might be the future, but today's decision-making process still largely depends on input from humans. User testing is the voice of the customer (or client), and it's often our best source of information about how people will use new features once they're ready for deployment.

User testing is essential for building up a backlog of feedback related to the entire product or service. This gives us more data to make decisions about the future of the wider business, no matter what stage it's at.

User testing is a critical part of our design process—helping us validate our findings and giving us insight into how real people use the product. It helps separate good ideas from bad—and great products from mediocre ones.

In short, ask the right questions and use innovation to bridge the gap—just like Ford did all those years ago.

Bring value to your project and embed continuous user testing. Start a conversation with us today.