A product can be technically functional yet difficult to use.
Usability evaluation helps teams understand how effectively, efficiently and comfortably people can use a product to achieve their goals. It allows us to move beyond assumptions about the experience and observe what actually happens when real users interact with it.
The objective is not simply to ask:
Do users like the design?
A more useful set of questions is:
Can users achieve what they came to do? How easily can they do it? Where do they struggle, and why?
Formative and Summative Evaluation
Usability evaluation can take place throughout the product development process.
Formative Evaluation
Formative evaluation happens while a product or experience is still being designed.
Teams might test sketches, wireframes, clickable prototypes, or early versions of a product to identify problems and improve the design before making significant investment.
The objective is learning.
At this stage, teams might observe:
- Whether users understand what to do
- Where they hesitate or become confused
- Whether they can find important information
- Whether the proposed workflow matches their expectations
- Which assumptions about user behaviour appear to be wrong
Because the purpose is to learn and iterate, prototypes do not need to be highly polished.
In fact, testing an idea before it looks finished can make it easier to change direction when evidence suggests that the solution is not working.
Summative Evaluation
Summative evaluation is generally conducted on a more mature product or experience.
It focuses more heavily on measuring performance against defined usability objectives or comparing different versions of a product.
This might include:
- Task completion rates
- Time to complete a task
- Error rates
- Behavioural analytics
- Customer satisfaction measures
- Comparison with an existing product or benchmark
In practice, modern product teams often combine formative and summative approaches rather than treating them as completely separate activities.
Test at the Appropriate Level of Fidelity
The fidelity of a prototype should depend on what the team wants to learn.
A simple sketch may be enough to test whether users understand a proposed information structure. A clickable prototype might help evaluate a user journey. A working product may be required to understand real-world behaviour over time.
Low-fidelity prototypes do not necessarily require laboratory testing, and high-fidelity products do not always need to be tested “in the wild”.
Both moderated and unmoderated testing can take place remotely or in person.
The important question is:
What is the simplest experiment that will give us reliable evidence about the assumption we are testing?
Measuring Usability
Usability is often evaluated across several dimensions.
Effectiveness: Can Users Complete the Task?
The first question is whether users can successfully achieve their objective.
Measures might include:
- Task completion rate
- Error rate
- Number of failed attempts
- Need for assistance
- Accuracy of the outcome
A product that looks beautiful but prevents users from completing an important task has a fundamental usability problem.
Efficiency: How Much Effort Does It Take?
Once users can complete a task, we can examine how easily they can do it.
Possible measures include:
- Time on task
- Number of steps
- Number of interactions
- Frequency of errors
- Amount of unnecessary navigation
However, fewer clicks do not automatically mean better usability.
An additional step that provides clarity or prevents a costly mistake may improve the experience. Efficiency should therefore be evaluated in the context of the user’s goal rather than reduced to a single metric.
Learnability: How Quickly Can Users Understand the Experience?
Learnability describes how easily someone can begin using a product successfully.
We can observe whether users:
- Understand what actions are available
- Recognise familiar interaction patterns
- Require instructions or assistance
- Improve quickly after their first attempt
Comparing first-time performance with later attempts can help reveal how quickly users learn the system.
Memorability: Can Users Return and Use It Again?
Some products are used every day. Others may only be used occasionally.
For infrequently performed tasks, memorability becomes particularly important.
A user who successfully completed a process three months ago should ideally be able to return without having to relearn the entire experience.
Teams can evaluate this by observing returning users and comparing their performance with previous interactions.
Satisfaction: What Does the Experience Feel Like?
Usability is not purely about task completion.
A user might successfully complete a task while still finding the experience confusing, stressful or unnecessarily demanding.
This is why usability evaluation should also consider subjective experience.
Cognitive Experience
How much mental effort does the task require?
Are the steps intuitive?
Does the user need to remember information from one screen to another?
Are choices clear?
Does the interface match the user’s expectations?
Emotional Experience
How does the interaction make the user feel?
Users might experience:
- Confidence
- Frustration
- Uncertainty
- Satisfaction
- Anxiety
- Trust
Cognitive and emotional experiences are often connected. A confusing process can create frustration, while a clear and predictable experience can increase confidence and trust.
Quantitative and Qualitative Evidence
Good usability evaluation often combines different types of evidence.
Quantitative measures tell us what happened.
For example:
- 60% of users completed the task
- The average completion time was four minutes
- Users made an average of three errors
Qualitative research helps us understand why it happened.
Observation might reveal that users misunderstood a label. An interview might reveal that they expected the information to appear elsewhere.
The combination is more powerful than either approach alone.
From Usability Testing to Continuous Discovery
Usability evaluation should not be something that happens only before a product is launched.
In continuous discovery, teams regularly put ideas in front of customers to test assumptions and learn before making large investments.
This means usability evaluation can become part of an ongoing learning cycle:
Opportunity → Solution → Assumptions → Test → Evidence → Iteration
Imagine that a team creates a new workflow because it believes users will find it faster.
The question is not simply whether the interface is usable.
The team should also ask:
Is speed actually important to the customer?
Does this solution address the opportunity we identified?
Which assumptions are we testing?
What evidence would change our decision?
This distinction is important because a product can be perfectly usable and still solve the wrong problem.
Evaluate the Riskiest Assumptions First
Before running a detailed usability study, consider what needs to be true for the solution to succeed.
Perhaps users need to understand a new concept.
Perhaps they need to trust an automated recommendation.
Perhaps they need to change an existing behaviour.
Testing these assumptions early can prevent teams from spending time perfecting an experience that customers ultimately do not need or want.
Expert Evaluation
Not every usability problem requires direct user testing to identify.
Two established expert evaluation techniques are particularly useful.
Heuristic Evaluation
Evaluators review an interface against recognised usability principles or heuristics to identify potential problems.
This can be a fast and cost-effective way to identify obvious usability issues before testing with users.
Cognitive Walkthrough
A cognitive walkthrough examines a task step by step from the perspective of a user, particularly someone encountering the interface for the first time.
At each stage, the evaluator considers whether the user is likely to understand what action to take and recognise whether they are making progress.
These methods complement user research but should not be treated as substitutes for observing real users.
When Should You Stop Evaluating?
Usability evaluation is inherently iterative.
Research produces evidence. Evidence informs changes. Changes create new assumptions that may need to be tested.
The process might look like:
Design → Test → Learn → Iterate → Test again
The goal is not to achieve a theoretically perfect interface.
The goal is to reach a point where the experience enables users to achieve their objectives effectively, with an appropriate level of effort and a satisfactory experience.
And even then, learning does not necessarily stop.
As customer needs, behaviours and products evolve, new usability challenges emerge.
The most effective product teams, therefore, treat usability not as a final quality check, but as part of a continuous process of understanding customers and improving the experience.
Usability testing helps us understand whether users can use our solution. Continuous discovery asks the bigger question: are we building the right solution in the first place?
To learn more, check on:
Usability Evaluation 101 by Usability.Gov
Interaction Design – Chapter 15 – Usability Evaluation.
WQUsability – More than Easy to use.
Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated? by CHI 2000.
Usability 101 – Introduction to Usability – by Nielsen Norman Group
Data With Purpose 
Leave a comment