Participants and Sample Size

“5 users will uncover approximately 80% of all usability problems”

– Nielsen Norman Group

https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Sample Size & Reliability

The more people find the same problem, the more reliable the data set becomes.

Multiple Rounds of Testing

Multiple round of testing on 5 different users covers more issues than a single round of testing with the same number of users.

User 1		User 6		User 11			User 1	User 9
User 2		User 7		User 12			User 2	User 10
User 3		User 8		User 13		>	User 3	User 11
User 4		User 9		User 14			User 4	User 12
User 5		User 10		User 15			User 5	User 13
							User 6	User 14
							User 7	User 15
80%		80%		80%			User 8	User 16
of issues		of remaining issues		of remaining issues			80%
							of issues

 

Selecting the right people

You need to select the participants based on the following criteria:

1. User Profiles
2. Personas
3. General Data about your User base

and factors:

1. The users understanding of the product.
2. The Familiarity with the Product or Type of Product.
3. Amount of overall Experience

And Recruiting criteria:

1. Sample Size – how many people can you effort to bring in to test.
2. Characteristics of the user – about the criteria being defined.
3. Number of Participants for Each Personas or Characteristics.

 

Tasks	vs.	Scenarios
Quick simple functions		Provide the partecipant context and direction
Easily completed without any background information given		Set the stage, tell the story
Find the Search button		Increase overall understanding
Where would one go to out information about specific terms		Help the user apply the situation to themselves
		Give them an action to perform

 

Usability Scenarios

1. Uses the vocabulary that the participant uses.
2. The actions are clear and precise.
3. Provides all the details needed to complete the goal.
4. Does not provide steps and definitely does not give away the answer.
5. You would like to buy a car, you would like to lease it for 36 months term with 1500 down. How would you do this?

Rating your results

a. Easy or under a minute to find/answer – – – —2 points

b. Difficult, takes 1-2 minutes to find/answer —-1 point

c. Fail, takes over 2 minutes to find/answer ——0 point