4.3 Usability Test Plan

Question 1

Before beginning any test, its important to create a …?

Answer 1

Usability test plan

Question 2

What is a usability test plan?

Answer 2

A document that outlines the scope, goals, and logistical details of your session in advance

Question 3

What are the main elements of a basic test plan?

Answer 3

Introduction, Background, Goals, Test objectives, Methodology, Participants, Schedule, Sessions, Equipment, Metrics, Scenarios/Test Tasks, and Script

Question 4

What are test objectives?

Answer 4

More specific than goals and highlight the particular features we’ll be testing in order to achieve our test goal

Question 5

What are test goals?

Answer 5

An overview of your research and what we hope to get out of it

Question 6

What are Jakob Nielsen’s Five Components of Usability?

Answer 6

Satisfaction, Learnability, Efficiency, Memorability, Errors

Question 7

How pleasant is it to use the design?

Answer 7

Satisfaction

Question 8

How easy is it for users to accomplish basic tasks the first time they encounter the design?

Answer 8

Learnability

Question 9

Once users have learned the design, how quickly can they perform tasks?

Answer 9

Efficiency

Question 10

When users return to the design after a period of not using it, how easily can they re-establish proficiency?

Answer 10

Memorability

Question 11

How many errors do users make, how severe are these errors, and how easily can they recover from the erros?

Answer 11

Errors

Question 12

Learnability

Answer 12

refers to how easily users can accomplish basic tasks the first time they encounter a design

Question 13

Learnability can be measured by looking at ….

Answer 13

success rate

Question 14

Success rate measures what?

Answer 14

the percentage of tasks that users complete correctly

(S + (P x .5)) / total number of task attempts = Success Rate %

Question 15

Efficiency

Answer 15

Once users have learned the design, how quickly can they perform tasks?

Question 16

Three-Click Rule

Answer 16

stating that a user of a website should be able to find any information with no more than three mouse clicks.

Question 17

Efficiency is measured by

Answer 17

time on task—the average amount of time it takes for participants to complete a given task.

Question 18

Memorability

Answer 18

When users return to a design after a period of time not using it, how easily can they re-establish proficiency? Memorability is an especially useful component to understand when designing something for occasional use.

Question 19

How do you measure memorability?

Answer 19

Memorability is trickier to measure because it requires revisiting a task with the same participants more than once. You’ll benchmark memorability using both your “success rate” (learnability) and your “time on task” (efficiency) over time. To do this, follow up with the same participants multiple times to measure the difference in the success rate and average amount of time on task when they revisit your prototype. Following up with participants to measure memorability is not required for your project, but feel free to explore it if you have participants willing to volunteer time for an additional test.

Question 20

Errors

Answer 20

How many errors do users make, how severe are these errors, and how easily can they recover from the errors?

Question 21

How do you measure errors?

Answer 21

There are various ways to categorize and rank the severity of errors, but for consistency and clarity’s sake, we’ll refer to an adapted version of Jakob Nielsen’s rating scale—one of the most commonly used metrics for usability errors. We recommend that you use this scale in your test:

0 = I don't agree that this is a usability problem at all
1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix and should be given high priority
4 = Usability catastrophe: imperative to fix before product can be released

Question 22

Satisfaction

Answer 22

How pleasant is it to use the design? This is measured by your product’s satisfaction. Satisfaction can be one of the hardest components to measure, quantify, and report. That being said, there are a few qualitative and quantitative methods that can help us understand customers’ satisfaction—satisfaction surveys and follow-up questionnaires.

Question 23

Test plans help designers define

Answer 23

the focus, scope, and logistics of carrying out a usability test and are a critical part of the testing process.

Question 24

Which of the following is an element to cover in a basic usability test plan?

a. BRD
b. Persona
c. Equipment
d. Style guide

Answer 24

c. Equipment

Question 25

You’ve come up with your research goals, and now you must get more specific, breaking down exactly what you wish to look for with each particular feature. What is this called?

a. Test goals
b. Test scenarios
c. Test schedule
d. Test objectives

Answer 25

d. Test objectives

Question 26

According to Nielsen’s Five Components of Usability, which of the following components can be used to measure how enjoyable it is to use a product?

a. Learnability
b. Memorability
c. Efficiency
d. Satisfaction

Answer 26

d. Satisfaction

Question 27

Mo Stevens has completed his initial designs and would like to plan some tests. Before doing so, he lays out a document with information about the scope, goals, and details of his tests. What is this document called?

a. Design layout plan
b. Product road map
c. User journey map
d. Usability test plan

Answer 27

d. Usability test plan

Question 28

You’re conducting a test in which the question you’re attempting to answer is: “When users return to our app after one week of not using it, how easily can they re-establish proficiency?” Which of Nielsen’s Five Components of Usability are you testing?

a. Errors
b. Memorability
c. Satisfaction
d. Learnability

Answer 28

b. Memorability