Final Flashcards
2 different metric types
observable + quantifiable
3 common questions with metrics
- “Which metric shall I (we) use?”
- “How shall I (we) obtain the components needed to calculate it?”
- “Is this metric reliable enough to give a realistic picture of the degree to which my (our) system is usable (or not)?
3 general metrics names
Performance Metrics
Issue-Based Metrics
Self-Reported Metrics
9 performance metrics names
- Completion rate (task success, effectiveness)
- Task time
- Errors
- Efficiency (Page views/clicks)
- Lostness
- Conversion rate
- Learnability
- Eye-tracking
- Biometric data
1 combined metric name
Single usability metric
3 self-reported metric names
- Task-level satisfaction (self-report)
- Expectations
- Test-level satisfaction (self-report)
1 issue-based metric name
Usability problems
Performance metrics (there ae six of them)
- Task Success (Completion Rates)
- Binary Success
- Levels of Success - Time on Task
- Errors
- Effectiveness
- Efficiency
- Learnability
Effectiveness formula explanation
- measure completion rate.
- fundamental usability metric/completion rate is calculated by assigning a binary value of ‘1’ if the test participant manages to complete a task and ‘0’ if he/she does not.
- average Task Completion Rate is 78%
Effectiveness formula
(Number of tasks completed/Total number of tasks) X 100%
Effectiveness example. 5 tasks and a user completes 3 of them.
3/5 X 100% = 60%
3 Levels of success
Complete Success
Partial Success
Failure
How to resolve a task when a user is not successful
Tell the users at the beginning of the session that they should continue to work on each task until they either complete it or reach the point at which, in the real world, they would give up or seek assistance (from technical support, a colleague, etc.).
Time on task explanation
Time on task (sometimes referred to as task completion time or simply task time).
Caveat: Sometimes, slower is better
Ex: They are truly engaging with the website
4 steps to make efficiency quantible
- Identify the action(s) to be measured
- Define the Start and end of an action
- Count the actions
- Actions must be meaningful
Lostness formula
N: # of DIFFERENT webpages visited
S: total # of webpages (duplicate included)
R: minimum # of pages required to visit
L = sqrt(N/S-1)^2 +(R/N-1)^2
2 types of rating scales
likert and semantic differential scales
4 types of self-reported metrics
- Post-Task ratings
- Post session ratings
- using sus to compare designs
- online services
Likert scale definition
5 point rating scale following:
1. Strongly disagree
2. Disagree
3. Neither agree nor disagree
4. Agree
5. Strongly agree
Semantic Differential scales definition
Will have a scale with two values on each side. These two values will be opposites such as: weak and strong, ugly and beautiful, cool and warm. example below
weak o o o o o o o strong
SUS meaning and definition
System Usability Scale - consists of a 10 item easy questionnaire with five response options for respondents; from Strongly agree to Strongly disagree
10 questions used in SUS
- I think that I would like to use this system frequently.
- I found the system unnecessarily complex.
- I thought the system was easy to use.
- I think that I would need the support of a technical person to be able to use this system.
- I found the various functions in this system were well integrated.
- I thought there was too much inconsistency in this system.
- I would imagine that most people would learn to use this system very quickly.
- I found the system very cumbersome to use.
- I felt very confident using the system.
- I needed to learn a lot of things before I could get going with this system.
CSUQ meaning and definition
Computer System Usability Questionnaire - has 19 questions about usability; from strongly disagree to strongly agree
3 severity ratings
low, medium, high