Lecture 11: Usability Evaluation Methods

Question 1

What is a heuristic evaluation?

Answer 1

A heuristic evaluation is a type of interface evaluation performed by an expert. It can be used in low and high fidelity prototypes; on buggy and more complete implementations.
There are four main stages to the evaluation:
1. Training
2. Evaluation
3. Severity Rating
4. Debriefing

Question 2

What is the first stage of the heuristic evaluation and what does it involve?

Answer 2

Training
The training stage involves a meeting between the design team and evaluators, where the application will be introduced and explain the user population, domain and scenarios.

Question 3

What is the second stage of the heuristic evaluation and what does it involve?

Answer 3

Evaluation Process
It involves FIVE steps:
1. Choose a set of heuristics to apply
2. Identity tasks to be evaluated
3. Go through the interface once; to get the feel of the system
4. Go through the interface again; to focus on usability issues.
5. For each task to be evaluated: assess against heuristics, list every problem you encounter and explain & justify the problem using heuristics.

Question 4

What are the FIVE things that must be done in order to write a good heuristic evaluation?

Answer 4

1. include positive comments as well as criticisms.
2. Be clear
3. Be specific
4. Justify every problem with a heuristics
5. List every problem you find.

Question 5

What is the third stage of the heuristic evaluation and what does it involve?

Answer 5

Severity Ratings
The severity of an issue is based on three factors:
1. Frequency - how often does it occur? (once, rarely, frequently)
2. Impact - how easy can the user overcome it? (small, medium, large)
3. Persistence - how often does the user face it ? (occasional, persistent)
Taking these three factors into account a scale is used to rate the severity of the issue:
1. Not a problem
2. Cosmetic
3. Minor
4. Major
5. Catastrophic

Question 6

What is the final stage of the heuristic evaluation and what does it involve?

Answer 6

Debriefing
This stage takes place after all evaluation and severity rankings.
All evaluators and observers meet with representatives from the design team
Opportunity to brainstorm for ideas and fixes to the problems, and a good opportunity to discuss positive aspects.

Question 7

What are the THREE advantages of the heuristic evaluation?

Answer 7

1. The heuristics focus the evaluators’ attention on particular issues.
2. Heuristic evaluation does not carry the ethical and practical issues / problems associated with inspection methods involving real users.
3. Evaluating designs using a set of heuristics can help identify usability problems with individual elements and how they impact on the overall user experience.

Question 8

What are the THREE disadvantages of a heuristic evaluation?

Answer 8

1. Choosing appropriate heuristics is extremely important; if the wrong set of heuristics is employed, certain usability problems may be overlooked.
2. Training evaluators takes time; Nielsen and Mack estimate the average training period is one week.
3. Problems identified by evaluators can be false alarms.