Heuristic Evaluation (Quiz 5)

Question 1

Heuristic Evalution

Answer 1

A rational or logical (as opposed to empirical) expert review method where heuristics are used to find and highlight usability problems.

Involves a small team of evaluators to evaluate an interface based on recognized usability principles.

Question 2

Heuristics

Answer 2

“rule of thumb”

Question 3

HE In a Nutshell

Answer 3

1. Individual Evaluation
2. Group Process
3. Write a report

Question 4

Individual Evaluation

Answer 4

Each evaluator compares design with known usability principles of rules of thumb (heuristics)

Writes down all the problems in Usability Aspect Report (UAR)

Question 5

Group Process

Answer 5

1. Group combines the problems

2. Assigns severity ratings

Question 6

HE method

Answer 6

Inspection method

Question 7

Usability Testing (UT) method

Answer 7

Empirical method

Question 8

HE people involved

Answer 8

Best with 3-5 evaluators, one manager

Question 9

UT people involved

Answer 9

Best with 5 users, one experimenter

Question 10

Should HE evaluators be left without help?

Answer 10

Evaluators should NOT be left without help

Question 11

Should UT users be left without help?

Answer 11

Users are left without help to struggle

Question 12

Duration of HE sessions

Answer 12

1-4 hrs/session

Question 13

Duration of UT sessions

Answer 13

0.5-1 hr/session

Question 14

How does HE find problems?

Answer 14

Finds hardest-to-find problems, but may miss common problems

Question 15

How does UT find problems?

Answer 15

UT finds most common problems, including conceptual model problems

Question 16

Neilsen’s 10 Heuristics

Answer 16

1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of user
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and communication

Question 17

1. Visibility

Answer 17

“The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.”

Refers to both visibility of system status and use of feedback.

Any time the user wonders what state the system is in, or the result of some action, this is a visibility violation.

Question 18

2. Real World Match

Answer 18

“The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.”

Refers to word and language choice, conceptual model, metaphor, mapping, and sequencing problems.

Question 19

3. User In Control

Answer 19

“Users often choose system functions by mistake and will need a clearly marked ‘emergency exit’ to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.”

Not just for navigation exits, but for getting out of any situation or state.

Question 20

4. Consistency

Answer 20

“Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.”

Question 21

Internal consistency

Answer 21

consistency throughout the same product.

Question 22

External consistency

Answer 22

consistency with other products in its class

Question 23

5. Error Prevention

Answer 23

“Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.”

Try to commit errors and see how they are handled. Could they have been prevented?

Question 24

6. Recognition not Recall

Answer 24

“Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.”

Deals with visibility of features and information (where to find things), whereas #1 deals with visibility of system status and feedback (what is going on).

Problems with affordances may go here

Question 25

Should the user ever carry a memory load?

Answer 25

no

Question 26

Example of problem with affordance

Answer 26

hidden affordance means you have to remember where to act; false affordance means you have to remember it is a fake!

Question 27

7. Flexibility and Efficiency

Answer 27

“Accelerators—unseen by the novice user—may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.”

Question 28

Examples of Flexibility and Efficiency

Answer 28

Concerns anywhere users have repetitive actions that must be done manually. Also concerns allowing multiple ways to do things.

Question 29

8. Aesthetic Design

Answer 29

“Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.”

Question 30

Issues with Aesthetic Design

Answer 30

1. clutter 2. overload of visual field 3. visual noise 4. distracting animations ...etc.

Question 31

9. Error Recovery

Answer 31

“Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.” Error prevention (#5) is about preventing errors before they occur. This is about after they occur.

Question 32

10. Help and Documentation

Answer 32

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large. This does not mean that the user must be able to ask for help on every single item. Remember, this is different from a “user manual.”

Question 33

Individual Evaluation

Answer 33

Briefing (HE method, Domain, Scenario) Evaluation by each individual. Passes through the interface: 1. Read all of the heuristics and their short descriptions beforehand. Queue them up in your mind. 2. Have a cheat sheet of the 10 heuristics in front of you. 3. Inspect flow. Inspect each screen, one at a time against heuristics. 4. Make a general pass, getting a feel for the interaction. Note obvious violations. 5. Make a second detailed pass, focusing on things that caught your attention earlier. HE is not a single-user empirical test

Question 34

More than One Evaluator

Answer 34

1. A single person will not be able to find all usability problems 2. Different people find different usability problems 3. Successful evaluators may find both easy and hard problems

Question 35

Evaluator Findings

Answer 35

1. Multiple evaluators independently produce a list of usability problems. 2. Aggregate findings into a single list of problems and the heuristics violated. Eliminate redundancies and clarify. 3. Independently review the list and assign a severity rating to each problem. 4. Create a summary report is that includes the average severity rating for each problem. The evaluators and design team then go through a debriefing session, discuss the problems, potential fixes, and add fix ratings to the summary report. 5. UI Redesign / Next Prototype

Question 36

Severity and Fixability Ratings

Answer 36

Used to allocate resources to fix problems and estimates of need for more usability effort. Combination of frequency, impact, and persistence

Question 37

Components

Answer 37

Combination of: 1. Frequency - Will the problem be common or rare? 2. Impact - Easy or difficult for users to overcome? 3. Persistence - One time that users overcome once they know about it, or repeatedly bothered, even if they know about it?

Question 38

How do we deal with persistence if problem leads users to abandon the application?

Answer 38

Do not rate low persistence if problem leads users to abandon the application.

Question 39

Advantages of HE

Answer 39

"Discount usability engineering" 1. Don't need to identify tasks, activities 2. Can identify low-cost, high-value fixes 3. Can expose problems user testing doesn't expose (e.g. with expert functions) 4. Provides a language for justifying usability recommendations 5. Can be used with any level of prototyping

Question 40

Disadvantages of HE

Answer 40

1. Non-empirical, thus un-validated 2. Unreliable (different people identify different problems) 3. Needs usability experts 4. Problems unconnected with tasks 5. Heuristics may be hard to apply to new technology 6. Coordination costs