§9- Heuristic Evaluation & Cognitive Walkthrough

Question 1

Bits of waterfall model relevant to HE and CW?

Answer 1

• evaluating and refining requirements

Question 2

Recap issues about usability

Answer 2

1. Ease of learning
2. Recall: remember how to use next time
   3: Productivity: execution time
   4: Minimal error rates: feedback on error, recovery
3. Satisfaction: user confident of success

Question 3

Benefits of ``discount usability testing’’ compared to formal usability testing

Answer 3

1. fewer resources and time than formal usability testing

Question 4

Three examples of discount usability testing

Answer 4

1. lofi prototyping
2. heuristic eval (heuristics, severities, process)
3. cognitive walkthrough

Question 5

Purpose of HE + high level overview

Answer 5

1. find usability problems in a user interface

2. 5 evaluators is optimal, check compliance with usability heuristics, compile problems, inform redesign

Question 6

List the ten usability heuristics

Answer 6

1. visibility of system status
2. match sys real wrld
3. user control, freedom
4. consistency, standards
5. error prevention
6. recognition rather than recall
7. flex/efficient use
8. aesthetic, minimalist design
9. help users recognise and recover from errors
10. help, documentation
    + skills
    + privacy,
    + pleasurable interaction/respectful

Question 7

Outline the steps needed to perform heuristic evaluation

Answer 7

1. pre eval training: train assessors about problem domain, scenarios
2. evaluate: go through UI according to scenarios, at least twice
3. collate
4. rate severity
5. feedback into next iteration of design

Question 8

issues related to severity ratings + meanings

Answer 8

1. determined from frequency, persistence, impact of problem
2. calculated after evaluations are complete
3. decide between more assessment and/or redesign
   - 0: don’t agree that it is a problem
   - 1: cosmetic
   2: minor usability problem
   3: major usability problem + important to fix
   4: usability catastrophe + imperative to fix

Question 9

H1 Visibility of System Status

Answer 9

Keep users informed about what is going on through appropriate feedback within reasonable time
- longer delays – progress bars

Question 10

H2 Match to real world

Answer 10

Speak the users’ language
 Follow real world conventions
e.g. itunes, photoshop tools

Question 11

H3 user control and freedom

Answer 11

Clearly marked “exits” for mistaken choices  undo/ redo
 Do not force down fixed paths

e.g home button

Question 12

H4 consistency and standards

Answer 12

Consistency within and between applications
- similar objects/terminology for similar actions

same primary menu options for Word, Excel, PowerPoint

Question 13

H5 error prevention

Answer 13

What is better than good error messages is a careful design, which prevents a problem from occurring in the first place

• e.g. PIN entry fields, google autocomplete

Question 14

H6 recognition rather than recall

Answer 14

Minimize user’s memory load
 Make objects, actions, options, and directions visible or
easily retrievable

• e.g. adobe photoshop filter thumbnails

Question 15

H7 flexibility for efficient use

Answer 15

accelerators – keyboard shortcuts,
allow tailoring – macros
Support frequent tasks and tasks
with high cognitive load – copy paste

Question 16

HE problems exam question points

Answer 16

1. which heuristics are violated and why
2. what is the severity rating
3. why - what is the effect on the user
   - how would you fix the problem

Question 17

H8 aesthetic and minimalist design

Answer 17

1. Draw user to focus on main subject at hand
2. categorise repetitive information into relevant sections
   - split long menus up
   - toolbars with icons/tooltips

Question 18

H9 help users recognise and recover from errors

Answer 18

1. help users recognise when an error has occurred
2. error messages in plain language to precisely indicate the problem
3. constructively suggest a solution

Question 19

H10 help and documentation

Answer 19

1. should be easy to search
   - focused on user’s task
   - list of concrete steps to carry out
   - not too long

Question 20

Drawbacks of HE

Answer 20

1. not task focussed
   - may be too high level
2. not using actual users
3. not rigorous

Question 21

CW-why

Answer 21

1. question assumptions about what users will be thinking
2. identify obscure/missing controls
3. find places that have inadequate feedback
4. suggest difficulties users may have with labels or prompts
5. identify problems users have on first use without training
6. not useful for evaluating usability over time (learning, speed of transition from beginner to intermediate)
7. assesses learnability of UI
8. identifies specific problems rather than general problems
9. no need for users to get involved

Question 22

Preparation for CW

Answer 22

1. description of prototype
2. task description
3. list of actions
4. idea of users, experience they bring

Question 23

5 steps of cognitive walkthrough

Answer 23

1. define inputs
2. get analysts
3. step through action sequences for each task
4. record important information
5. revise UI

Question 24

Step 1 : define inputs

Answer 24

1. who are the users,
2. what are the tasks
3. what are the action sequences for the tasks
4. have prototype/description of the interface ready

Question 25

Step 2: get analysts

Answer 25

1. don't need actual users 2. could imagine behaviour of entire class of users. 3. bear in mind cognitive limitations when performing walkthrough

Question 26

Step 3: step through actions

Answer 26

1. will users know what to do? 2. will users see how to do it? 3. will users understand from the feedback whether their actions are correct or not?

Question 27

Step 4: record important information

Answer 27

1. user knowledge just before and after each action 2. assumptions about users 3. side issues, design changes 4. credible success or failure story -- why would a user select or not select the correct action

Question 28

Step 5: revise UI

Answer 28

1. if user failed to select right action: eliminate, add prompt, change to make more visible 2. user does not know that correct action is available: make action more obvious 3. user does not know which action is correct: label controls based on user knowledge, check whether sequence of actions is natural/or seems awkward 4. if user can't tell things are going OK - give user feedback, say what happened in response to their action

Question 29

Problems found by CW

Answer 29

1. severe problems with usability 2. content related problems (since requires actual use of the interface to solve a goal) 3. scope -- more specific (thereby arguably superficial) than general

Question 30

Issues with CW

Answer 30

1. cannot evaluate all tasks, users may have different action sequences 2. each task is evaluated separately; what about cross task interactions 3. task free user centred method needed to catch problems that CW may have missed e.g. HE