evaluation and testing Flashcards

1
Q

evaluation

A

the process of assessing a design or product to find out what you need to change and why

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

formative evaluation

A

assessing iterations of the product

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

summative evaluation

A

assessing the final product

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

analytical evaluation

A

assessing the prototype without users but with the design team and external evaluators

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

what are two reasons for doing analytical evaluations before empirical

A

to get a systematic review which can identify big problems
user evaluation is expensive so you cant do that multiple times

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

empirical evaluation

A

collecting feedback from users

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

what are two reasons for using empirical evaluations over analytical

A

the best way to get insight on the product as analytical evaluations can only answer some questions
designers know how to use their product but not how others will use it

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what are three types of user studies

A

lab, field and usability studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

lab studies

A

focus on user performance and experience via controlled experiments that compare interfaces and systems
give imperial insight and can zoom in on a specific thing

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

field studies

A

evaluate the system irl in its natural context to give the best insight

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

usability tests

A

identify usability issues via asking users to complete a series of tasks
used for improvement of the design and validation that users can achieve their goals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

usability issues

A

problems uses encounter towards achieving their goals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

tasks

A

key functionality that the system performs usually based on scenarios

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

what are some criteria for tasks

A

should be a specific sequence of interactions that lead to completion
should be clear what constitutes as completion or failure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

objective data

A

factual and usually collected via data logs, observation and measurements

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

subjective data

A

collected via communication
e.g. SUS scores

17
Q

usability evaluation

A

controlled study with users and clear tasks that measures effectiveness, efficiency and satisfaction

18
Q

effectiveness in usability evaluation

A

can users complete the tasks that they need to
when and how do they fail and have errors

19
Q

efficiency in usability evaluation

A

how long does it take to complete tasks
measuring time

20
Q

satisfaction in usability evaluation

A

how good was the user experience

21
Q

what type of data does usability evaluation produce and why isnt this good

A

quantitative
it doesnt telllus why

22
Q

qualitative data

A

provide more details on the issue and can be analysed to pick out what’s useful

23
Q

pareto principle

A

20% of the effort will generate 80% of the result

24
Q

what is a benefit of prototyping in usability testing

A

can simulate features not created yet
e.g. a human can play the role of a computer

25
Q

wizard of oz study

A

a human controls the system responses

26
Q

think aloud protocol

A

the user speaks their thoughts out loud so we can get their pov

27
Q

how many users do we need usually

A

> 75% of issues will be found with 5 participants so its better to have multiple small iterations more than one big one

28
Q

what are the 4 types of analytical evaluation

A

expert
heuristic
cognitive
model based

29
Q

expert evaluation

A

a hci expert acts as a user
the inspection is based on a system walkthrough and they find problems and suggest improvements

30
Q

heuristic evaluation

A

low cost usability evaluation where one or more evaluators inspect the ui against a set of heuristics
they record errors that map to them and are graded by severity

31
Q

cognitive walkthrough

A

formal inspection of a ui learnability conducted by an expert acting as a user

32
Q

what happens during a cognitive walk through
what questions are asked and what ideas do they map to

A

for each scenario write down the action sequence and for each action ask the four questions
mental model; does the user understand that this step is needed
visibility; will they notice that this action is available
affordances; will they recognise the action as the correct one
feedback; if they perform the action will they know that they’ve made progress towards their goal

33
Q

model based evaluation

A

provide analysis of effort and complexity
eg.fitts law, klm