evaluation and testing Flashcards
evaluation
the process of assessing a design or product to find out what you need to change and why
formative evaluation
assessing iterations of the product
summative evaluation
assessing the final product
analytical evaluation
assessing the prototype without users but with the design team and external evaluators
what are two reasons for doing analytical evaluations before empirical
to get a systematic review which can identify big problems
user evaluation is expensive so you cant do that multiple times
empirical evaluation
collecting feedback from users
what are two reasons for using empirical evaluations over analytical
the best way to get insight on the product as analytical evaluations can only answer some questions
designers know how to use their product but not how others will use it
what are three types of user studies
lab, field and usability studies
lab studies
focus on user performance and experience via controlled experiments that compare interfaces and systems
give imperial insight and can zoom in on a specific thing
field studies
evaluate the system irl in its natural context to give the best insight
usability tests
identify usability issues via asking users to complete a series of tasks
used for improvement of the design and validation that users can achieve their goals
usability issues
problems uses encounter towards achieving their goals
tasks
key functionality that the system performs usually based on scenarios
what are some criteria for tasks
should be a specific sequence of interactions that lead to completion
should be clear what constitutes as completion or failure
objective data
factual and usually collected via data logs, observation and measurements
subjective data
collected via communication
e.g. SUS scores
usability evaluation
controlled study with users and clear tasks that measures effectiveness, efficiency and satisfaction
effectiveness in usability evaluation
can users complete the tasks that they need to
when and how do they fail and have errors
efficiency in usability evaluation
how long does it take to complete tasks
measuring time
satisfaction in usability evaluation
how good was the user experience
what type of data does usability evaluation produce and why isnt this good
quantitative
it doesnt telllus why
qualitative data
provide more details on the issue and can be analysed to pick out what’s useful
pareto principle
20% of the effort will generate 80% of the result
what is a benefit of prototyping in usability testing
can simulate features not created yet
e.g. a human can play the role of a computer
wizard of oz study
a human controls the system responses
think aloud protocol
the user speaks their thoughts out loud so we can get their pov
how many users do we need usually
> 75% of issues will be found with 5 participants so its better to have multiple small iterations more than one big one
what are the 4 types of analytical evaluation
expert
heuristic
cognitive
model based
expert evaluation
a hci expert acts as a user
the inspection is based on a system walkthrough and they find problems and suggest improvements
heuristic evaluation
low cost usability evaluation where one or more evaluators inspect the ui against a set of heuristics
they record errors that map to them and are graded by severity
cognitive walkthrough
formal inspection of a ui learnability conducted by an expert acting as a user
what happens during a cognitive walk through
what questions are asked and what ideas do they map to
for each scenario write down the action sequence and for each action ask the four questions
mental model; does the user understand that this step is needed
visibility; will they notice that this action is available
affordances; will they recognise the action as the correct one
feedback; if they perform the action will they know that they’ve made progress towards their goal
model based evaluation
provide analysis of effort and complexity
eg.fitts law, klm
