user studies Flashcards
framework
steps when designing an experiment
objective
purpose and criteria
variables
measurements and factors
design
structure setup and tasks
sampling
who is taking part and how many
analysis
conclusions being drawn
ethical guidelines
safety
voluntary and not coerced
informed about what the study entails
informed consent
filling out a form before the study begins to state that you know what youre doing
independant variables
what being controlled and manipulated
factors and levels
factors are manipulated across multiple levels and each combination creates a different test
dependant variables
what youre measuring
human behaviour and response
performance measures
what users do whilst using the system
user reported measures
how users feel about the system
how do we prevent learning time from skewing results
dividing the study into blocks letting you see if users improve over time as they get used to the system
design
structure of the experiment
factorial design
number of factors * number of levels
within subject design
every participant performs the same task
learning effect
may be better the 2nd time as they got to practice the first
fatigue effect
tired if the task is demanding
bored/ less attentive if the task is repetitive
what are the two order effects
learning and fatigue effects
between subject design
participants are put into groups and complete different tasks
task
what participants do in the study
typical tasks
represent the real application
abstract tasks
more control and observes how performance depends on conditions
how do we prevent order effect and what does that entail
counterbalancing; participants are divided into groups and are given the test conditions in different orders therefore the tasks are equally represented across participants
procedure
specific instructions that guide in completing the tasks and conducting the study
