11 - User Studies and Experiments in HCI Flashcards
what is the difference between correlation and causation?
The correlation says, that there is a relation, but does not espicifies wether there is a direct causation and which is the direction of the relation. If A correlates with B, there are three possible cases:
* if the probability of A increases, than the probability of B increases.
* if the probability of B increases, than the probability of A increases.
* there is a unknown factor, the “Tertium Quid”, which contributes for the increase in the probability of both A and B.
Explain independent variable (IV), dependent variable (DV), covariantes, error variable and confounding variables
IV: The variable that you control (e.g UI version)
DV: The variable measured in the experiment (e.g number of clicks, time to reach activity)
covariates: What we know but don’t control (e.g. handedness, glasses,..)
error variable: random factors, e.g people are different in many ways
confounding variables: finding realtionship /results, that are not true. eg. numner of clicks with B is lower, but simply because users that tested B had more sleep.
Explain what is a Hypotheses and how H0 and H1 are used on user tests
Hypotheses is a formal statement about the relationship between the dependent and independent variable. Two Hypotheses are defined for this:
H0: There is no effect of the independent variable on the dependent variable.
H1: The independent variable has some effect on the dependent variable.
NHST are used to evaluate user tests. In this approach, if the probability is smaller than a threshold then reject H0. Thus H1 should be true.
You want to find out if app suggestions are a useful feature.
Write down:
▪ A concrete research question for your experiment
▪ The Null Hypotheses and Alternative Hypotheses
▪ Your IV(s) and DV(s)
▪ IV(s): What do you compare?
▪ DV(s): What do you measure?
▪ What could we use as a baseline design here?
research question: Does app suggestions decreases the time to find the right app?
H0: There is no effect of the app suggestion in the time needed for a user to find a app.
H1 : With app suggestions, a user needs less time in average to find the app he was looking for.
IV: UI(with or without app suggestion)
DV: Time user takes to find a app
basesline desing: avarage time a person needs to navigate trough and find a app.
Which type o errors can be found in hypothesis testing?
type 1: reject H0 although it is true.
type 2: fail to reject H0, although it is false
what are the advantages and disadvantages of between-subjects desgin and within-subject design?
in between-subject, each participant only goes through one experimental conditional. It helps to avoid people carry- over effects but the measured difference might be differences between people, not UI related.
in Within-subject, each participant goes through all experimental conditions. It helps to accounts for inter-individual differences, but possible task care-over and fatigue effects can affect the results(e.g. maybe people are slower because they are tired from the first task)
what is the definition of latin square in how it is used to balance user expirement?
A Latin square is a nxn array filled with n different symbols, each occurring exactly once in each row and each column. In user expirement, each row define the sequence a participant with try the UI defined by the symbols. It is used as comprimise, for the users to try different UIs, since not all permutations are aways possible,
