Week 7 Flashcards
What is validity and reliability and why are they key concerns?
Validity: is the evaluation correctly assessing the system?
Reliability: are the evaluation results independent of the evaluators? (The “evaluator effect”)
What’s an experiment in HCI and when is it used?
Scientifically rigorous method for measuring the performance of a user interface. The basic idea is to vary one condition of a situation and observe the outcome. Example: Search times for a user with new and old search feature.
What are the constituents of an experiment?
The experimental design governs the independent variables (intervention) and dependent variables (measured performance). Handling nuisance factors, such as humans being different, by controlling for them in the experimental design. Determine that experiment is valid.
What’s the role of the hypothesis and how is it formulated?
Hypotheses are statements that link manipulations of the independent variables to differences in the dependent variables. Hypothesis can be used in a confirmatory way, where rigorous hypotheses are formulated and tested (hypothesis testing), or exploratory in which phenomena are hunted to find questions to be tested (hypothesis generation)
What’s independent and dependent variables?
Independent variables are factors manipulated by the experimenter to observe their effect. Dependent variables are measures reflecting the influence of independent variables.
How are dependent variables selected?
On a case-by-case basis. Common measures are task completion time, accuracy or error rates and questionnaire responses.
How are participants chosen?
Participants are chosen as broadly as possible given the context of the experiment from the population. This is to achieve generalizable results.
How are results from experiments analysed?
Results from experiments can be analysed using descriptive statistics, which is describing the relationships between variables in the dataset. It can also be analysed by inferential statistics by drawing conclusions about the population from the sample and the data.
What is the difference between qualitative and quantitative data?
Qualitative data usually involves natural language, such as verbal protocols, video recordings and interviews, and have historically required more abstract interpretation. In contrast, quantitative data is measurable resulting in numbers.
What is a field study and how is it different from an experiment?
A field study is an evaluation that consider the context of use usually done in the “real world”. The main difference to an experiment is it is not being done in a controlled setting in a laboratory, which means there is a lot of external factors to account for, but instead a lot of realism is gained.
