Lecture 9: Research Methods I

Question 1

Sketch the cycle of empirical research

Answer 1

Question 2

What are the steps for scientific research

Answer 2

1. Question/Theory
2. Hypothesis
3. Study design
4. Data collection
5. Analysis
6. Report results

Question 3

Name four quality criteria for a hypothesis and briefly describe aspects of them

Answer 3

Question 4

What is an alternative hypothesis vs. a null hypothesis

Answer 4

Alternative Hypothesis is a hypothesis that states an effect (or a difference or a relationship)

Null-Hypothesis is a hypothesis that states that there is not statistical difference (or relationship or effect) between groups or variables

Hypothesis formulates an assumption, speculation, theory that can be tested or validated by experimentation

Question 5

Which types of empirical studies in HCI do you know? Briefly describe them.

Answer 5

Question 6

Describe what dependent and independent variables are. Name examples for Objective and Subjective values

Answer 6

Independent variables are manipulated by the experimenter (Conditions under which the tasks are performed)

Dependent variables are affected by the independent variables (Measured in the user study)

Objective values: time to complete a task, number of words typed

Subjective values: ease of use, preferred option

Question 7

Which types of Experimental Designs do you know? Briefly describe them.

Answer 7

Question 8

What is for all types of experiments crucial?

Answer 8

Randomization

In within-subject designs - order of conditions
In between-group designs - allocation to groups

Question 9

Quantitative Data vs. Qualitative Data: What is the utlity of both data types?

Answer 9

Quantitative Data (generation of numerical data)
Usage:
* Verify performance benefits of new input/output devices or interaction techniques
* Determine differences between user groups

Qualitative Data

Used to:
* Find problem areas
* Find conceptual errors
* Find missing functionality

Question 10

List examples for subjective and objective data

Answer 10

Subjective data (Quantitative and qualitative)
* Self-reports (“For me it was easy to use the device”)
* Other-reports (“The device made X happy”)

Objective data (typically quantitative)
* Physiological data (e.g. heart-rate)
* Behavioral data(e.g. click rates)

Question 11

What is the best practices for data generation and handling?

Answer 11

Combine different sources of data
qualitative and quantitative as well as subjective and objective data

Question 12

List the four criteria of emperical evidence and briefly describe them.

Answer 12