Chapter 3 - Experimental design

Question 1

Explain the differences among the three types of study: experiment, quasi­-experiment,
and non-­experiment.

Answer 1

­ If a study involves multiple conditions and the participants are randomly assigned to each condition, it is a true experiment.

If a study involves multiple groups or measures but the participants are not randomly assigned to different conditions, it is a quasi­experiment.­

If there is only one observation group or only one measure involved, it is a non­experiment.
­
A true experiment is based on at least one testable research hypothesis and aims to validate it. There are usually at least two conditions og groups, one research and one control. The dependent variables are normally measured through quantitative measurements. The results are analyzed through various statistical significance tests. It should be designed and conducted with the goal of removing potential biases. It should be replicable with different participant samples, at different times, in different locations, and by different experimenters.

Question 2

What are the major issues that need to be considered when designing experiments?

Answer 2

Universal research issues: research hypotheses, measurement of the dependent
variables, and the control of multiple conditions.

­Human subject related issues: learning effect, participants’ knowledge background, and the size of the potential participant pool. (s.43)
­
An issue to consider is the fact that the experiment can be conducted in several different ways, and it’s up to the researcher to determine the right one based on goals, earlier research and the hypothesis.

­Another issue to consider is how to control the independent variables to create multiple experiment conditions. Wizard of Oz is one solution.

Question 3

What is a between-­group design? Explain the advantages and disadvantages.

Answer 3

In between ­group each participant is only exposed to one experiment condition.
­
Advantages:
­From a statistical perspective, between­group is a cleaner design. Since the participants
is only exposed to one condition, the users do not learn from different task conditions.
­Shorter involvement for each participant.
­
Disadvantages:
­Comparing two groups of people, large sample sizes, individual differences, harder to get statistically significant results

Question 4

What is a within­-group design? Explain the advantages and disadvantages.

Answer 4

In within­group each participant is involved in multiple experiment conditions.
­
Advantages:
Small sample size, no individual differences,
­
Disadvantages:
­Learning effect of doing multiple tasks, fatigue from multiple experiments. (s.44)

Question 5

When should a between­group design be considered for an experiment?

Answer 5

Simple tasks with limited individual Differences, tasks that would be greatly influenced by learning effect, or problems that cannot be investigated through a within­group design.

Some experiments can’t use within­group. (ex. ting som er “i” personen - som mellom grupper med forskjellig utdanningsbakgrunn)

Question 6

When should a within­group design be considered for and experiment?

Answer 6

tasks with large individual differences

­task with limited learning effect

Question 7

What is the benefit of a factorial design compared to experiments that investigate one factor at a time?

Answer 7

Pros factorial: Får testet flere IV på en gang, får se den samlede effekten, får testet sammenhenger mellom flere ting.
(s.53)
Ex. Sjekke skrivehastighet (DV), Tastaturtype (IV) og tekstklilde (IV).

Pros basic design: lettere å utføre, men kan gå glipp av noe.

Question 8

What is a split-­plot design?

Answer 8

Using both within­ and between­-group with multiple groups.

Question 9

Explain the differences between random errors and systematic errors.

Answer 9

Random errors will cancel each other out with a large enough group.

Systematic errors you have to actively avoid.

Du kan ikke kontrollere tilferdige-feil. Du må bare måle omstendig og redusere påvirkningen de har.

Systematiske feil er designers ansvar, og må arbeides for å gå det.

Question 10

What are the major sources of systematic errors, or biases?

Answer 10

Målefeil, prosedyrefeil, irrelevante oppgaver, instrument-feil, participants (ex personlig interesse (olje/miljø), experimentor behaviour, experimental environment (s.57-8)

Question 11

What can we do to reduce systematic errors in experiments?

Answer 11

do a better experiment, have more control.

Question 12

Describe the typical procedure of an experiment that involves human subjects.

Answer 12

Ensure the systems or devices being evaluated and the related instruments are ready for the experiment.
* Greet the participants.
* Introduce the purpose of the study and the procedures.
* Get the consent of the participants.
­* Assign the participants to a specific experiment condition
* According to the pre­defined randomization method..
­* Participants complete training task.
­* Participants complete actual task.
­* Participants answer questionnaires (if any).
­* Debriefing session.
­* Payment (if any).

Question 13

*Hva bestemmer om det er basic eller factorial design?

Answer 13

Om det er en eller flere uavhengige variabler.

Question 14

*Interaction effects

Answer 14

(s.56)

Question 15

*Hva er det fem punktene som karakteriserer experiment?

Answer 15

1. Testbar hypotese
2. Minst to contitions (holder med en treatment condition og en kontroll contidion)
3. Kvantitative mål (measurments)
4. Analysert gjennom statistisk signifikanstest
5. Designet for å fjerne bias
6. Skal kunne gjentas

(s. 42)