Lecture 3: Experimental Design, Counterbalancing and Control

Question 1

Purpose of an experiment

Answer 1

To establish cause and effect relationships

Dangers

Confounding variables
Inappropriate control conditions
Inappropriate variables

Question 2

Desires

Answer 2

Clearly identifiable Independent and Dependent variables
Closely matched control conditions
No confounding variables
Appropriate counterbalancing
Easy to analyse

Question 3

Steps for designing an experiment

Answer 3

Determine hypothesis
Identify dependent and independent variables
Identify potential confounding variables
Devise control for confounding variables
Determine appropriate experiment design
Plan statistical analysis and check that design can be analysed
Devise counterbalancing strategy for things that cannot be controlled.
Devise randomisation strategy for things that cannot be controlled or counterbalanced.
Define procedure (manipulations, measures, task etc)

Question 4

Potential confounds in experiments

Answer 4

Order effects:
Tiredness.
Time of day.
Habituation.
Training effects.
…
Group differences (other than the thing you are trying to test).
			Match participants.
			Include baseline measures.
			Measure relative performance.
			Control group.

Question 5

Types of Design

Answer 5

Participants 
- Different participants
Independent samples
Between subjects
- Same participants
Paired samples
Within subjects
- Multiple groups of participants but each do several conditions.
Mixed design
Split-plot

Factors
- Single factor (variable)
One-way
Two factors (variables)
Two-way
Three factors (variables)
Three-way

Levels
The number of different values / groups / conditions for each factor

Question 6

Factors vs Levels

Answer 6

Factors are distinct independent variables:
Eg. Age and Recall Delay are different Factors.
Each variable can take multiple values

Levels
Levels are the values that you plan to test.

Each variable can take multiple values
Each Factor can have multiple Levels

Question 7

Examples

Answer 7

(Look at Powerpoint)

Question 8

Counterbalancing

Answer 8

(Look at Powerpoint)

Question 9

Blocked vs interleaved conditions

Answer 9

Blocked conditions
Participant undertakes a batch or block of trials belonging to the same condition followed by a batch for another condition etc..
Reduced uncertainty
Increases habituation
Good for splitting conditions across sessions but be careful of ‘time of day’ effects.
Interleaved conditions
Participants see all conditions each session in either a counterbalanced or randomised order.
Reduces habituation
Increases uncertainty
Can still be split into multiple sessions.

Question 10

CounterbalancingBetween and within participants

Answer 10

Counterbalancing between Participants
Each participant undertakes conditions in a different order all orders presented but not all participants see all orders.
Counterbalancing within Participants
In some experiments – especially those involving imaging and EEG the presentation order for individual trials / stimuli matters.
Brains reaction to one stimulus may affect its response to a subsequent stimulus.
Counterbalance the order of stimuli within a test session (interleaved presentation).
Aim is to have show all possible sequential orderings to ALL the participants.
Eg A is followed equally often by B and C.

Question 11

Randomisation

Answer 11

Participants to “Treatments”: If you have two or more treatments or manipulations to be applied to different groups drawn from the same population randomise the allocation of people to treatment groups.
Alternative to counterbalancing: As an alternative to counterbalancing you can present conditions in a random order.
This is especially useful when there are a lot of conditions.
Works best with interleaved conditions such that (potentially) each consecutive trial belongs to a different condition.
Can also be applied to blocked conditions such that blocks appear in a different random order for each person.
Usually done such that all conditions appear equally often (eg Shuffling).