Experimental design

Question 1

What are 3 experimental designs to avoid?

Answer 1

One-group post-test only - number of possible reasons for any observed change
Non-equivalent control group
One-group-pretest-posttest - again a number of reasons for change

Question 2

Why do we have to be careful with correlation studies?

Answer 2

Correlation does not equal cause - in non-experimental designs we can’t control for the number of extraneous variables which could actually be causing an effect, so we can’t rule out any of these competing explanations

Question 3

Why is random allocation in true experiments so important?

Answer 3

Eliminates variation between groups caused by participant variables, making the probability of participant differences very minimal

Question 4

What is the importance of placebo control groups?

Answer 4

Pure control groups i.e. complete absence of any treatment don’t control for the possibility that treatment effects are purely psychological
Placebo groups receive something exactly like the active level of the IV but ineffective

Question 5

Why are experimental designs better for causal direction?

Answer 5

In correlations we can’t be sure which variable is a cause and which is the effect, so experimental designs help to eliminate one possibility

Question 6

What are 4 critiques of experiments?

Answer 6

Can’t always do them e.g. for ethical reasons
Highly artificial and limit range of study topics as require all variables to be operationally defined - non-experimental designs are better suited to study of naturally occurring phenomena
Reactivity effects - participants react to knowledge of being in an experiment
Qualitative researchers say that experiments prescribe participant behaviour i.e. lacks personal view and participants become passive

Question 7

What are 3 advantages of experimental designs?

Answer 7

Can isolate cause-effect relationships because IV and extraneous variables controlled
Alternative explanations of effects can be investigated/eliminated in extensions of original experiment - easy to replicate
Control extraneous variables so validity high and alternative explanations are eliminated/weakened

Question 8

What is meant by an Independent Samples experimental design?

Answer 8

One group in experimental condition, an independent group of people in control condition
These two conditions are our LEVELS of the IV

Question 9

What is a major weakness of independent samples designs?

Answer 9

Participant variables which threaten validity of conclusions - differences may not necessarily be caused by IV but may result from too many people with a certain quality in one of the groups (by chance)

Question 10

How can we reduce the likelihood of participant variables influencing results?

Answer 10

Random allocation - works best in large groups
Pre-testing participants - show that both groups are similar in relevant performance prior to applying experimental condition (can also pre-test on potential confounding variables)
Representative allocation - pre-testing is time-consuming and doesn’t eliminate all problems of non-equivalent groups e.g. doesn’t reflect how participants will vary when IV applied; to make groups representative on several variables we have to decide which ones are most important to balance for our given topic and aims

Question 11

What is meant by a repeated measures design?

Answer 11

One group does both conditions - good way to control for participant differences!
Look at differences within individual subjects following IV rather than comparing between different groups
Any differences should be from manipulating the IV (barring random error)

Question 12

What is the difference between repeated measures and multiple testing?

Answer 12

In repeated measures exactly the same measure is taken under two levels of IV
In other studies two different measures may be taken e.g. a test of neuroticism and a test of extroversion - in these types of study we can only correlate pairs of scores

Question 13

What is meant by order effects?

Answer 13

Effects from the order in which participate in the conditions in a repeated measures design, “practise effects”
Might perform worse in second condition due to being disheartened, bored etc.

Question 14

What is meant by counterbalancing?

Answer 14

For example, if we have conditions A and B one subgroup could do AB and one could do BA, and any effect of having one treatment before the other will cancel out

Question 15

What is meant by asymmetrical order effects?

Answer 15

A complication of counterbalancing wherein the practise effect in AB order is not equivalent to that in BA order - we lose the error-balancing effect of counterbalancing and may come to incorrect conclusions
We need to inspect results very carefully to make sure such mistakes are not made, and in such circumstances we may want to use a between-subjects design instead

Question 16

What is meant by complex counterbalancing?

Answer 16

Another way to control for asymmetrical order effects (often where we have more than 2 conditions)
ABBA - score for A condition A will, for example, be the average of the scores in both A conditions
Multi-condition designs - if we have 3 conditions divide participants randomly into 6 groups and have them take part in the following orders: ABC, ACB, BAC, BCA, CAB, CBA

Question 17

What are 4 other ways to deal with order effects?

Answer 17

Randomisation of condition order (when a fair few conditions) e.g. through random computer generation, essentially ensuring each item in a sequence is not predictable from previous item/sequence
Randomisation of stimulus items - rather than only randomising the order of conditions, randomly mix stimulus items in one condition e.g. present one list of 6 mixed anagrams rather than presenting one anagram for each of six different conditions, participants don’t complete one condition then next so order effects eliminated
Elapsed time - leave enough time between conditions for any learning/fatigue effects to dissipate
Use another design - e.g. move to independent sample design (but only as last resort)

Question 18

When should repeated measures designs be avoided?

Answer 18

When order effects cannot be eliminated/are asymmetrical, and also when participants need to be naïve in each condition e.g. when there is a need to keep research aim hidden so as not to influence participant behaviour
Also not appropriate where study requires an equivalent control group

Question 19

Summarise the key disadvantages of independent samples designs that serve as advantages of repeated measures designs?

Answer 19

Non-equivalence of samples
More expensive in independent samples (twice as many participants)
Homogeneity of variance - if there is too great a difference between statistical variances of independent groups we have to use non-parametric testing which isnt ideal

Question 20

What are the key disadvantages of repeated measures designs?

Answer 20

Order effects
Losing participants (lose from both groups)
Experimenter demands - research aim may become clear to participants, producing demand characteristics
Need to wait for effects to wear off while in independent samples conditions can be ran simultaneously

Question 21

What is meant by a matched pairs design?

Answer 21

Subjects are allocated to a group matched on a variable that has potential to confound results - it is essentially a compromise between IS and RM designs
e.g. we would identify the top two scores on a pre-test measure and allocate one individual to experimental group and one to control and so on
Doesn’t completely eliminate problem of participant variables as matching is never perfect but problem is minimised
Need to make sure matched variables correlate with the dependent variable (could weaken results if not)

Question 22

What is meant by single participant/small n designs?

Answer 22

Single participant across multiple conditions
Small numbers useful where long term training is required and where condition/variable being considered is specific/rare
Would use ABAB design

Question 23

What do we mean by the terms “related” and “unrelated” designs?

Answer 23

Related are repeated measures and matched pairs designs - score in one condition is directly paired with score in other condition
Unrelated is independent samples - each score in one condition cannot in any way be related to any specific score in the other conditions
Single participant designs can also be considered unrelated

Question 24

What do we mean by a 2x2 unrelated design?

Answer 24

E.g. experiment into how audience effects influence performance on a task
We would want to use two levels of the task IV (simple and complex) as evidence suggests effects different for both
We would also have two levels of audience IV, present/absent
So two levels each of two IVs, hence the 2x2

There is a statistical advantage to merging two separate experiments into one experimental design like this

Question 25

What are key advantages of the matched pairs design?

Answer 25

Don’t have to wait for participants to forget first condition
Can use same stimulus lists etc
Participant variables partly controlled for
No order effects
Homogeneity of variance not a problem

Question 26

What are key disadvantages of the matched pairs design?

Answer 26

May be hard to find perfect matches and therefore time consuming
Loss of one member of a pair entails loss of whole pair
Some participant variables still possible

Need to make sure pairs are RANDOMLY ALLOCATED to conditions