Week 2: Btn Subjects, threats to validity, experimental designs

Question 1

Do experimental designs allow us to establish causation?

Answer 1

Yes! This is the only one you can say yes to

Question 2

What is the directionality problem of experimental designs?

Answer 2

The problem whereby while a study may have established a cause and effect relationship, this does not always explain the direction of the relationship. What must be then solved is determining which variable is the cause and which is the effect.

Question 3

What’s an example of a directionality problem in experimental designs?

Answer 3

Exposure to violent video games causes increased aggressive behaviour
OR
Does increased aggressive behaviour cause a tendency to be drawn towards violent video games?

Question 4

What is meant by the third-variable problem in experimental designs?

Answer 4

The idea that changes in one variable are often naturally accompanied with changes in many other related variables.
Remember although a study may establish that two variables are related, its doesnt necessarily mean theres a direct causal relationship. No! Its possible that a third unidentified variable is controlling the two variables and responsible for the observed relations.

Question 5

What is an example of the third-variable problem in experimental designs?

Answer 5

If a study found that children’s maths ability increased with weight, this doesnt mean its necessarily causal. Maybe theres a correlation but it doesnt mean its causal.
Theres likely a third variable like ‘age’ that is also influencing this relationship.

Question 6

What are four ways to overcome the third variable and directionality problems in experimental designs?

Answer 6

> MANIPULATE the independent variable
MEASURE the dependent variable
CONTROL extraneous variables (more like control confounding variables)
COMPARE the scores between the treatments e.g. independent samples t test

Question 7

What are the two main disadvantages of experimental design?

Answer 7

• Need to know what to measure first
• Need to control fairly tightly

Also keep in mind its quite predictable, youre not going to find something you havent thought about

Question 8

What are the two main types of experimental designs?

Answer 8

Between subjects

Within subjects

Question 9

What is a between subjects design?

Answer 9

Its a type of experimental design that compares different groups of individuals. Its used to find the difference btn 2 or more treatments (or controls).

Question 10

What are more characteristics of a btn sjt design?

Answer 10

• Different treatments are applied to different but equivalent samples
• Comparisons are made between the groups
• Gives independent scores

Question 11

What is a btn sjt design also known as?

Answer 11

An independent measures experimental design

Question 12

What are 3 ways to analyse results from btn sjts design?

Answer 12

• Independent samples t-test (if 2 treatment groups)
• One-way ANOVA (if MORE than 2 group)
• Wilcoxon or Friedman’s ANOVA (if non-parametric)

Question 13

Advantages of btn sjts design?

Answer 13

• Not influenced by time-related factors e.g. history effects, maturation
• Not influenced by order effects e.g. Practice, Fatigue
• Not influenced by contrast effects (e.g. a 50 degree temperature room feels hot a 30 degree room, but a 50 degree room may feel cool after an 80 degree room)

Question 14

Disadvantages of btn sjt design?

Answer 14

• Requires larger number of participants
• Vulnerable to some types of confounds

between sjts takes longer to recruit, need more ppl, vulnerable to some threats to validity: 1) individual differences (all ppl are different which increases noise) 2) environmental variables

Question 15

What are the two main confounding variables in between subject experimental designs? Describe them

Answer 15

• INDIVIDUAL DIFFERENCES… which can then lead to assignment bias. Ind differences are any participant characteristics that differ from one person to another. If characteristics are diff from one group to another, then experiment is confounded by assignment bias. e.g. one group may have ppl of higher SES than other.
• ENVIRONMENTAL VARIABLES: any characteristics of environment that might differ e.g. one group conducted in morning and other at night or one in small room and one in big room. May cause differences btn groups that arent attributed to the IV! Rude!

Question 16

Individual differences can become ………….

and can create higher ……….. between groups.

Answer 16

confounding … i.e. assignment bias

variance

Question 17

How can we correct the problem of individual differences?

Answer 17

1) We can have equivalent groups that are
• Created equally
• Treated equally
• Composed of equivalent individuals

2) Randomisation
• Randomly assign participants to the groups
• Means that potentially confounding variables randomly distributed
• Can restrict the randomisation – also called pseudorandomisation

Question 18

Whats another way to avoid problem of individual differences being a confounding variable altogether?

Answer 18

Hold variables constant / restrict range 
e.g. 
• Ensure all participants are male
• Restrict age range
• Restrict IQ range

Question 19

What is restricted randomisation?

Describe one to me:

Answer 19

any procedure used with random assignment to achieve balance between study groups in size or baseline characteristics
E.g. if concerned everyone in group a will be young and all in group b will be old, can pair up ages with their closest pair then put one in group A and one in group B, decided by a coin toss.

Question 20

What is another example of restricted randomisation:

Answer 20

Block randomisation: blocking is the arranging of experimental units in groups (blocks) that are similar to one another. Typically, a blocking factor is a source of variability that is not of primary interest to the experimenter. An example of a blocking factor might be the sex of a patient; by blocking on sex, this source of variability is controlled for, thus leading to greater accuracy.

Question 21

What is something to think about with reference to individual differences and variance ?

Answer 21

IMP learn, understand directions.
• Trying to test for difference between your treatment conditions
• T-test etc tests for ratio of between group difference to within group variance
• So need to keep variance within groups low and variance between groups high to get a significant difference
• So if individual differences increase within group variance, makes it less likely you will detect between group difference!

(more overlap across between subject groups and youre less likely to find sig results)

Question 22

List some examples of environmental threats

Answer 22

• Times of day
• Rooms taken in
• Weather
• Noise

Question 23

What can we do to control for environmental threats?

Answer 23

• Run participants at same time of day

* Use same room

Question 24

What is Differential attrition?

(what is it a threat to, what is attrition, differential attrition, example, implications).

Answer 24

Another kind of threat to internal validity.

Attrition= participant withdrawal from study before completion

Differential attrition=when the attrition rates DIFFER from group to group

Example= if large no. of people leave 10 week diet treatment group, groups are no longer the same, left with highly motivated/conscientious people in one group.

Implications = threat to int validity cos now we don’t know whether the obtained differences between treatment conditions are caused by the treatments or by differential attrition.

Question 25

What is diffusion?

(what is it a threat to, what is it, example, implications).

Answer 25

Threat to internal validity

Diffusion = spread or DIFFUSION of information on the treatment, from the treatment group to control

Example = treatment group tells control about their meditation tasks they have to do as part of their depression treatment. Control members then start trying them out.

Implications= reduces the differences btn the two conditions. The true effects of the treatment have been hidden or masked by the shared info, that is, there appears to be no diff btn the groups, but thats only cos they’re actually getting same treatment.

Question 26

What is compensatory equalisation?

(what is it a threat to, what is it, example, implications).

Answer 26

Threat to internal validity

Compensatory equalization = when control learns about treatment being received by other group and demands for the same, EQUAL treatment.

Example= boys in control group watching neutral video on play school find out in the next room boys are playing world of warcraft (much more violent), and demand EQUAL treatment. OR parents finding out new educational program is being tested on a group of kids (not their own) and demands EQUAL treatment.

Implications= can wipe out true effects of the treatment and make it look as though there are no differences btn the groups, when really they’re receiving same treatment!

Question 27

What is compensatory rivalry?

(what is it a threat to, what is it, example, implications).

Answer 27

Threat to internal validity

Compensatory rivalry = when control works extra hard to prove a point to their RIVALS (treatment group) that they can perform just as well even without the treatment.

Implications = performance of control is much higher than would otherwise me, masks true possible beneficial effects of the treatment.

Question 28

What is resentful demoralisation?

(what is it a threat to, what is it, example, implications).

Answer 28

Threat to internal validity.

Resentful demoralization= when control group gives up when they learn that another group is getting special treatment.

Implications= control is less motivated, give up, dont care, poor performance and the effects of the treatment group appear to be a lot better than what it truly is.

Question 29

How do you overcome threats to internal validity like these ones: resentful demoralisation, compensatory rivalry, compensatory equalisation, diffusion and differential attrition?

Answer 29

DONT LET THEM TAlK. Separate groups as much as possible. WHATEVER HAPPENED TO THE SEPARATION OF THE CLASSES!