Week 2: Btn Subjects, threats to validity, experimental designs Flashcards
Do experimental designs allow us to establish causation?
Yes! This is the only one you can say yes to
What is the directionality problem of experimental designs?
The problem whereby while a study may have established a cause and effect relationship, this does not always explain the direction of the relationship. What must be then solved is determining which variable is the cause and which is the effect.
What’s an example of a directionality problem in experimental designs?
Exposure to violent video games causes increased aggressive behaviour
OR
Does increased aggressive behaviour cause a tendency to be drawn towards violent video games?
What is meant by the third-variable problem in experimental designs?
The idea that changes in one variable are often naturally accompanied with changes in many other related variables.
Remember although a study may establish that two variables are related, its doesnt necessarily mean theres a direct causal relationship. No! Its possible that a third unidentified variable is controlling the two variables and responsible for the observed relations.
What is an example of the third-variable problem in experimental designs?
If a study found that children’s maths ability increased with weight, this doesnt mean its necessarily causal. Maybe theres a correlation but it doesnt mean its causal.
Theres likely a third variable like ‘age’ that is also influencing this relationship.
What are four ways to overcome the third variable and directionality problems in experimental designs?
> MANIPULATE the independent variable
MEASURE the dependent variable
CONTROL extraneous variables (more like control confounding variables)
COMPARE the scores between the treatments e.g. independent samples t test
What are the two main disadvantages of experimental design?
- Need to know what to measure first
- Need to control fairly tightly
Also keep in mind its quite predictable, youre not going to find something you havent thought about
What are the two main types of experimental designs?
Between subjects
Within subjects
What is a between subjects design?
Its a type of experimental design that compares different groups of individuals. Its used to find the difference btn 2 or more treatments (or controls).
What are more characteristics of a btn sjt design?
- Different treatments are applied to different but equivalent samples
- Comparisons are made between the groups
- Gives independent scores
What is a btn sjt design also known as?
An independent measures experimental design
What are 3 ways to analyse results from btn sjts design?
- Independent samples t-test (if 2 treatment groups)
- One-way ANOVA (if MORE than 2 group)
- Wilcoxon or Friedman’s ANOVA (if non-parametric)
Advantages of btn sjts design?
- Not influenced by time-related factors e.g. history effects, maturation
- Not influenced by order effects e.g. Practice, Fatigue
- Not influenced by contrast effects (e.g. a 50 degree temperature room feels hot a 30 degree room, but a 50 degree room may feel cool after an 80 degree room)
Disadvantages of btn sjt design?
- Requires larger number of participants
- Vulnerable to some types of confounds
between sjts takes longer to recruit, need more ppl, vulnerable to some threats to validity: 1) individual differences (all ppl are different which increases noise) 2) environmental variables
What are the two main confounding variables in between subject experimental designs? Describe them
- INDIVIDUAL DIFFERENCES… which can then lead to assignment bias. Ind differences are any participant characteristics that differ from one person to another. If characteristics are diff from one group to another, then experiment is confounded by assignment bias. e.g. one group may have ppl of higher SES than other.
- ENVIRONMENTAL VARIABLES: any characteristics of environment that might differ e.g. one group conducted in morning and other at night or one in small room and one in big room. May cause differences btn groups that arent attributed to the IV! Rude!
Individual differences can become ………….
and can create higher ……….. between groups.
confounding … i.e. assignment bias
variance
How can we correct the problem of individual differences?
1) We can have equivalent groups that are
• Created equally
• Treated equally
• Composed of equivalent individuals
2) Randomisation
• Randomly assign participants to the groups
• Means that potentially confounding variables randomly distributed
• Can restrict the randomisation – also called pseudorandomisation
Whats another way to avoid problem of individual differences being a confounding variable altogether?
Hold variables constant / restrict range e.g. • Ensure all participants are male • Restrict age range • Restrict IQ range
What is restricted randomisation?
Describe one to me:
any procedure used with random assignment to achieve balance between study groups in size or baseline characteristics
E.g. if concerned everyone in group a will be young and all in group b will be old, can pair up ages with their closest pair then put one in group A and one in group B, decided by a coin toss.
What is another example of restricted randomisation:
Block randomisation: blocking is the arranging of experimental units in groups (blocks) that are similar to one another. Typically, a blocking factor is a source of variability that is not of primary interest to the experimenter. An example of a blocking factor might be the sex of a patient; by blocking on sex, this source of variability is controlled for, thus leading to greater accuracy.
What is something to think about with reference to individual differences and variance ?
IMP learn, understand directions.
• Trying to test for difference between your treatment conditions
• T-test etc tests for ratio of between group difference to within group variance
• So need to keep variance within groups low and variance between groups high to get a significant difference
• So if individual differences increase within group variance, makes it less likely you will detect between group difference!
(more overlap across between subject groups and youre less likely to find sig results)
List some examples of environmental threats
- Times of day
- Rooms taken in
- Weather
- Noise
What can we do to control for environmental threats?
- Run participants at same time of day
* Use same room
What is Differential attrition?
(what is it a threat to, what is attrition, differential attrition, example, implications).
Another kind of threat to internal validity.
Attrition= participant withdrawal from study before completion
Differential attrition=when the attrition rates DIFFER from group to group
Example= if large no. of people leave 10 week diet treatment group, groups are no longer the same, left with highly motivated/conscientious people in one group.
Implications = threat to int validity cos now we don’t know whether the obtained differences between treatment conditions are caused by the treatments or by differential attrition.
What is diffusion?
(what is it a threat to, what is it, example, implications).
Threat to internal validity
Diffusion = spread or DIFFUSION of information on the treatment, from the treatment group to control
Example = treatment group tells control about their meditation tasks they have to do as part of their depression treatment. Control members then start trying them out.
Implications= reduces the differences btn the two conditions. The true effects of the treatment have been hidden or masked by the shared info, that is, there appears to be no diff btn the groups, but thats only cos they’re actually getting same treatment.
What is compensatory equalisation?
(what is it a threat to, what is it, example, implications).
Threat to internal validity
Compensatory equalization = when control learns about treatment being received by other group and demands for the same, EQUAL treatment.
Example= boys in control group watching neutral video on play school find out in the next room boys are playing world of warcraft (much more violent), and demand EQUAL treatment. OR parents finding out new educational program is being tested on a group of kids (not their own) and demands EQUAL treatment.
Implications= can wipe out true effects of the treatment and make it look as though there are no differences btn the groups, when really they’re receiving same treatment!
What is compensatory rivalry?
(what is it a threat to, what is it, example, implications).
Threat to internal validity
Compensatory rivalry = when control works extra hard to prove a point to their RIVALS (treatment group) that they can perform just as well even without the treatment.
Implications = performance of control is much higher than would otherwise me, masks true possible beneficial effects of the treatment.
What is resentful demoralisation?
(what is it a threat to, what is it, example, implications).
Threat to internal validity.
Resentful demoralization= when control group gives up when they learn that another group is getting special treatment.
Implications= control is less motivated, give up, dont care, poor performance and the effects of the treatment group appear to be a lot better than what it truly is.
How do you overcome threats to internal validity like these ones: resentful demoralisation, compensatory rivalry, compensatory equalisation, diffusion and differential attrition?
DONT LET THEM TAlK. Separate groups as much as possible. WHATEVER HAPPENED TO THE SEPARATION OF THE CLASSES!
