Module 8- Experimental Design Flashcards
Purpose of the experimental design
- make causal statements
- only design where we can make causal statements about the IV and DV
Sound experimental design allows for
- determine the effect of the IV on the DV
- rule out alternative explanations; threats to internal validity
Single Group Design
- involves one group of participants
- measuring one level of the IV and DV
- not a sound experimental design
- does not make causal statements
- does not rule out threats to internal validity ^ does not allow us to determine the effect of the IV on DV
group > treatment (IV) > Measurement (DV)
Comparison Group Design
- involves a comparison group that does not receive the IV/ Treatment
- helps us get an indication of the potential impact of the IV
BUT not a sound experimental design bc does on rule alternative explanations/ threats to internal validity
Group A > Treatment (IV) > Measurement
Group B> No Treatment > Measurement
Random Assignment (R)
- an aspect of a true experimental design
- helps control threats to internal validity and confounding variables
- how we assign participants to groups after they have been selected from the population
- R is based on chance and no bias
- all known and unknown confounds are randomly distributed therefore confounds do not affect the DV and not a threat to internal validity
Basic randomized control group design
R Group A> Treatment> Measurement
R Group B> NO treatment> Measurement
group B= control group (called a control group and not a comparison group bc we are “controlling” threats to internal validity)
Randomized pre-post control group design
- includes a pretest measure of the DV
R Group A > Measurement> Treatment > Measurement
R Group B> Measurement> Treatment> Measurement
Advantages of randomized pre-post control group design
- allows us to verify that the groups do not differ on the DV prior to the IV; want groups to be equal prior to IV
- No initial differences on the DV - can give us a better understanding of the impact of the IV, by giving a baseline measurement of the DV
Disadvantages of randomized pre-post control group design
- pretest can interact with the treatment (IV) to impact the DV
- this can be a treatment to internal validity
- when this occurs use to Solomon Four Group Design
Solomon Four Group Design
- combines the 2 types of randomized control group designs ( basic and pre post)
- one involves a pre test
- one involves does not involve a pre test
- allows us to assess if the pre test interacts with the treatment to impact the DV
- assign participants to one of 4 groups
R Group A> Pre test> Treatment> Post Test
R Group B> Pre Test> No Treatment > Post Test
R Group C> Treatment> Post Test
R Group D> No Treatment> Post Test
3 criteria for a true experiment
- participants have to be randomly assigned
- at least 2 levels of the IV
- controls put in place for major threats to internal validity
all criteria must be met in order to be a true experiment and ^ make casual statements
Before the IV, F ratio should be
F=1
- bw group variance is equal to w/in group variance
- no systematic bw group differences ^ only accounting for chance/ error variance
If before the IV, F ratio is greater than one
- a confound is likely contributing to systematic bw group variance
- not good
- we want groups to be equal before the introduction of the IV
2 main types of experimental designs
- b/w subjects design
- within subjects design
Between Subjects Design
- participants have been randomly assigned to different groups and experience one study group/ level of the IV
- DV measures are taken and compared bw groups
- not experiencing all study groups/ levels of the IV
- each group defined by the level of IV received
Simple random be subjects design
R Group A > Treatment> Measurement
R Group B> Treatment (no level of the IV)> Measurement
- involves an experimental and control group
Multi level bw subjects design
R Group A >Treatment Level 1>Measurement
R Group B >Treatment Level 2 >Measurement
R Group C >Treatment Level 3> Measurement
R Group D> Treatment Level 4 >Measurement
why is bw subjects our go to design?
- high internal validity
What error is bw subjects prone to?
- type 2 errors; fail to reject the null hyp, when the null hyp is false
- bc gives a smaller F ratio than within subjects
- larger sample can reduce type 2 errors
Interpreting rejection of the null hypothesis
- if have 2 groups, we can look at the data and see which group has the larger mean on the DV
- BUT if have 3 or more groups, rejection of the null hypothesis tells us there is a difference somewhere, but doesnt say which groups differ
Post Hoc
- when have more than 2 groups
- tells which means differ
- done after the F test/ rejection of the null hypothesis
- involves testing each pairing of means to determine where the difference lies
- these keep type 1 errors controlled
Priori Tests
- comparing the means you expect to differ based on theory/hypothesis
- rather than testing every pairing of means like in the Post Hoc tests
- also done when have more than 2 groups
when comparing more than one test group use
T Test
when comparing more than one test group use
F test/ ANOVA
Within Subjects Design
- each participant experiences each level of the IV and provides a measure on the DV
- no random assignment bc only one group of participants
- same group of participants
Advantages of Within Subject Designs
- assures equivalence of groups bc it is the same group of people
- what is does to the F ratio; minimizes the denominator therefore making a larger F
- reduces the error term of the F ratio which increases the power and ability to detect the treatment effect
- more sentive to effects of the IV; can pick up on small treatment effects
- less prone to type 2 errors
- dont need as many participants
Order Effects
- disadvantage to within subjects design
- has to do with time; moving though sequential treatment conditions
- experiencing multiple conditions can impact scores on the DV
Practice and Fatigue effects
- disadvantage to within subjects design
- come about bc involved in numerous treatment levels
- ex. participants become more comfortable with the math problems or bored ^ make more errors
Carry over effects
- disadvantage to within subjects design
- due to the impact of one specific treatment condition
- impact of the condition bleeds over into subsequent IVs
- counterbalancing does not work bc order of treatment conditions does not matter
Should not use within subjects design when
- carry over effects are strong
- bc can’t be controlled and major threat to internal validity
Counterbalancing
- helps to correct order effects
- mixing up the order of treatment presentation so not all participants see the treatment in the same order
- distributes order effects across all conditions
Complete counterbalancing
- includes all possible orderings of treatment conditions
- each condition must be included equally as often and proceed each other a equal number of times
- becomes hard when conditions grow; number of orderings grow exponentially as the number of conditions grow
- effectively distributes order effects
Incomplete counterbalancing
- do when unrealistic to do complete counterbalancing
- helps distributes order effects
- every condition needs to be in the sequence but does not need to follow each condition equally
- do when have a large number of condtions
Latin Square
- most common incomplete balancing procedure
- number of presentations equals the number of conditions you have
- all possible orders are not used
