Module 7- Validity Issues Flashcards
Before the introduction of the IV, F should be
F=1
- bc have not introduced IV yet. therefore there should be no systematic difference in DV between the groups
- variances between the groups are equal
- only contains random/ error variance
- groups should not differ
F value very small
- p value is greater than the alpha (o.o5)
- difference bw the 2 groups due to chance
- larger denominator; chance variance
- fail to reject the null hypothesis
F value is large
- p value is less than the alpha (0.05)
- reject the null hypothesis
- difference bw groups is not due to chance
- bigger numerator; bw group difference
BUT BW GROUP DIFFERENCES COULD BE DUE TO IV VARIANCE OR CONFOUNDS
Confounds
- contribute systematic difference to the bw group variance (numerator of the F ratio)
- unintentional IVS
- act like IVs by creating systematic differences in the DV, but is not the IV we are interested
-contribute to change in the DV
Confounds threaten
Internal Validity
- lower the confidence that the DV is due to the IV alone
- represent alternative explanations for bw group differences
to be a confound…
- groups must differ on that variable
- must influence the DV
- if does not have both, than it is not a confound
Confound Hypothesis
- look at this after rejecting the null hypothesis
- ask; can a variable other than the IV account for bw group differences?
History
- threat to internal validity
- anything external to the study that has an outcome
- events in the environment that change the conditions of the study
ex. testing new drug for depression but pandemic (external event) has an influence
Maturation
- Threat to internal validity
- effect time has on individuals
- people change over time ^ can affect findings
- can see a change in pretest vs post test, but is that bc of the IV or maturation
ex. conducting reading ability on 2nd graders. take a pre test and give a 10 month intervention (IV). the post test showed different results but was that due to the intervention or maturation?
Testing
- threat to internal validity
- occur in designs with more than one testing phase; like having a pre test and a post test
- 2 types;
1. learning effects
2. fatigue effects
Learning Effects
- increase DV performance
- bc participants becoming more understanding of the task, developed strategies to do better, practiced it
ex. get a high score on the post test than the pre test of the math quiz bc have developed strategies to do better
Fatigue effects
- decrease DV performance
- bc of mental or physical fatigue
ex, get lower scores on the math post test bc participant is bored
Instrumentation
Threat to internal validity
- measuring instrument (device, survey, interviews, observation…) changes over time
- physical device could be mis calibrated
- can also happen if the researcher is the measuring instrument
Selection Bias
Threat to Internal Validity
- groups are not equivalent prior to the study due to how they were chosen
- groups differ on a participant related variable
- causes the difference in the DV to be attributed to a pre existing variable and not the IV
- ex, testing drug on one group of men and another group of women. Difference in Dv could be due to gender
Attrition
Threat to Internal Validity
- participants drop out of the study before it is finished
- creates unequal groups in salient characteristics
Diffusion of Treatment
Threat to internal validity
- procedures in one group contaminate or bleed into the other group
- communication outside the study bw the experimental and control groups
- exchange of information of IV manipulation can influence DV
- causes similar scores in the DV
Participation Effects
Threat to Internal Validity
- participants behave differently because they know they are being watched
- causes changes in the DV
ex. become more cooperative, defensive…
Demand Characteristics
participant behaves in a way the researcher wants them to
- can be because of the study procedure or inadvertent cues
Experimenter Effects
Threat to Internal Validity
- experimenter bias
- unintentional
- researchers giving subtle cues to the participant to influence their behaviour
- treat diff groups differently to cause the predicted difference
- bc of the expectation of the hyp, cause the experimenter to be biased
Regression to the Mean
Threat to internal validity
- when have extreme high or low scores, tendency for them to move (regress) towards the population mean in subsequent tests
Goal of the F ratio
- want it to be big
- maximize experimental/ IV variance
- control confound variance
- minimize error variance
Ways to control threats to internal validity
- control related to participant assignment
- control related to experimental design
- control related to logic of the experimentation
Participant Selection
- does not control threats to internal validity
- related to external validity
- how we choose a representative sample from the population
- select the sample through random sampling
Participant Assignment
- controlling threats to internal validity
- how we assign participant to groups after they have been randomly selected
- want groups to be equivalent before the introduction of the IV ( ^ no confounds and f=1)
