Chapter 11 Confounding and Obscuring Variables Flashcards
why is the one-group pretest/posttest design a bad design?
there is no comparison group/there is only 1 IV level
3 threats to internal validity
design confounds
selection effects
order effects
6 main potential threats to internal validity
maturation
history
regression to the mean
attrition
testing
instrumentation
***combined threats
3 additional general threats to internal validity
observer bias
demand characteristics
placebo effects
maturation threats and examples
a change in behaviour that emerges spontaneously over time
ex: gaining experience, developmental changes, fatigue, boredom, hunger
how can maturation threats be prevented/detected?
comparison groups
history threats
an external event affects most members of the treatment group at the same time as the treatment (systematically)
how can history threats be prevented/detected?
comparison group
regression to the mean
statistical concept in which extremely low/high performance at time 1 is likely to be less extreme at time 2 (closer to the mean)
regression threats only occur:
-in pre/posttest design AND
-when a group has an extreme pretest score
why can regression to the mean occur?
-random error in measurement
-when measures have low reliability
-doesnt happen all the time
attrition
a reduction in participants from pretest to posttest
attrition is only a threat if it’s…
systematic
how can attrition be prevented/detected?
-remove the pretest scores of the participants who drop out
-inspect if pretest scores of those who dropped out are extreme
testing threats
when the very act of completing a pretest influences responses on the posttest
why testing threats may occur
-participants are aware of the hypothesis
-re-evaluate the DV
-practice causes improvement (order effect)
-consistency pressures: people want to give off the impression that they’re consistent/on all the time
how can testing threats be prevented/detected?
-avoid pre/posttest design (harsh)
-use alternative equivalent forms of test for DV at pretest and posttest
-comparison groups show smaller pre/posttest fx than treatment group
instrumentation threats/instrumentation decay
when a measuring instrument changes over time
in what 2 ways can measuring instruments change over time?
-different observers or changed criteria by the same observers
-non-equivalent forms of test to measure the DV
how can instrumentation threats be prevented?
-keep the same observers
-highly structured coding standards
-posttest only design
-equivalent forms of test
-counterbalancing pretest/posttest
2 combined threats
selection-history threats: an outside event/factor systematically affects participants at 1 level of the IV
selection-attrition threats: participants in 1 experimental group experience attrition
observer bias
when observers expectations influence either the interpretation of participants behaviours or the outcome of the study
when may observer bias occur?
when the DV is behavioural
adding a comparison group cannot fully solve the issue of this threat to internal validity and may even increase its threat
observer bias; may increase the threat if the observers know who’s in which group
observer bias threatens which validities?
internal and construct
demand characteristics
when participants discover what a research study is about, it may change their behavior in the “expected” direction
true or false: adding a comparison group can fully solve the demand characteristic issue
false
what does Kihlstrom say about human participants
they are curious creatures who constantly think about what is happening to them, evaluating the proceedings, figuring out what they’re supposed to do and planning their response
solutions to demand characteristics
double-blind design
single-blind/masked design: observers don’t know participants conditions
placebo effects
when people receive a treatment and improve but only because they believe they are receiving an effective treatment
-it’s not imaginary. there’s an actual physical or psychological improvement
double-blind placebo control study goal
to find out if the treatment is effective
what else can the double-blind placebo control study suggest other than placebo fx?
may instead suggest maturation, history threats, regression to the mean, testing threats or instrumentation threats
“to assess whether an IV is a cause of variation in the DV, we assess…”
how much of the total variability in the DV is due to the IV
total variability in the DV =
b/w groups variability and w/in groups variability
how much of each variability do we want (in total variability in the DV)
large b/w groups and minimal w/in groups
null effect
no significant covariance, causal effect or correlation b/w IV and DV
why were null effects rarely seen in peer-reviewed/popular articles in the past?
because science used to be biased in only reporting results w/ significant results
power in research
the likelihood that a study will yield a statistically significant result when the IV really has an effect
the chance that a study will produce a statistically significant result when the IV actually affects the DV
solutions to increasing b/w groups variability and reducing w/in groups variability are attempts to increase the _______
power of a study
define sunk cost
money, time, effort, etc spent (wasted) on non-sig/null studies
null effects should be reported ________
transparently
publication bias past vs present
past: null results weren’t likely to be published
present: publish results regardless of significance
