Exam 2 chapter 11 Flashcards
design confound
Poorly designed. Another variable varied systematically along with the intended independent variable
selection effect
Different independent variable groups have systematically different types of participants
Ex: first half of the class that arrives is handwritten and second half that arrives is laptop. We can’t use this to see if there was improvement because the earlier kids may be more serious about school
order effect
Alternative explanation because the outcome might be caused by an independent variable, or the order the levels were presented in.
Ex: chocolate study: chocolate will taste better the first time because its their first time eating it
One group pretest posttest
The researcher recruits one group of participants, measures them on a pretest, exposes them to a treatment intervention or change, and then measures them on a posttest. This differs from the normal pretest posttest because there is only one group not two and therefore theres no comparison groups
What does POIROT HeaD SMAC
P- placebo effect
O- order effect
I- instrumentation threat
R- regression threat
O- observer bias
T- testing threat
H- history threat
Ea
D- design confound
S- selection effect
M- maturation effect
A- attrition threat
C-demand characteristic
Maturation threat to internal validity
A change in behavior that emerges more or less spontaneously over time. To avoid this, include a comparison group to see if the treatment group was significantly more than the comparison group; if it was, it was not due to maturation. Reveals whether there is an effect of the IV above any maturation effect
Ex: children get better at walking, talking and math; plants grow taller but not because of any outside intervention
History threats internal validity
A historical or external factor that systematically affects most members of the treatment group at or around the same time as the treatment itself, making it unclear if the change was caused by the treatment. Must affect most people in the group, not just a few. Comparison group would help prevent history threats because it can reveal whether there is an effect of the IV above and beyond any effect of history
Ex: seasons change, political events (measuring kilowatts used in september and november there is a decrease but thats because people are using less AC in november)
Ex: rambunctious campers may have a sad event happen and then everyone calmed down (an external factor caused a change)
regression threats to internal validity
A statistical concept called regression to the mean. When a group average mean is unusually extreme at time 1, the next time that group is measured (time 2), it is likely to be less extreme and closer to its typical or average performance (closer to the mean) Works at both extremes an usually good performance will regress downward next time and an unusually bad performance will likely progress upward next time
Only occurs when a group is measured twice and only when the group has an extreme score at the pretest. Prevention: comparison group and a careful inspection of the pattern of results
Ex: a person is usually happy but the bad weather made you cranky that day the next time your mood is measured, you will likely be happy (your average)
Attrition threats to internal validity
When a certain kind of participant drops out. Problem when its systematic (related to IV). Prevention: when participants drop, researchers remove their data from the study or check the pretest scores of the dropouts. If they have extreme scores on the pretest, their attrition is more of a threat to internal validity, then if the scores are closer to the group average.
Ex: a study measuring rambunctious behavior at a camp. If a normal camper leaves early, its not a big deal, but if a rambunctious camper does, then it is a problem for the study
Testing threats to internal validity
Kind of order effect. A change in the participants caused by experiencing a DV more than once. May become more practiced leading to improved scores or more fatigued that could lead to worse scores. Testing threats include practice effects
Prevention: use only posttest, or if pretest is also used then they may opt to use alternative forms of testing. Comparison group: if treatment group has a larger change than the comparison group then testing threats can be ruled out.
instrumentation threat
When a measuring instrument changes over time. Prevention: use posttest only, or make sure pretest and posttest are measured equally
Ex: people judging the campers became more lenient the campers arent becoming less destructive
Or a researcher uses different forms for the pretest and posttest but the two forms are not sufficiently equivalent
combined threats
Selection-history threat: Study with pretest posttest. An outside event or factor affects those at one level of the independent variable
Selection-attrition threat
observer bias
When researchers expectations influence the interpretation of the results. Comparison groups can’t help with this. Threatens internal and construct validity.
demand characteristics
Problem when participants guess what is supposed to happen in a study and change their behavior in the expected direction
how do you avoid observer bias and demand characteristics?
Double bind study: neither participants nor researchers who evaluate them know who is in the treatment and comparison group
Masked design: participants know which group theyre in but observers do not
placebo effect
When people receive a treatment and really improve but only because the recipients believe they are receiving a valid treatment
Effects aren’t imaginary; they have been shown to reduce real symptoms. Double blind placebo: a study that uses a treatment group and a placebo group in which neither the researchers nor the participants know who is in which group
null effects
the IV does not make a significant difference in the DV
(the 95% CI of the effect includes zero
What if the IV does not make a difference?
* not enough between-groups difference
* too much within-groups variability
* really no difference
ex: two bowls of salsa one with 2 shakes of hot sauce and one with 4 shakes someone may not be able to tell the difference (not enough difference between groups) or each bowl contains many ingredients, making it hard to detect a change in hot sauce because of the other flavors (too much within group variability)
what happens when there not enough differences between groups
weak manipulations
insensitive measures
ceiling effect
floor effect
measurement error
A human or instrument factor
that can randomly inflate or deflate a person’s score on the dependent variable
All DVs involve a certain amount of
measurement error
Researchers try to keep those
errors as small as possible
A group’s mean on a DV will reflect
the true mean +/- random measurement error
When the distortions of measurement are random, they cancel each other out and do
not affect the group mean
A lot of measurement errors will result in scores that are more “spread out”, making it
harder to detect a difference between groups
Solutions: Use reliable, precise measurements: measurement errors are reduced when researchers use measurement tools that are reliable (internal,
interrater, test/retest) and valid (construct validity), measure more instances: if a researcher can’t find a reliable and valid measurement tool, the best alternative is to measure a larger sample of
participants
what does noise mean
Too much unsystematic variability within each group
what happens when there is less within group variability
Less within group variability our estimate of the group difference is more precise
the more unsystematic variability the overlap
more
statistical validity concern
The greater the overlap the less precisely the two group means are estimated and the smaller the standardized effect size
weak manipulations
the difference between levels of the IV is too small to matter or be meaningful
