Lecture 7 Flashcards
____ is the key to controlling the outcomes from experimental research.
design
6 general issues in experimental design:
- internal validity
- external validity
- frame of reference
- longitudinality
- frequency
- nested factors
Internal validity is …
the basic minimum without which any study is not interpretable
Internal validity is the extent to which the results of a study can:
- be attributed to the design of the study
- eliminate any rival hypothesis or events influencing the study
To gain internal validity, the researcher attempts to control and/or eliminate possible _____ variables.
extraneous
Internal validity is particularly important in ______ studies as well as other studies that establish _____-_____ or _____ _____.
- experimental
- causal-effect
- causal relationships
When we think the independent variable produces a change in the dependent variable, what do we have to make sure of?
- must rule out the possibility of other factors producing the observed effect
- must consider the appropriateness of the methods (participants, experimental design, measurements)
Main threat in internal validity:
unmeasured process accounting for the observed results
Secondary threat in internal validity:
responses do not correctly reflect underlying dimensions
How to minimize threats to internal validity:
- random assignment of participants into treatment and control groups
- ID confounding factors
- multiple methods approach provides evidence leading to similar results
3 main categories to threats to internal validity:
- single group threats
- multiple group threats
- social threats
Single-group threats:
a threat to internal validity that occurs in a study that uses only a single program or treatment group and no comparison or control
Multiple-group threats:
an internal validity threat that occurs in studies that use multiple groups
Social threats:
an internal validity threat that arise because social research is conducted in real-world human contexts where people will react to not only what affects them, but also to what is happening to others around them
Types of single-group threats:
- history
- maturation
- testing
- instrumentation
- mortality
- regression
History (single-group threat):
- events occurring during he experiment that are not part of the treatment
- eg. children watching Sesame Street and math score
Maturation (single-group threat):
- process within the participants that operate as a result of time passing
- eg. children mature and gain muscle strength
Testing (single-group threat):
the effects of one test on subsequent administrations of the same test
Instrumentation (single-group threat):
changes in instrument calibration, including lack of agreement within and between observers
Mortality:
loss of participants from comparison group for nonrandom reasons
Regression:
the fact that groups selected based on extreme scores are not as extreme on subsequent testing
Multiple group threats to validity is ____ ____:
- selection bias
- any factor other than the program that leads to posttest differences between groups
Multiple-group threats types:
- selection-history
- selection-maturation
- selection-testing
- selection-instrumentation
- selection-mortality
- selection-regression
Selection-history:
A threat to internal validity that results from any other event that occurs between pretest and posttest that the groups experience differently
Selection-maturation:
A threat to internal validity that arises from any differential rates of normal growth between pretest and posttest for the groups.
Selection-testing:
Selection-testing: A threat to internal validity that occurs when a differential effect of taking the pretest exists between groups on the posttest
Selection-instrumentation:
A threat to internal validity that results from differential changes in the test used for each group from pretest to posttest
Selection-mortality:
A threat to internal validity that arises when there is a
differential nonrandom dropout between pretest and posttest
Selection-regression:
A threat to internal validity that occurs when there are
different rates of regression to the mean in the two groups
Social interaction threats occur because…
- the various groups participating in the
study and the researchers carrying out the study are aware of each other’s existence and role in the study - many of these people come in contact with each other
- try to isolate
Why is it hard to randomly assign and isolate to get rid of social interaction threats?
it introduces external validity (generalizability) concerns
4 types of social interaction threats:
- diffusion or imitation of treatment
- compensatory rivalry
- resentful demoralization
- compensatory equalization of treatment
Diffusion or imitation of treatment:
Occurs because a
comparison group learns about the program either directly or indirectly from program group participants.
Compensatory rivalry:
Occurs when one group knows the program another group is getting and, because of that, develops a competitive attitude with the other group
Resentful demoralization:
Occurs when the comparison group knows what the program group is getting and becomes
discouraged or angry and gives up