Internal and External Validity and Experimental Research Designs Flashcards
When an investigator conducts a research study, he or she ordinarily not only wants to find out if there is a ____ ____ ____ and ____ ____ but also wants to determine if that relationship is ____ to ____ ____ and ____.
Relationship Between Independent and Dependent Variables; Generalizable to Other People and Circumstances
The basic research questions that an investigator wants to answer can be expanded to include a third question:
Is there a relationship between the independent and dependent variables? If so, is the relationship a causal one? Can the relationship between the independent and dependent variables be generalized to other people, settings, times, and operations?
A research study is said to have ____ ____ to the extent that it provides accurate answers to the first two research questions and to have ____ ____ to the degree that it produces an accurate answer to the third question. The specific research designs are susceptible, to varying degrees, to factors that can ____ a study’s internal and external validity. Potential threats to internal and external validity are described in this section.
Internal Validity; External Validity; Limit
A study has ____ ____ when it allows an investigator to determine if there is a causal relationship between independent and dependent variables Internal validity is threatened whenever an investigator cannot control the three sources of variability described in the previous section.
Internal Validity
If an investigator cannot ____ the ____ of the ____ ____, ____ the ____ of any ____ ____, and/or ____ the ____ of ____ ____, he or she cannot be certain whether ____ ____ (or lack of variability) in the dependent variable is attributable to the independent variable or to some other factors Campbell and Stanley have identified eight “generic” extraneous variables that can threaten a study’s internal validity.
Maximize the Effects of the Independent Variable, Control the Effects of any Extraneous Variables, and/or Minimize the Effects of Random Error; Observed Variability
____ refers to any biological or psychological change that occurs within subjects during the course of a study as a function of time, is not relevant to the research hypothesis, and affects the status of most or all subjects on the dependent variable in a systematic way. Fatigue, boredom, hunger, and physical and cognitive development are potential ____ ____ that can limit a study’s ____ ____.
Maturation; Maturation Effects; Internal Validity
____ would threaten Study #1 if the psychologist assesses the effects of the self-control procedure by comparing the academic achievement test scores of a single group of children with ADHD before and after they are trained in the procedure and there is a five-month interval between testings. In this situation, any improvement in test scores might be due to learning that would have occurred without training in the self-control procedure or to a natural decrease in symptoms rather than to the effects of the procedure.
Maturation
The best way to control maturation is to include ____ than ____ ____ in the ____ and ____ ____ ____ to ____. Since subjects in all groups should be ____ to the ____ ____ ____, any observed differences between them on the DV at the end of the study can be ____ to the _ rather than to ____.
More than One Group in the Study and Randomly Assign Subjects to Groups; Susceptible to the Same Maturational Effects; Attributed to the IV; Maturation
____ threatens a study’s internal validity when an external event systematically affects the status of subjects on the dependent variable. Historical events are most likely to be a problem when a study includes only ____ ____ and the event occurs at the ____ ____ the ____ ____ is ____. In this situation, any difference in dependent variable performance before and after the intervention is applied might be due to ____ rather than to the _.
History; One Group; Same Time; Independent Variable is Applied; History; IV
In Study #2, if the psychologist uses a single-group pretest-posttest design to assess the effects of a moderate dose of a phenothiazine, any observed effect might be due to the drug or to other changes that occurred at about the same time the drug was administered (e.g., a change in hospital staff or policy). History is controlled by including ____ than ____ ____ in the study and ____ ____ ____ to ____. This procedure helps ensure that subjects in all groups are ____ ____ in terms of ____ to ____ ____ so that the investigator is better able to conclude that an observed difference between groups is actually due to the _.
More than One Group; Randomly Assigning Subjects to groups; About Equal; Exposure to External Events; IV
____ threatens a study’s internal validity whenever exposure to a test alters subjects’ performance on retesting (e.g., when taking a pretest affects subjects’ scores on the posttest). This threat can be controlled by administering the DV measure only ____ as a ____, by designing the measure in a way that ____ ____ and ____ ____, or by including at least ____ ____ in the study with all groups completing the ____ and ____ so that any difference between groups on the posttest cannot be attributed to the effects of the ____.
Testing; Once; Posttest; Minimizes Memory and Practice Effects; Two Groups; Pre- and Posttests; Postests
____: Changes in the ____ or ____ of measuring devices or procedures during the course of a study can confound the study’s results. For instance, if a rater’s ____ ____ over ____, any change in subjects’ pretest and posttest performance might be due to the ____ ____ ____ rather than to the effects of the _.
Instrumentation; Accuracy or Sensitivity; Accuracy Improves over Time; Rater’s Increased Accuracy; IV
____ is controlled by including more than ____ ____ in the study and ensuring that all groups are subject to the ____ ____ ____, by using the ____ ____ ____ and ____ with all ____, and by making sure that measuring devices and procedures ____ ____ ____ during the course of the study.
Instrumentation; One Group; Same Instrumentation Effects; Same Measuring Devices and Procedures with all Subjects; Do Not Change
____ ____: The tendency of extreme scores on a measure to “regress” (move) toward the mean when the measure is readministered to the same group of people is called ____ ____.
Statistical Regression; Statistical Regression
Statistical regression threatens a study’s internal validity whenever subjects have been selected because of their ____ ____ on the _ or a measure that ____ with the _. This threat is avoided by not including ____ ____ ____ in the study or by including ____ than ____ ____ and ensuring that all groups consist of subjects who are ____ ____.
Extreme Status; DV; Correlates; DV; Only Extreme Scores; More than One Group; Similarly Extreme
____ is a threat to a study’s internal validity whenever the method used to assign subjects to treatment groups results in systematic differences between the groups at the beginning of the study. Selection is often a problem when ____ ____ are used. It is controlled by ____ ____ ____ to ____ or, when random assignment is not possible, by administering a ____ to subjects to determine if the groups ____ ____ regarding the _.
Selection; Intact Groups; Randomly Assigning Subjects to Groups; Pretest; Differ Initially; DV
____ (____) poses a threat to a study’s internal validity when subjects who drop out of one group differ in an important way from subjects who drop out of other groups. Attrition is ____ to control, but ____ can help determine if dropouts and non-dropouts ____ with regard to their ____ ____ on the _.
Attrition (Mortality); Difficult; Pretesting; Differ; Initial Status; DV
____ with ____: When groups are initially nonequivalent, ____ can act alone and/or can interact with other factors to threaten a study’s internal validity. There would be an ____ between ____ and ____, for example, when one group of subjects is unintentionally exposed to an external event that does not affect subjects in other groups and the exposure affects their scores or status on the DV.
Interactions with Selection; Selection; Interaction; Selectin and History
____ comes from “out there” and occurs at around the same time that the IV is administered, while ____ reflects changes that occur within subjects as the result of the passage of time. ____ is actually an assignment problem.
History; Maturation; Selection
When a study has adequate internal validity, the investigator can conclude that observed variations in the (1) ____ variable were caused by variations in the (2) ____ variable rather than by other factors. Campbell and Stanley identified several “generic” extraneous variables that can limit a study’s internal validity.
(1) dependent; (2) independent
(3) ____ refers to an external event that is irrelevant to the research hypothesis but that occurs during the course of a study and affects subjects’ status on the (4) ____ variable. (5) ____ refers to changes that occur within subjects during the course Of a study as the result of the passage of time and that have a systematic effect on the DV.
(3) History; (4) dependent; (5) Maturation
Statistical (6) ____ is the tendency for very high and very low scores to move toward the mean on retesting. It threatens a study’s internal validity whenever subjects are selected to participate in the study because of their extreme scores on the (7) ____ variable measure. (8) ____ is a problem when subjects in different treatment groups are not similar in terms of important characteristics at the beginning of the study.
(6) regression; (7) dependent; (8) Selection
(9) ____ limits a study’s internal validity when subjects who drop out of one group differ in an important way from those who drop out of another group. Finally, (10) ____ can interact with history and threaten a study’s internal validity if one group of subjects is exposed to an external condition that does not affect subjects in other groups.
(9) Attrition; (10) selection
A research study has ____ ____ when its results can be generalized to other people, settings, and conditions. When discussing external validity, some authors distinguish between “____ ____” and “____ ____”: The former is used to describe the generalizability of research results to other people, while the latter refers to the generalizability of results to other settings. Ecological validity is a particular problem in ____ ____, which examine the relationship between independent and dependent variables in a laboratory or other non-naturalistic setting.
External Validity; Population Validity; Ecological Validity; Analogue Studies