5.2 Internal and External Validity Flashcards
Internal Validity
Asks if the independent variable really made a difference or change to the dependent variable. Researchers must rule out other factors that could affect the relationship between the variables (sources of bias)
THREATS TO INTERNAL VALIDITY
History
- A specific event occurs either inside or outside the experiment setting that may have an affect on the dependent variable
- You are having an intervention at a historically black university about colorectal cancer and during this study a historically black actor dies of colorectal cancer.
Maturation
- Development (biological or psychological) processes that operate with an individual as a function of time and are external to the events of the study
- An experiment was done with highschoolers perception of STD risk collected annually over their 4 years of highschool.
Testing
- Research uses the same test with the recognition that the same test could influence subjects responses
- BSN students were tested with a pre-test and post-test on empathy using the same measure for both tests. All students scored higher on the second test
Instrumental Threats
- Changes in measure/observational technique of the variable that could account for changes in obtained measurement
- BP’s were measured with a machine in multiple facilities. One facility had broken machines so BP was measured by hand. This may throw results off
Mortality
Loss of study participant from the first data collection point to the second data collection point
Selection Bias
- Researchers do not use precaution in their attempt to gather a representative sample.
- A survey was done on satisfaction of hospice care. Families with positive experiences were more likely to participate in the survey (this is bias)
Internal Validity
- Threatened if there is an alternative explanation for study results
Threats associated with participants of the study (Maturation, Selection, Selection By Maturation)
Maturation - Alternative explanation formed by natural change. People develop and change, and sometimes issues disappear with natural aging. Ways to counteract this could be the use of a control group.
Selection - Any systematic difference in characteristics other than manipulated cause. Solution is to use a method of assignment to groups that assures that systematic differences is highly unlikely (randomization)
Selection by Maturation - Groups systematically differ in their rate of maturation. Can be eliminated by randomized assigned groups.
External Validity
Concerns generalizability of one study finding to the general population and other environmental settings.
- Questions under what condition and with what type of subjects the same result can be expected.
Selection Effects
Generalizability of results to other populations. Types of sampling methods and how subjects are assigned to testing conditions affects the generalizability of the study.
- Researcher has a limited sample size of patients with anemia due to rare nature of the disease. This limits generalizability of findings.
Reactive Effects
- Subjects response to being studied (Hawthorne Effect)
- Participants become nervous and behavior is altered because they know they are being studied by researchers
Measurement Effect
- Administration of pre-test and post-test in a way that limits generalizability
- Study on academic stress gives a pre-test. Students recognize their level of stress from the pre-test which impacts the degree of participation in stress-management interventions.
External Validity
- Whether hypothesized relation holds for other persons
History Threat
- Effect doesn’t generalize to other time periods.
Setting Threat
- Observed effect only holds in a specific setting
Setting Threat Artificiality
- Pretesting threat - Observed threat is only found when a pretest is preformed. Pretest makes people realize how bad their problem is making them more receptive to treatment
Reactivity - Participant reacts to the fact that they are participating in a research study.
Selection Threat - Hypothosized relation only holds for a specific subset of people or selection of participants for study is bias due to under or over representation of a certain subset.
History/Setting threats can be reduced by replicating study in a different time or repeating the study in different settings
Artificiality of research settings - repeat study in a more natural environment
Replication can also reduce threat of selection by repeating study with different group of subjects. Selection can also be reduced by using random sampling or probability sampling
Internal vs External Validity
Internal
- Selection bias
- Mortality
- Maturation
- Instrumentation
- Testing
- History
External
- Selection Effects
- Measurement Effects
- Reactive Effects
