10: Data Interpretation Flashcards
Components of Validity
- Internal Validity
- External Validity
Internal Validity
When there has been proper selection of study groups and a lack of error in measurement. Goal is to be able to link observed effect to exposure.
internal validity is concerned with the appropriate measurement of…
exposure, outcome, and association between exposure and disease
External Validity
implies the ability to generalize beyond a set of observations to some universal statement
Sources of Error
- Random Errors
- Systematic Errors
Random Errors
reflect fluctuations around a true value of a parameter because of sampling variability
Factors that contribute to Random Error
- poor precision
- sampling error
- variability in measurement
Systematic Errors
measurement bias
Poor Precision
Occurs when the factor being measured is not measured sharply. Can be increased by increasing sample size or the number of measurements.
Sampling Error
Occurs when the sample selected is not representative of the target population. Arises when statistics obtained for sample differ from values of parent population. No way to prevent a non-representative sample from occurring, but increasing the sample size can reduce the likelihood of sampling error.
Variability in Measurement
The lack of agreement in results from time to time reflects random error inherent in the type of measurement procedure employed. Goal is to obtain data that is objective, reliable, accurate and reproducible.
Systematic Errors (Bias)
Deviation of results or inferences from the truth, or processes leading to such deviation. Much more serious problem for validity than random errors. Can be introduced at any point in an investigation.
Systematic Errors grouped into…
- Selection Bias
- Information Bias
- Confounding
Selection Bias
Distortions that result from procedures used to select subjects or influence participation in the study. Arises when the relation between exposure and disease is different for those who participate and those who would be eligible for study but do not participate.
Information Bias
Can be introduced as a result of measurement error in assessment of both exposure and disease.
Types of Information Bias
- Recall Bias
- Interviewer/Abstractor Bias
- Prevarication (lying) Bias
Recall Bias
Better recall among cases than among controls
Interviewer/Abstractor Bias
Interviewers probe more thoroughly for an exposure in a sase than in a control. Avoided through blinding.
Prevarication (lying) Bias
Participants have ulterior motives for answering a question and thus may underestimate or exaggerate an exposure.
Confounding
The distortion of the estimate of the effect of an exposure of interest because it is mixed with the effect of an extraneous factor. An independent variable that varies systematically with the hypothetical variable under study. Occurs when the crude and adjusted measures of effect are not equal. (Difference of at least 10% between crude and adjusted measures)
Confounding errors can only be controlled for in the….
data analysis
Criteria of Confounders
- Be a risk factor for the disease
- Be associated with the exposure under study in the population from which the cases derive.
- Not be an intermediate step in the causal path between exposure and disease.
Simpson’s Paradox
Demonstrates that association can be reversed when confounding factors are controlled.
Reducing Selection Bias among Cases
- Develop explicit (objective) case definition.
- Enroll all cases in defined time and region.
- Strive for high participation rates.
- Take precautions to ensure representativeness.
Reducing Selection Bias among Controls
- Compare prevalence of exposure with other sources to evaluate credibility.
- Attempt to draw controls from a variety of sources.
Reducing Information Bias
- Use memory aids; validate exposures
- Blind interviewer, provide training, and use standardized data collection forms
- Blind participants as to study goals and classification status
Prevention Strategies
Attempt to control confounding through the study design itself. (Randomization, Restriction, and Matching)
Randomization
Attempts to ensure equal distributions of the confounding variable in each exposure category.
Randomization Advantages
convenient, inexpensive, and permits straightforward data analysis
Randomization Disadvantages
need control over the exposure and the ability to assign subjects to study groups in Intervention Study, and works well with only large sample sizes.
Restriction
May prohibit variation of the confounder in the study groups. Provides complete control of known confounders.
Restriction Disadvantages
- unlike randomization, can’t control for unknown confounders.
- may shrink pool of subjects to unacceptable level.
- findings may not be generalizable.
Matching
Matches subjects in the study groups according to the value of the suspected or known confounding variable to ensure equal distributions. Frequency vs. Individual
Frequency Matching
The number of cases with particular match characteristics is tabulated.
Individual Matching
The pairing of one or more controls to each case based on similarity in sex, race, or other variables.
Matching Advantages
- Fewer subjects required than in unmatched studies of same hypothesis.
- May enhance validity of a follow-up study
Matching Disadvantages
- Costly due to extensive searching and record keeping required to find matches.
- May introduce confounding rather than control for it.
Analysis (Statistical) Strategies to Control Confounding
- Stratification
- Multivariate Techniques
Stratification
Evaluate the effect of an exposure with levels (strata) of the confounder. Define homogeneous categories or narrow ranges of confounding variable. Then, combine stratum-specific efforts into overall effect by statistics
Stratification Advantages
- performing analyses within strata is direct and logical
- minimum assumptions must be satisfied for analysis to be appropriate
- computation procedure is straightforward
Stratification Disadvantages
- small numbers of observations in some strata
- variety of ways to form strata with continuous variables
- difficulty in interpretation when several confounding factors must be evaluated
- categorization produces loss of information
Multivariate Technique
Computers used to construct mathematical models that describe simultaneously the influence of exposure and other factors that may be confounding the effect. Able to use continuous variables vs. categoric data.
Multivariate Technique Advantages
- continuous variables don’t need to be converted to categorical variables
- allow for simultaneous control of several exposure variables in a single analysis
Multivariate Technique Disadvantages
- potential for misuse
- people don’t understand the different models and misapply multivariate techniques
Publication Bias
Occurs because of the influence of study results on the chance of publication. Positive results more likely to be published than negative results. May result in preponderance of false-positive results in literature. Bias compounded when published studies are subjected to meta-analysis.
Meta-analysis
combining several studies that address a set of related research hypotheses
