Reliability and Validity Flashcards
What is validity?
The extent to which we accurately measure what we intend to measure in the study population
Sources of invalidity
- Chance (random error)
- Bias (systematic error)
- Confounding (3rd variable)
What is internal validity?
Validity in the current context of the study
What is external validity?
Generalisability of the findings to other populations
What is chance invalidity?
Random errors of measurement or population selection
What is the effect of small sample size on validity?
Greater chance that it will not be representative of the study population, even if well-selected
Confidence intervals will be wide
Selection bias
A systematic error in the selection of participants into a study
Results in over- or under-estimation of association
Sources of information bias (measurement)
- Subject variation e.g. recall bias
- Observer variation e.g. unblinded observer
- Deficiency of measurement tools e.g. poorly-designed questionnaire
How to avoid selection bias
- Ensure representative sample
- Clear protocols
- Clear inclusion and exclusion criteria
- Minimize non-response and loss to follow-up
How to avoid information bias
- Clear ways of determining exposure and outcome
- Ensure tools are valid, reliable, standardised and calibrated
- Blind subjects and observers
- Train all staff well
- Validate data, test tools and pilot study
Difference between validity and reliability?
Validity = truth
Reliability = repeatability
Reliability is a requirement for validity
How to measure validity?
- Compare tool to gold standard
- Assess magnitude and direction of bias
Reliably wrong
Accurately repeatable measure which does not reach the truth
Unreliably wrong
Inaccurate measure which does not estimate the truth
Unreliable
Unreliable therefore invalid
