Validity and Reliability

Question 1

What is the validity?

Answer 1

A valid measure of association describes the true (real) situation accurately

Question 2

Types of validity.

Answer 2

Internal – current context/study
– External – can the study findings be
extrapolated to other groups? (Generalisability)

Question 3

What is reliability?

Answer 3

Reliability is about CONSISTENCY/ REPEATABILITY of measurements

Question 4

What affects internal validity?

Answer 4

Chance (Random error)
* Bias (Systematic error)
* Confounding – 3
rd variable

Question 5

How to prevent random errors?

Answer 5

Big enough sample size
– Test
questionnaire/equipment

Question 6

Define bias.

Answer 6

Systematic error in the design, conduct (collection of data) or analysis of a study that
leads to an incorrect estimate of the exposure-disease association – invalid
■ The researcher influences the results towards a particular outcome through the
collection, analysis, interpretation, publication or review of the data

Question 7

Describe the 2 types of Biases.

Answer 7

Selection Bias
■ Systematic errors in the selection of study participants
– Information Bias (measurement or misclassification bias)
■ Systematic errors in the collection of information from study participan

Question 8

How to prevent selection bias?

Answer 8

Understand your study population and how to access them and keep them in your study;
Define your population
Ensure sample is representative
– Clear protocols
– Clear inclusion and exclusion criteria
– Minimize non-response and loss to follow-up

Question 9

How to prevent information bias?

Answer 9

Clear ways of determining the exposure and the outcome
– Ensure tools are valid, reliable, standardized & calibrated (NB questionnaire lecture)
– Blind subjects and observers
– Train all staff well
– Validate data, test tools and pilot study

Question 10

Describe confounding.

Answer 10

■ A ‘third variable’ that confuses the association
between exposure and outcome
■ Lead us to the wrong conclusion in our examination
of the E-D association
■ Can create the appearance of an association when
none exists, or hide it when it does exist
■ Impact: overestimation or underestimation of
association Poor validity
The possible confounders in a particular study will depend on what the exposure and
outcome are.
– They are risk factors for various health outcomes

Question 11

How do you identify a confounding factor?

Answer 11

1) Is it a risk factor for the
outcome?
2) Is it associated with the
exposure?
3) Is it in the causal pathway
between the exposure and
outcome?

Question 12

How can we control confounding factors?

Answer 12

■ Randomisation: If random allocation of participants to
exposure status is successful, then there will be equal
distribution of the confounder between exposed and
unexposed
■ Matching:
– Case-control studies
– Select controls to make sure they are similar to the case
with respect to the confounder
– Matching may be of two types:
■ group matching- the controls are selected in a way that the
proportion of controls with a certain characteristic is
identical to the proportion of cases with that characteristic
■ individual matching- a control is selected similar to the
case with respect to the characteristics considered to be
potential confounders
– Leads to analytic complexities….. don’t try without expert
input
■ Restriction
– Limit who can get into your study– prevent variation in
the confounding variable
– Only enrol those C+ or C- participants – not both
– E.g. if age is a confounder, restrict entry into the study to
only the young (or the old

“Statistical adjustment”
Calculating the exposure-disease association in a way that removes the effect of the
confounding variables
■ Stratified analyses
■ Standardisation
■ Regression models