Lecture 7: Validity, Accuracy, Sensitivity and Friends Flashcards
What is validity?
The extent to which a method gives you the “correct” answer i.e. measures what you want it to measure
What is absolute validity?
the extent to which a measurement or method accurately reflects the true value or reality of what it is intended to assess, without bias or error
What does the reference method require for absolute validity?
Reference method requires discreet or hidden observation or recording of what a person consumes, without their awareness, to ensure accurate and unbiased data
What is relative validity?
the degree to which the results of one measurement or method agree with those of a reference or gold-standard method, rather than directly reflecting an absolute true value
What is the reference method for relative validity?
Reference method is another indirect method of assessing intake
What is repeatability?
The extent to which a method gives you the same answer each time you use it
What are issues to consider in an FFQ “validation” study?
- Appropriate reference method
- Study sample
- Timeframe
- Methods
What is accuracy?
Describes the extent to which the measurement is close to the true value
What is measurement error?
Difference between measured value and true value
What is the observed intake equation?
observed intake = true intake + measurement error
What are the two types of measurement error?
- Random error
- Systematic error
What does measurement error lead to?
- Attenuation of relationships
- Bias
What is random error due to?
Chance or normal variation
What does’t random error change?
The mean
What does random error increase?
Variability around the mean (stretches the tails)
What does random error decrease?
Reproducibility
What are the main sources of random error?
- Daily variation in consumption
- Inaccurate portion size estimation
Can we remove random error?
Can’t remove totally
What does systematic error change?
The mean - causes results to depart from true value in consistent direction (=bias)
What does systematic error decrease?
Validity
What are the main types of systematic error?
- Selection bias (non-representative sample)
- Measurement bias (incorrect calibration)
Can systematic error be removed?
Can’t be removed by statistical analysis
What is non-response bias?
When a specific subset of people:
- don’t volunteer to participate
- don’t adhere to intervention
- drop out
Non-response bias makes the sample…
Non-representative
What are stratagies to remove non-response bias?
- Minimize respondent burden
- Reminders
- Offer rewards
- Train interviewers
- Identify non-responders
- Over-sample groups at risk of non-responding
What is respondent bias?
Bias introduced by the respondent
What are the three main types of respondent bias?
- Low energy reporting
- Over reporting
- Social desirability and approval biases
What are stratagies to reduce respondent bias?
- Pre-test method improvements
- Interviewer training
- Private interviews
- Identify flawed data
- Identify participants at risk
What is interviewer bias?
Bias introduced by the interviewer
What can interviewer bias be caused by?
- Incorrect recording
- Intentional omissions
- Poor cultural sensitivity
What are stratagies to avoid interviewer bias?
- Standardised computer interviews
- Train interviewers
- Identify problem interviewers
What are respondent memory lapses?
Errors of omission and commission (forgotten so lie instead)
When may respondent memory lapses be more likely?
- Longer time period to be recalled
- Men
- Age extremes
- Distracting environment
What are stratagies to avoid respondent memory lapses?
- Multiple pass interviewing techniques (interrupted recall, probe questions)
- Minimize time between intake and recall
- Work with information retrieval (visualisation)
What is probably the largest source of error in dietary assessments?
Incorrect portion estimation
What is incorrect portion estimation?
Failure to accurately quantify amount eaten
When does incorrect portion estimation vary?
With type and size of food
What are stratagies to decrease incorrect portion estimation?
- Measurement aids
- Train interviewers
- Train respondents
Why is omission of supplements common?
- No uniform definition of a supplement
- No standard method to collect supplement data
What does omission of supplements lead to?
- Systematic underestimation of nutrient intake
- Overestimation of prevalence inadequacy
What are stratagies to avoid omission of supplements?
- Structured questionnare on long-term intake
- Questions on: brand, amount, frequency, duration of use, chemical form
- Interviewer sights supplement
What is sensitivity?
How good the test/method is at correctly identifying people who have the disease/low status
What is specificity?
How good the test/method is at correctly identifying people who are well/have adequate status
