Lecture 7: Validity, Accuracy, Sensitivity and Friends

Question 1

What is validity?

Answer 1

The extent to which a method gives you the “correct” answer i.e. measures what you want it to measure

Question 2

What is absolute validity?

Answer 2

the extent to which a measurement or method accurately reflects the true value or reality of what it is intended to assess, without bias or error

Question 3

What does the reference method require for absolute validity?

Answer 3

Reference method requires discreet or hidden observation or recording of what a person consumes, without their awareness, to ensure accurate and unbiased data

Question 4

What is relative validity?

Answer 4

the degree to which the results of one measurement or method agree with those of a reference or gold-standard method, rather than directly reflecting an absolute true value

Question 5

What is the reference method for relative validity?

Answer 5

Reference method is another indirect method of assessing intake

Question 6

What is repeatability?

Answer 6

The extent to which a method gives you the same answer each time you use it

Question 7

What are issues to consider in an FFQ “validation” study?

Answer 7

• Appropriate reference method
• Study sample
• Timeframe
• Methods

Question 8

What is accuracy?

Answer 8

Describes the extent to which the measurement is close to the true value

Question 9

What is measurement error?

Answer 9

Difference between measured value and true value

Question 10

What is the observed intake equation?

Answer 10

observed intake = true intake + measurement error

Question 11

What are the two types of measurement error?

Answer 11

• Random error
• Systematic error

Question 12

What does measurement error lead to?

Answer 12

• Attenuation of relationships
• Bias

Question 13

What is random error due to?

Answer 13

Chance or normal variation

Question 14

What does’t random error change?

Answer 14

The mean

Question 15

What does random error increase?

Answer 15

Variability around the mean (stretches the tails)

Question 16

What does random error decrease?

Answer 16

Reproducibility

Question 17

What are the main sources of random error?

Answer 17

• Daily variation in consumption
• Inaccurate portion size estimation

Question 18

Can we remove random error?

Answer 18

Can’t remove totally

Question 19

What does systematic error change?

Answer 19

The mean - causes results to depart from true value in consistent direction (=bias)

Question 20

What does systematic error decrease?

Answer 20

Validity

Question 21

What are the main types of systematic error?

Answer 21

• Selection bias (non-representative sample)
• Measurement bias (incorrect calibration)

Question 22

Can systematic error be removed?

Answer 22

Can’t be removed by statistical analysis

Question 23

What is non-response bias?

Answer 23

When a specific subset of people:
- don’t volunteer to participate
- don’t adhere to intervention
- drop out

Question 24

Non-response bias makes the sample…

Answer 24

Non-representative

Question 25

What are stratagies to remove non-response bias?

Answer 25

• Minimize respondent burden
• Reminders
• Offer rewards
• Train interviewers
• Identify non-responders
• Over-sample groups at risk of non-responding

Question 26

What is respondent bias?

Answer 26

Bias introduced by the respondent

Question 27

What are the three main types of respondent bias?

Answer 27

• Low energy reporting
• Over reporting
• Social desirability and approval biases

Question 28

What are stratagies to reduce respondent bias?

Answer 28

• Pre-test method improvements
• Interviewer training
• Private interviews
• Identify flawed data
• Identify participants at risk

Question 29

What is interviewer bias?

Answer 29

Bias introduced by the interviewer

Question 30

What can interviewer bias be caused by?

Answer 30

• Incorrect recording
• Intentional omissions
• Poor cultural sensitivity

Question 31

What are stratagies to avoid interviewer bias?

Answer 31

• Standardised computer interviews
• Train interviewers
• Identify problem interviewers

Question 32

What are respondent memory lapses?

Answer 32

Errors of omission and commission (forgotten so lie instead)

Question 33

When may respondent memory lapses be more likely?

Answer 33

• Longer time period to be recalled
• Men
• Age extremes
• Distracting environment

Question 34

What are stratagies to avoid respondent memory lapses?

Answer 34

• Multiple pass interviewing techniques (interrupted recall, probe questions)
• Minimize time between intake and recall
• Work with information retrieval (visualisation)

Question 35

What is probably the largest source of error in dietary assessments?

Answer 35

Incorrect portion estimation

Question 36

What is incorrect portion estimation?

Answer 36

Failure to accurately quantify amount eaten

Question 37

When does incorrect portion estimation vary?

Answer 37

With type and size of food

Question 38

What are stratagies to decrease incorrect portion estimation?

Answer 38

• Measurement aids
• Train interviewers
• Train respondents

Question 39

Why is omission of supplements common?

Answer 39

• No uniform definition of a supplement
• No standard method to collect supplement data

Question 40

What does omission of supplements lead to?

Answer 40

• Systematic underestimation of nutrient intake
• Overestimation of prevalence inadequacy

Question 41

What are stratagies to avoid omission of supplements?

Answer 41

• Structured questionnare on long-term intake
• Questions on: brand, amount, frequency, duration of use, chemical form
• Interviewer sights supplement

Question 42

What is sensitivity?

Answer 42

How good the test/method is at correctly identifying people who have the disease/low status

Question 43

What is specificity?

Answer 43

How good the test/method is at correctly identifying people who are well/have adequate status