11 Assessing Validity and Reliability

Question 1

Reliability

Answer 1

The degree of stability exhibited when a measurement is repeated under identical conditions
If you do something over and over again, will you get the same result?

Question 2

Validity

Answer 2

A measurement

An expression of the degree to which a measurement measures what it is trying to measure

Question 3

Precision and accuracy

1. Gold standard
2. Silver standard
3. Off-base model
4. Hit or miss model

Answer 3

1. Both
2. Good accuracy, poor precision
3. Good precision, poor accuracy
4. Poor both

Question 4

Screen versus diagnose

Answer 4

Screen: no preset notion of having a disease (it is done in an apparently healthy population - applying a test to a general population)
Diagnose: trying to find a disease in a particular person

Question 5

Screening definition

Answer 5

The identification of unrecognized disease or defect through application of tests, examinations or other procedures in apparently healthy people
No concern in that person about the disease being screened for
No diagnostic intent
Very low to low disease prevalence

Question 6

Diagnosis definition

Answer 6

Confirmation of presence or absence of disease in someone who is suspected of having it
People with symptoms of disease
Diagnostic intent
Low to high disease prevalence

Question 7

In order to have a screening program, what is necessary?

Answer 7

A pre-clinical phase

Question 8

Primordial prevention

Answer 8

Alter societal structures and thereby underlying determinants
Preventing the risk factors themselves

Question 9

Primary prevention

Answer 9

Alter exposures that lead to disease

Prevent the disease from occurring

Question 10

Secondary prevention

Answer 10

Detect and treat pathological process at an earlier stage when treatment can be more effective
Prevent symptomatic manifestation
Screening is here

Question 11

Tertiary prevention

Answer 11

Prevent relapses and further deterioration via follow up care and rehabilitation
Attempt to minimize the adverse impact of the diseases

Question 12

How do we decide if a screening program is worthwhile?

Answer 12

Disease must be important
Early treatment must be better than late
Prevalence of preclinical disease has to be high enough to justify costs
Simple to administer
Those being screened should be likely to comply
Should provide a true measure
Reproducible results
Cost-effective

Question 13

Sensitivity

Answer 13

The test correctly identifying those who have the disease
True positive rate
a/(a+c)

Question 14

Specificity

Answer 14

The test is negative in those who do not have the disease
True negative rate
d/(b+d)

Question 15

Positive predictive value

Answer 15

Percentage of correct results in those with a positive test result
a/(a+b)
True positives / (true positives + false positives)

Question 16

Negative predictive value

Answer 16

Percentage of true negative results in those with a negative test result
d/(d + c)
True negatives/ (true negatives + false negatives)

Question 17

Pre-test probability

Answer 17

Same as the prevalence, but applied to an individual

Question 18

Post-test probability measurements

Answer 18

The probability of the disease in an individual after a given test result

Question 19

Lead-time bias

Answer 19

Apparent increased survival duration because of screening

Screen detected cases survive longer without benefit of early treatment (no prolongation of life)

Question 20

Selection bias

Answer 20

Volunteers or compliers are better educated and more health conscious, thus they they have a better prognosis

Question 21

Length-time bias

Answer 21

Screening preferentially identifies slower growing or less progressive cases that have a better prognosis
At one point in time, more people are likely to have slow growing cancers compared to aggressive cancers

Question 22

Overdiagnosis bias

Answer 22

Too many false positives

Question 23

How can we properly evaluate if a screening diagnostic program or test is effective?

Answer 23

Randomized control trial