10 - Diagnostic Test Validity & Screening

Question 1

what is pre-test probability?

Answer 1

• decided by the clinician (after hearing history/exam etc) based on his/her experience w the disease and the signs/symptoms expressed by the patient
• clinician prioritizes based on most likely/most urgent etc
• could represent pre-test probability with a range and conduct a sensitivity analysis

Question 2

describe diagnostic uncertainty (3 possibilities)

Answer 2

• slide 3
  1) probability of the disease is low - no further testing warranted
  2) probability of the disease is high so decision to treat needs no further confirmation
  3) uncertainty remains -> diagnostic test
• we want to include people within the thresholds for our phase 3 type studies

Question 3

what is the “best” diagnostic test?

Answer 3

• provides the greatest reduction in uncertainty
• is the least invasive (w same validity)
• is the least expensive (w same validity)

Question 4

what is the study design for diagnostic testing?

Answer 4

• start w representative sample of patients whose diagnosis is uncertain (btw the 2 thresholds!!)
• undergo the new test and gold standard test
• construct the table comparing GS test and new test (slide 5)
• note that representativeness of sample = external validity

Question 5

how are patients selected for diagnostic testing?

Answer 5

• must enroll a representative sample for whom the clinician would face diagnostic uncertainty (or else risk spectrum bias)
• not “normals” who we know to be normal
• not patients w other diagnoses that are not suspected of having the disease of interest
• note the explanatory vs pragmatic approaches of selecting people from previous lecture (explanitory = including normals and severes)
• patients collected consecutively (more applicable/phase 3, not picking through that suggest a spectrum bias, but more consecutively -any/all patients that came into the clinic, even if they probs do not have diagnosis)

Question 6

what is spectrum bias?

Answer 6

• if we want to know how well the test will do in a population, have to make sure population is representative and recognize that this population changes as practice changes
• Practitioner has broad spectrum maybe of people w disease (ie mild to very severe) vs Tertiary care practice might be mostly severe

Question 7

what is the first thing you do for a diagnostic test validity study? next?

Answer 7

• determine if the GS test is appropriate (validity/reliability)
• next determine who will assess the test/gold standard - will they be blinded/are they skilled at the tests?

Question 8

what is a verification bias?

Answer 8

• when results of the test have influence on the decision to perform the gold standard
• GS extremely invasive or expensive maybe (so if person I study tests –’ve, clinicians maybe choose not to include them in the study then (now we are messing with sample representativeness and probably including in the final sample, more positive tested patients, which are prob more severe, making the test look better therefore)

Question 9

list the summary measures for diagnostic test validity

Answer 9

• sensitivity
• specificity
• positive predictive value
• negative predictive value
• ROC curves
• positive and negative likelihood ratio

Question 10

describe sensitivity

Answer 10

• how many patients w disorder have a positive result
• if a test has a high sensitivity, person w negative result can be ruled out for disease
• a/a+c = so from gold standard positive results, a=number positive, c = number negative)

Question 11

describe specificity

Answer 11

• the proportion of patients without the target disorder who have a negative test result
• if the test has a high specificity, a positive test rules in the disorder
• d/b+d = so from GS negative results, d= number of negative, b= number of positive)

Question 12

what is positive predictive value

Answer 12

• proportion of patients with a positive test result who have the disorder
• a/(a+b): of the positive results in the new test, which proportion are +/- in the GS test
• we don’t use this too much, it is going away from sensitivity/specificity

Question 13

what is negative predictive value

Answer 13

• proportion of patients w a negative result who do not have the disorder
• d/(c+d)

Question 14

what is a likelihood ratio/how do you calculate it

Answer 14

• the odds that a given level of a diagnostic test would be expected in a patient with the disorder
• positive LR = sensitivity/1-specificity (the odds that a positive test would be expected in a patient w the disorder)
• negative LR = 1-sensitivity/specificity (the odds that a negative test would be expected in a patient w the disorder)
• good test = high positive, low negative value!

Question 15

how do you convert a pre-test probability to a post-test odds? then back to post-test probability?

Answer 15

pre-test odds x likelihood ratio = post-test odds

• for pre-test odds -> say its 50% -> 1:1 chance of having outcome, so odds = 1
• sub into formula = (1:1 x 6.4) = 6.4:1 or 6.4
• convert 6.4 (probability as a decimal = odds ratio) back to % (just probability) = odds/odds +1 = 86.5%

Question 16

what is the post-test odds?

Answer 16

• the odds that the person has the disease

Question 17

what are the 2 questions addressed by diagnosis testing?

Answer 17

1. what is the diagnosis? (if levels are possible use these)
2. what is the best cutoff to determine positive or negative? (ie at which level of result do I make the least false predictions? at which point do we switch from negative to positive?)

Question 18

explain how to establish the cutoff point for determining positive and negative values

Answer 18

• which cut-off point would tell us that the GS test is required?
• it would be ideal if overlap btw those who do/do not require surgical repair is minimal
• use the ROC curve (slide 30/31)
• best cut point is the one in the upper left corner!

Question 19

why screen people?

Answer 19

• early diagnosis, insurance, protection of others, establish baseline for when things are “normal”

Question 20

define biologic onset

Answer 20

• interaction between individual, causal factors, and the environment
• actual time the disease starts (as early as conception)

Question 21

define early diagnosis

Answer 21

• individual is free of symptoms but changes can be detected if the right test is selected
• can detect presence before symptoms start

Question 22

define clinical diagnosis

Answer 22

• symptoms appear and the person seeks help

Question 23

define outcome

Answer 23

• recovery, disability, death

- what outcomes we are trying to avoid w the disease

Question 24

describe PICO for screening

Answer 24

• Population: those who could develop the disease but are currently disease-free (ie would not include men for ovarian cancer screening)
• Intervention: diagnostic test
• Control: diagnosis at time of signs and symptoms (receives what usually happens, so if this is typically the clinical diagnosis, this is what this group gets)
• Outcome: common outcomes experienced by patients with the disease

Question 25

describe screening design/methodology

Answer 25

• How would we design a study to detect whether screening occurs at an appropriate critical point?
• Randomize patients to screening or no screening, Follow over time, Treat disease as it is detected in both groups
• Same design as RCTs
• Will have same concerns (allocation concealment concerns, blind, outcome measures etc)

Question 26

screening is worthwhile only when…

Answer 26

When those who were screened (i.e. disease identified early) have a more favorable outcome than those who were not screened

Question 27

when will screened people have a better outcome?

Answer 27

• The test must detect disease when treatment is more effective or easier to apply than when it would had the disease been detected without screening

Question 28

what are the critical points for screening?

Answer 28

• the points which can make a difference in outcome
• CP 1 = after this point in the disease, can’t make difference in outcome (screening also not warranted) just telling us earlier that it’s too late to make a difference in outcome
• CP 2 = the only point where screening test makes sense
• CP 3 = CP falls after what you could diagnose in clinic (meaning can still make difference in outcome at time diagnosed in clinic – screening not necessary) – can still make a diff when signs and symptoms occur – bc of imperfect specificity and sensitivity, if you wait for symptoms to appear, more likely tp be accurate in diagnosing those who have disease
• RCT will figure this out for us = whether screening test appears before CP 2

Question 29

what are some screening paradoxes? list (3)

Answer 29

Early diagnosis will always appear to improve survival even when it is useless

• volunteer bias
• lead-time bias
• length-time bias

Question 30

screening - what is volunteer bias?

Answer 30

• Volunteers are healthier people (representativeness of population)

Question 31

screening - what is lead-time bias?

Answer 31

• The earlier you start counting years of survival, the longer people appear to survive
• slide 10

Question 32

screening - what is length-time bias?

Answer 32

• Preferential detection of slowly progressing disease
• Those who take longer to progress more likely to be caught by screening before critical point 2
• Need to make sure we are representative of population (don’t over-include people who progress slowly or quickly – representative of how many people progress at which phase through disease)
• If we include higher proportion of people who progress slowly, more likely to show that test is effective (overestimating its effectiveness)
• If we include higher proportion of people who progress quickly, more likely to show that test is not effective (underestimating its effectiveness)
• slide 11

Question 33

how do you know if screening is warranted? last 4 slides

Answer 33

• Does early diagnosis lead to improved clinical outcomes? (Must be able to show that the long-term beneficial effects of early treatment outweigh the detrimental effects of labeling and treatment)
• Is the accuracy of the test reasonable? (Psychosocial effects of mislabeling or being labeled earlier)
• Does the burden of disability warrant the action? (costs associated, is the disease rare, etc)
• Will patients comply with early treatment? (Are the consequences of non-compliance with early treatment sufficiently “bad”? Are the side effects of complying with treatment sufficiently mild?)