Diagnostic Test Accuracy Flashcards
What are the 3 criteria for a diagnostic test?
3 Ps
Predicts= predicts whether patient has condition
Probability= Alters probability of patient having condition
Prognosis= Guides treatment choices of patient
What are the reasons for performing a diagnostic test?
Detection or exclusion of condition
Reassurance
Medico-legal
Following protocol
What is evaluation bypass and why does it occur?
Where tests bypass the evaluation stage lead to tests with low efficacy and low accuracy being used
Due to:
- enthusiasm and convictions of researchers or clinicians backing the test
- commercial pressures when money involved in the test being passed and used
What are the consequences of using diagnostic tests inappropriately?
Waste of money and resources
Consequences associated with test errors if the test has low specificity or sensitivity
I.e. consequence of FP and FN
What is test accuracy? What can be looked at for indications of test accuracy?
How good a test is at reaching the correct diagnosis
Indications:
-sensitivity and specificity
-FP and FN rate = test errors
I.e. calculated based on comparison between the disease state estimated by index test and best estimate of true disease state measured by reference standard
What other important features does test accuracy need to be balanced with to determine efficacy of test?
Cost- is new test cheaper
Accessibility- is new test easier to perform
Less invasive
Safer
Quicker results (intervention can be started sooner)
What components of the testing process can influence patient outcomes?
Test process
-invasive test might come with own risk factors that might effect patient outcomes i.e. ERCP and pancreatitis
Timing
-length of test might mean it is not suitable for all patients or settings
Feasibility of test
-monetary or resource cost of performing the test
Timing of results
-can effect promptness of starting intervention
Interpretability
-does it require specialist training to interpret
Accuracy
-risk of test errors
Timing of diagnosis
Decision confidence and yield
What are the advantages of test accuracy studies?
Readily available compared with RCT
Don’t require large sample sizes
Answers easy to obtain
Cheap
What are the components of a diagnostic test accuracy question?
PITR
Participants
- prior tests outside of index and reference standard i.e. clinical examination or history
- presentation i.e. duration and severity of symptoms
- do they reflect the general/target population
Index test
-need to consider the conduct (experience and skill of operator) and technology
I.e. factors which can influence or lead to variation in the index test outcome
-less accurate than reference standard
Target disorder
-needs to be specific
Reference standard
- most accurate method available to detect target disease
- often comprises of more than one test
What are the different components of a critical appraisal of diagnostic test accuracy?
Internal validity
- characteristics of population i.e. has there been exclusion of particular population i.e. hard to treat patients
- bias = spectrum/review/verification
Results
- has 2x2 table been constructed
- summary measures of accuracy
- precision of estimates
- statistical significance between index tests
Applicability/external validity
- clear study questions
- population characteristics
- cost and acceptability by patients
- does definition of target condition match
Impact of using the test
- were all outcomes important to the patient/population considered
- other methods of test evaluation to consider
What is spectrum bias?
When does it occur in the study process?
What are the consequences of this form of bias?
What should you look for in the study as signs that efforts have been made to minimise spectrum bias?
Specific patients excluded from having the index test meaning that the population tested is not representative of general population the test will be used on
I.e. hard to diagnose patients or patients that clinician thinks will struggle to comply with the tests
Occurs at study sample stage
Makes the test appear more accurate than it is
Look for:
- clear description of characteristics of population being tested
- whether there has been any exclusion based on specific characteristics
- whether the patient population is representative of the desired population wanting to test
What is review bias?
When does it occur in the study process?
What are the consequences of this form of bias?
What should you look for in the study as signs that efforts have been made to minimise spectrum bias?
When the index test and reference standard are not reviewed blind to the results of the other which can influence the interpretation of the tests (especially in subjective tests)
Occurs when interpreting the results of tests
Makes the index test appear more accurate due prior knowledge of reference standard influencing interpretation
Look for:
- person interpreting the index test is independent to the person interpreting the reference test and is not informed of the results of the other test until after interpretation
- try and have the index test performed prior to reference standard to remove risk of influence
What is verification bias?
When does it occur in the study process?
What are the consequences of this form of bias?
What should you look for in the study as signs that efforts have been made to minimise spectrum bias?
What are the 2 types of verification bias?
When participants do not receive an index test and reference test meaning that the results of index cannot be verified against reference standard
Can occur if participant has negative index test so reference standard not performed
Leads to either and under or over estimation of accuracy of index test
Look for:
-all participants received index and reference standard regardless of results of the tests
2 types:
- partial= proportion of population do not receive both
- differential= more than one reference standard use
What is specificity?
What does a high specificity mean?
What is it used to indicate in a DTA?
Proportion of healthy individuals who are correctly identified as healthy
TN/TN+FP
High specificity:
-most have low FP rate i.e. inverse of specificity
I.e. 88% specificity= 12% FP
-positive results is more likely to be TP than FP
Used to RULE IN conditions
-due to positive results being more likely to be TP than FP
What is sensitivity?
What does a high sensitivity mean?
What can is it used for in DTA?
The proportion of people with a outcome correctly identified
TP/TP+FN
High sensitivity:
-means that there must be a low FN
I.e. 70% sensitivity= 70% of people with outcome correctly identified and FN rate= 30%
-negative result is more likely to be TN compared with FN
Used to RULE OUT conditions due to negative results more likely to be TN than FN with high sensitivity
If a test has a low specificity, what are the consequences?
Low specificity= high FP rate
- therefore positive results more likely to be FP than TP
Consequences of FP:
- anxiety and stigma
- unnecessary further testing and investigations
- might discourage patient from further investigations
If a test has low sensitivity, what are the consequences?
Low sensitivity= high FN
-negative results is more likely to be FN than TN
Consequences of FN:
-Delayed diagnosis meaning disease might have progressed
I.e. might be harder to treat and have worse outcomes
-can affect public is condition is infectious i.e. can lead to spread of disease
What is the relationship between sensitivity and specificity? Why is this important to consider?
They are inversely proportional
I.e. increase in one will lead to decrease in another
Need to consider whether it is more important to have higher specificity or sensitivity in the context of the disease the test is testing for
I.e. are the consequences of FP or FN worse?
Why might the accuracy of a test differ between primary care and ED setting?
ED patient more likely to have already been assessed prior to having the test I.e. might already have idea of diagnosis
ED patient more likely to have severe condition i.e. need to have test with high sensitivity to maximise the proportion of people correctly identified
GPs might interpret tests differently due to having less practice
GPs may only perform small number of these tests
What criteria are you looking for to see if test can be applied to own population?
Whether there is clear questions
If population spectrum is similar to own patient population
If index test can be applied on the same way
(Do you have the resources i.e. monetary and skilled individuals)
Definition of target audience same as the one identified in own population
Will the index test or reference standard have higher sensitivity/specificity?
The reference standard will always have higher sensitivity and specificity because it is the best available test for the condition
What can diagnostic test accuracy studies not be used to show? What could potential influence this? What could be done to assess the influence of diagnostic test on patient outcomes?
Cannot provide evidence of impact of test on patient outcomes
Factors influencing the impact of test on patient outcomes:
- correct interpretation of test
- trust in results and own ability to interpret
Could do RCT comparing difference in patient outcomes with test use vs no test use
