Diagnostic Test Accuracy Flashcards
What are the 3 criteria for a diagnostic test?
3 Ps
Predicts= predicts whether patient has condition
Probability= Alters probability of patient having condition
Prognosis= Guides treatment choices of patient
What are the reasons for performing a diagnostic test?
Detection or exclusion of condition
Reassurance
Medico-legal
Following protocol
What is evaluation bypass and why does it occur?
Where tests bypass the evaluation stage lead to tests with low efficacy and low accuracy being used
Due to:
- enthusiasm and convictions of researchers or clinicians backing the test
- commercial pressures when money involved in the test being passed and used
What are the consequences of using diagnostic tests inappropriately?
Waste of money and resources
Consequences associated with test errors if the test has low specificity or sensitivity
I.e. consequence of FP and FN
What is test accuracy? What can be looked at for indications of test accuracy?
How good a test is at reaching the correct diagnosis
Indications:
-sensitivity and specificity
-FP and FN rate = test errors
I.e. calculated based on comparison between the disease state estimated by index test and best estimate of true disease state measured by reference standard
What other important features does test accuracy need to be balanced with to determine efficacy of test?
Cost- is new test cheaper
Accessibility- is new test easier to perform
Less invasive
Safer
Quicker results (intervention can be started sooner)
What components of the testing process can influence patient outcomes?
Test process
-invasive test might come with own risk factors that might effect patient outcomes i.e. ERCP and pancreatitis
Timing
-length of test might mean it is not suitable for all patients or settings
Feasibility of test
-monetary or resource cost of performing the test
Timing of results
-can effect promptness of starting intervention
Interpretability
-does it require specialist training to interpret
Accuracy
-risk of test errors
Timing of diagnosis
Decision confidence and yield
What are the advantages of test accuracy studies?
Readily available compared with RCT
Don’t require large sample sizes
Answers easy to obtain
Cheap
What are the components of a diagnostic test accuracy question?
PITR
Participants
- prior tests outside of index and reference standard i.e. clinical examination or history
- presentation i.e. duration and severity of symptoms
- do they reflect the general/target population
Index test
-need to consider the conduct (experience and skill of operator) and technology
I.e. factors which can influence or lead to variation in the index test outcome
-less accurate than reference standard
Target disorder
-needs to be specific
Reference standard
- most accurate method available to detect target disease
- often comprises of more than one test
What are the different components of a critical appraisal of diagnostic test accuracy?
Internal validity
- characteristics of population i.e. has there been exclusion of particular population i.e. hard to treat patients
- bias = spectrum/review/verification
Results
- has 2x2 table been constructed
- summary measures of accuracy
- precision of estimates
- statistical significance between index tests
Applicability/external validity
- clear study questions
- population characteristics
- cost and acceptability by patients
- does definition of target condition match
Impact of using the test
- were all outcomes important to the patient/population considered
- other methods of test evaluation to consider
What is spectrum bias?
When does it occur in the study process?
What are the consequences of this form of bias?
What should you look for in the study as signs that efforts have been made to minimise spectrum bias?
Specific patients excluded from having the index test meaning that the population tested is not representative of general population the test will be used on
I.e. hard to diagnose patients or patients that clinician thinks will struggle to comply with the tests
Occurs at study sample stage
Makes the test appear more accurate than it is
Look for:
- clear description of characteristics of population being tested
- whether there has been any exclusion based on specific characteristics
- whether the patient population is representative of the desired population wanting to test
What is review bias?
When does it occur in the study process?
What are the consequences of this form of bias?
What should you look for in the study as signs that efforts have been made to minimise spectrum bias?
When the index test and reference standard are not reviewed blind to the results of the other which can influence the interpretation of the tests (especially in subjective tests)
Occurs when interpreting the results of tests
Makes the index test appear more accurate due prior knowledge of reference standard influencing interpretation
Look for:
- person interpreting the index test is independent to the person interpreting the reference test and is not informed of the results of the other test until after interpretation
- try and have the index test performed prior to reference standard to remove risk of influence
What is verification bias?
When does it occur in the study process?
What are the consequences of this form of bias?
What should you look for in the study as signs that efforts have been made to minimise spectrum bias?
What are the 2 types of verification bias?
When participants do not receive an index test and reference test meaning that the results of index cannot be verified against reference standard
Can occur if participant has negative index test so reference standard not performed
Leads to either and under or over estimation of accuracy of index test
Look for:
-all participants received index and reference standard regardless of results of the tests
2 types:
- partial= proportion of population do not receive both
- differential= more than one reference standard use
What is specificity?
What does a high specificity mean?
What is it used to indicate in a DTA?
Proportion of healthy individuals who are correctly identified as healthy
TN/TN+FP
High specificity:
-most have low FP rate i.e. inverse of specificity
I.e. 88% specificity= 12% FP
-positive results is more likely to be TP than FP
Used to RULE IN conditions
-due to positive results being more likely to be TP than FP
What is sensitivity?
What does a high sensitivity mean?
What can is it used for in DTA?
The proportion of people with a outcome correctly identified
TP/TP+FN
High sensitivity:
-means that there must be a low FN
I.e. 70% sensitivity= 70% of people with outcome correctly identified and FN rate= 30%
-negative result is more likely to be TN compared with FN
Used to RULE OUT conditions due to negative results more likely to be TN than FN with high sensitivity
