DTA Flashcards
Why do we use diagnostic tests?
Detect/exclude disease, reassure we are not missing something, medico-legal or financial reasons, follow protocol, easy
How would you evaluate histories/examinations?
Consider them at diagnostic tests (DTA studies)
Why do we evaluate a medical test?
See if it will benefit a patients, cost considerations with similar efficacy, avoid evaluation bypass
Are medical tests or treatments regulated more?
Treatments
Which agency regulates medical tests?
Medical healthcare regulatory agency (MHRA)
What does the MHRA require of tests?
They are safe and measure what they say they do
What can be a big contributing pressure to medical tests and why?
Commercial pressure, as there is no requirement for the manufacturers to prove the benefit or say how it is best used.
What are consequences of inappropriate testing?
Cost, anxiety, discomfort, pain, unnecessary further treatments and adverse effects
What are consequences of false negatives?
Wrongly reassure patient, missed diseases, infectious diseases spread
What makes clinicians use one test over another?
Accuracy, cost, ease of use, quicker to give results, less invasive, safer, test process
What do DTAS compare?
Presence of condition estimated with index test compared to reference standard
Which 3 types of bias are important for internal validity of DTAS?
Spectrum, verification and review
When is a test useful?
Changes our ability to predict if person has a condition or not
In what circumstances does improved test accuracy not lead to improved patient outcome?
Not acted upon appropriately, practitioner error, diagnosis not changed, treatment errors
What aspects of a test are evaluated?
Accuracy, safety, benefit to patient, harm to patient, will a colleague get same result
In what ways can the administration of a test impact the patient outcomes?
Test process, timing test, feasibility of test
In what ways can the production of test results affect the patient?
Diagnostic test accuracy, timing of results, interpretability of results
In what ways can the diagnostic decision based on a test affect the patient?
Timing of diagnosis, diagnostic yield, diagnostic confidence
In what ways can the way the treatment decision is made based on a test affect the patient?
Therapeutic yield and confidence
In what ways can treatment implementation based on a diagnostic test vary and affect the patient?
Adherence and timing of treatment
What 2 things impact patients in outcome of a diagnostic test?
Treatment and test harms and direct effects
What are diagnostic tests used for?
Screening, diagnosis and surveillance
What percentage of diagnostic tests have no diagnostic errors?
1%
What are the 2 components of test accuracy?
Sensitivity and specificity
What is sensitivity?
The proportion of those with the disease that the test detects
What is specificity?
The proportion of those without the disease that have a negative result (e.g. 100% = no false positives)
What are the 4 steps of the basic design to assess test accuracy?
- Collect of patients suspected to have target disorder (similar to those test would be used on in practice)
- Perform index test
- Perform reference standard (assumed to be more accurate)
- Blinded cross-classification to compare results
What are the components of a test accuracy question?
PITR: Participants, index test, target disorder, reference standard
What is important to identify about the population in a DTA question?
Presentation of disorder (presence of symptoms, severity and duration) and any prior tests
What is important to identify about index tests in the question in order to apply in own setting?
The conduct (experience/skill of operator) and technology (old/new, fixed/mobile)
Can the reference standard comprise of more than 1 test?
Yes
What is the composite reference standard?
Combines results of several available tests to produce better indicator of true disease status?
What 4 questions do we need to appraise with DTAS?
- Internal validity (bias potential)
- What are results? (Numerical expression of test accuracy)
- Can I apply results to own patients?
- What would be impact of using index test in own patients?
What is spectrum bias?
‘Difficult to diagnose’ patients purposefully excluded so test appears more accurate
What shows a DTAS does not have spectrum bias?
Characteristics of population clearly described and the study avoids inappropriate exclusions (representative of those who would be tested in practice as suspected of having disease)
What is review bias?
Interpretation of an index test is not independent (and blind to) the reference standard and vice versa
What is verification bias?
A tendency for patients with negative index tests not to get the reference standard (esp if more invasive or difficult to access)
What does verification bias result in?
A tendency in over or under estimation of accuracy of index test
What description shows that all patients had all tests?
Fully paired
What should you consider when looking at the results of DTAS?
- Sensitivity & specificity: including precision and statistical significance of these measures of accuracy
- False negatives/positives and their consequences
Does sensitivity correlate to false positives or negatives?
False negatives
Does specificity correlate to false negatives or positives?
False Positives
When is sensitivity useful?
Ruling out a condition
When is specificity useful?
Ruling in a condition
What does a sensitivity or specificity of 0.5 indicate?
The test is uninformative
What adds up to 100% with sensitivity?
False negative rate
What does the 95% CI indicate?
That 95% of the time the true result is within this limit
Which 4 questions should you ask to determine applicability of DTAS?
- Clear study question?
- Population similar to own patients?
- Can index test be applied in the same way?
- Does definition of target condition match the one you want to identify in practice?
What elements of the study population should you consider when looking at applicability?
- Presentation of symptoms in terms of severity, duration and associated symptoms
- Prior tests: e.g. HX, examination
Which elements of the index test should you look at for applicability of DTAS?
- Technology: old/new, manufacturers, fixed/mobile, lab/bedside (rapid)
- Who performs and interprets: experience & skill
Which outcomes important for the patient should be considered by DTAS?
Safety, anxiety, repeatability, benefit for patient
Does the reference standard or index test need to be more accurate?
Reference standard!
What variation could change the prior tests, target condition and index tests so must be mentioned in the question?
The setting! E.g. GP vs A&E
When is a medical test useful?
Changes our ability to predict if an individual has a condition or not
What determines the potential for medical tests to do harm?
The size and type of medical test errors
What is important to assess when evaluating the validity of a test accuracy study?
Spectrum bias, review bias and verification bias
What are key factors to consider when judging generalisability of test accuracy estimates?
Spectrum of population to be tested and way index test was conducted
Which framework do you use to critically appraise DTAS?
CASP
What do the 2 initial screening questions in the CASP tool look at?
PITR and the reference standard
What is partial verification bias?
The result of the index test determined if the reference standard is received
What is differential verification bias?
Different patients receive different reference standards
Which aspects of the disease status should be clearly described to eliminate the possibility of spectrum bias?
Presenting symptoms, prior tests, disease severity, exclusion criteria
What does an ROC curve plot?
Sensitivity against specificity
If more than 1 index test is being compared, how do you assess if accuracy differences have occurred by chance?
Using p values and if CIs for estimates of sensitivity and specificity overlap
What is an artificial decision?
Where test accuracy is not based on using real time clinical decisions. eg if clinician knows their diagnosis is being reviewed they may have more confidence leading to falsely increased accuracy of index test.
What should you consider when looking at the outcomes importance to the individual or population?
Whether maximising sensitivity (correct identification of positives) or specificity (reducing harm due to false positive) is more important
Which research method do you need to adopt to identify impact of test use on patient outcome?
Trial of test use
What is pre-test probability?
The probability a patient will have the disease prior to index test e.g. based on signs and symptoms (clinical evaluation)
What is post-test probability based on?
The probability the patient has a disease based on the pre-test probability, sensitivity and specificity and result of the index test
If a post-test probability is not altered much by getting a positive result, what does this mean?
The specificity is low and the number of false positives is large
