Critical Appraisal Flashcards
Critical appraisal algorithm/mnemonic?
WSSI CCF TCA CHL
What is Internal Validity?
Did the intervention definitely result in the change in the dependent variable.
Affected by confounds, bias, error
What is External Validity?
Whether your results can be applied to groups other than the one you studied. Considering: Setting, Time, Population factors.
What is selection bias what are the two important (imo) types?
Selection bias is bias that has been introduced through the selection of groups/individuals in a study.
Sampling bias
- Sample is non-random, this can affect external validity
Attrition bias
- Loss of participants has caused a potential difference in exposure to the intervention
What is the difference between efficacy and effectiveness trials?
Efficacy can be defined as the performance of an intervention under ideal and controlled circumstances.
Effectiveness refers to its performance under ‘real-world’ conditions.
What are the different types of variables? (Psychology, let’s hope it applies)
Categorical
- Ordinal: they have a rank
- Nominal: No rank/order
Continuous
- Ratio: has true zero, e.g. height, weight, bp
- Interval: no true zero, e.g. temperature
What questions do you need to ask when thinking about the justification of the study?
- Was there a clear gap
- Does this study address this gap
- Were the hypotheses clearly set out
For patient characteristics what percentage should they roughly be within each other?
Roughly within 5%, especially for larger studies
What is a confounder variable?
A variable that could have effected to the outcome. (It is correlated to both the intervention and the outcome)
If added to model and it has an effect on the results it is a confounder.
What percentage follow up is acceptable?
80% is good
70% is okay
60% is poor
What is Type 1 and Type 2 error?
How do you assess for each?
Type one error is false positive
- to assess look at the p value, <0.05 is 95%, and 0.01 is 99%.
Type two is false negative
- To assess essentially look for power
- 80% is good, 90% is better (90=0.1)
What percentage or beta value is acceptable for power?
80% is acceptable, 90% is good
0.1 = 90%
0.2 = 80%
What can a confidence interval tell you about an intervention, how can you work this out?
Confidence interval can help to tell the likely effect, so they are the spread of likely effects. i.e. If the 95% confidence intervals are -5 to -15, then the effect could be as small as -5 mmHg but as large as -15mmHg.
What is Intention to Treat (ITT)?
ITT is the principle that whatever group the pt was in, even if they stopped the intervention, they are still analysed in that group
If it is done, then it reflects practice well, it takes into account the people who will stop taking the drug due to side effects.
What is Per protocol analysis?
Per protocol analysis, this is the opposite of ITT, and uses only the patients who complied with the treatment, shows the effect if people do adhere.
What is Number needed to treat (NNT), how do you calculate this?
Calculate by doing 100/difference in percentage between groups.
e.g. if 50% of people die (categorical) following a new ‘life saving drug’ in the intervention arm and 60% in the placebo arm, the absolute difference is 10% and the NNT= 100/10=10.
What does MCID stand for? What can it be used for?
Minimally clinically important difference.
Needed if the measurement is a questionnaire, can be used to tell if the intervention is meaningful if it has achieved the MCID.
What is opportunity cost?
Opportunity cost - what service might need to be stopped if the NHS is cash strapped.
What is the QALY?
Quality adjusted Life Year (QALY) for an intervention – below £30,000 for each year it provides fo good quality life.
WHat do you need to consider in the sample point?
- Think about the sample exclusion/inclusion criteria. It can be appropriate to introduce exclusion criteria in order to target the population that may benefit, this however needs to be stated clearly. If they have not clearly stated this then the results may be misleading, as they have reduced the variance to get a statistically significant result.
- The sample may not apply to your patient (or will apply loosely)
- Method for Inclusion/exclusion criteria. Scan the baseline table for patient characteristics – think how this relates to your patient(s)
- Look at consort diagram for any drop off
What do you need to consider in the setting point?
- Think about setting: Community care, Primary care, Secondary care, tertiary, UK or other? This matters as:
- The spectrum of comorbidities differ
- The ease of diagnosis differs
- The health system may function differently
Therefore, the Intervention may work better in certain settings.
What do you need to consider in the intervention and control point?
- Is the intervention actually going to be applicable to your practice, think about time, resources and input.
- Is it an efficacy or effectiveness trial?
- What type of control is it?
- Attention control
- Usual care control
Usual care is usually actually the most meaningful, as it is pragmatically what patients will receive.
Read the Methods section.
What do you need to consider in the Confounding/randomisation point?
- Did randomisation work?
- Read the method – see if anything in there could have undermined the randomisation
- Eyeball the baseline table with age and gender variables, they should be within 5% of each other for large studies
- If you have identified a problem ask whether that variable could be a confounder, they might have added it to the statistical model and seen if it had an effect?
If it is a confounder, have they attempted to control for it? (e.g. ANOVA etc)
What do you need to consider in the clinically important outcomes included point?
- Any key outcomes missing. E.g. quality of life.
- Were the outcomes valid e.g. all the measurements measuring exactly the same thing, if questionnaire need the Minimally clinically important difference (MCID).
- Were the outcomes measured the same way in both groups.
