Critical appraisal Flashcards
QR
Question
- type of study
- hierarchy of evidence
Relevance
- disease prevalence
- serious/fatal/public health issue
- implications of disease
PICOK
Population
- number and demographics of patients
Intervention
- have they justified it? e.g., previously shown promise, know therapeutic drug doses
Control
- placebo controlled
- compared to gold standard
Outcome
- primary outcome
Key findings
- brief summary
RAMBOS - internal validity
(RAM part)
Recruitment
- consecutive (reduces risk of selection bias) vs non-consecutive
- multicentre vs single site –> from primary, secondary or tertiary care
- are they representative of disease population?
- what are the inclusion/exclusion criteria? Is it too strict
Allocation
- randomisation reduces risk of confounders and selection bias
- blinding reduces risk of performance bias and placebo effect vs open label
- stratification vs minimisation
Maintenance
- attrition bias i.e., drop out rate
- why were they lost? e.g., side effects,
laboursome intervention, lack of motivation
- generally <20% to preserve power and type 2
error
- treatment outside the intervention e.g., number of further interactions with HCP
RAMBOS - internal validity
(BOS part)
Baseline characteristics
- similar to target population
- similar between groups –> shows if randomisation was successful
- accounts for co-morbidities
Outcome measures
- TRADO
- what are they not looking at?
- safety outcomes
Statistics
- does the data justify their conclusions?
- sufficient sample size? check full paper for power calculation
- appropriate statistical tests for outcome measurements?
- Cohen’s d effect size
- intention to treat vs protocol analysis
TRADO
Type e.g., clinical/composite/surrogate or objective/subjective
Relevance - to patients and HCP e.g., looking at side effects/QoL
A priori if a registered RCT –> reduces risk of data dredging
Duration of follow up - is it adequate?
Observation bias - have they standardised measurements and cut offs e.g., severity indexes rather than interviews which can lead to recall and response biase
Bias
Selection - patient not representative of intended population – reduced by randomisation
Performance - patients and HCPs may behave differently when they know whether they are receiving the intervention or not – reduced by blinding
Observation - tendency to see what we want or expect to see – reduced by blinding
Attrition - unequal loss of participants from RCT arms – reduced by using intention to treat analysis
Reporting - findings influenced by direction of results – reduced by undertaking a comprehensive search strategy/Publication bias
Define power
Likelihood of detecting a significant difference when it exists
Therefore avoiding type 2 error
Depends on sample size
Define p value
Probability that the association detected has arisen by chance
Typically 0.05 is considered significant to reject null hypothesis
Depends on sample size
Define CI
Range of values within one can be 95% certain that the true population value lies
For ratios → if the value crosses 1 = not statistically significant
For absolute values → if the value crosses 0 = not statistically significant
Define incidence
Rate of new cases over a defined time period (cohort study)
Define prevalence
Proportion of the population with outcome at a specific point in time (cross-sectional study)
Define absolute risk reduction (ARR)
Risk of event in control (5%) - risk of of event in intervention (2%) = 3%
Define relative risk reduction (RRR)
Absolute risk reduction (3%)/ risk of event in control (5%) = 0.6%
Define number needed to treat (NNT)
1/ARR
Define odds ratio
Ratio of exposed to non-exposed in an outcome of interest
1.6 = 60% more likely to have been exposed if disease is present
Define hazard ratio
Used for survival analysis at any given time point
0.79 = 21% less likely to die than control at any given time
Define type 1 error/ alpha error / false positives
rejecting the null hypothesis when it could be true i.e. concluding there is significance when there is none
Define type 2 error/ beta error / false negatives
Inadequate sample size so significance cannot be detected
Accepting null hypothesis when it is false
Define sensitivity
how likely you are to get a true positive
Define specificity
how likely you are to get a true negative
Define PPV
how likely your positive result is a true positive
Define NPV
how likely your negative result is a true negative
Statistical tests
RP = external validity
Resources
- specialised equipment? drug availability? available throughout the world?
- cost effectiveness and NNT
- QALYs
Population
- demographics
- recruitment
FEC
Funding
- pharma company? university? research institution
- funding does not automatically mean this is a bad study –> read full paper for level of access
Ethics
- RCTs go through research governance to acquire ethical approval
- placebo - treatment with no active therapeutic effect
- ethical if standard of care is available
- active placebo = no drug but gives you side
effects
- Data Safety Monitoring Board
- 4 pillars of ethics - autonomy, beneficence, non maleficence, justice
- capacity, consent, confidentiality, children
Conclusion
- ideally I would look at the full paper and a systematic review/meta-analysis/Cochrane review
- I would/would not incorporate this study into my clinical practice
Define equivalence trial
to show an intervention is just as effective as control, only done if there is a clear benefit to new intervention e.g. less side effects or cheaper
Define non-inferiority trial
to show an intervention is no worse than the control, requires smaller sample sizes, is cheaper to run, better to do per protocol analysis as will take side effects into account
Define confounder
variable that has a triangular relationship between the exposure and outcome but not on the causative pathway
Define diagnostic purity
participants in trials are all confirmed to have a specific disease but in clinical practice there is always diagnostic uncertainty so the trial sample sizes are often too pure
Define clinical equipoise
assumption that there is no ‘better’ intervention at the time of trial design, shows the trial is necessary to further knowledge
Define efficacy
impact of an intervention under optimal trial conditions = measure of internal validity
Define effectiveness
intended effect in ordinary clinical circumstances = measure of external validity
