Critical appraisal Flashcards
QR
Question
- type of study
- hierarchy of evidence
Relevance
- disease prevalence
- serious/fatal/public health issue
- implications of disease
PICOK
Population
- number and demographics of patients
Intervention
- have they justified it? e.g., previously shown promise, know therapeutic drug doses
Control
- placebo controlled
- compared to gold standard
Outcome
- primary outcome
Key findings
- brief summary
RAMBOS - internal validity
(RAM part)
Recruitment
- consecutive (reduces risk of selection bias) vs non-consecutive
- multicentre vs single site –> from primary, secondary or tertiary care
- are they representative of disease population?
- what are the inclusion/exclusion criteria? Is it too strict
Allocation
- randomisation reduces risk of confounders and selection bias
- blinding reduces risk of performance bias and placebo effect vs open label
- stratification vs minimisation
Maintenance
- attrition bias i.e., drop out rate
- why were they lost? e.g., side effects,
laboursome intervention, lack of motivation
- generally <20% to preserve power and type 2
error
- treatment outside the intervention e.g., number of further interactions with HCP
RAMBOS - internal validity
(BOS part)
Baseline characteristics
- similar to target population
- similar between groups –> shows if randomisation was successful
- accounts for co-morbidities
Outcome measures
- TRADO
- what are they not looking at?
- safety outcomes
Statistics
- does the data justify their conclusions?
- sufficient sample size? check full paper for power calculation
- appropriate statistical tests for outcome measurements?
- Cohen’s d effect size
- intention to treat vs protocol analysis
TRADO
Type e.g., clinical/composite/surrogate or objective/subjective
Relevance - to patients and HCP e.g., looking at side effects/QoL
A priori if a registered RCT –> reduces risk of data dredging
Duration of follow up - is it adequate?
Observation bias - have they standardised measurements and cut offs e.g., severity indexes rather than interviews which can lead to recall and response biase
Bias
Selection - patient not representative of intended population – reduced by randomisation
Performance - patients and HCPs may behave differently when they know whether they are receiving the intervention or not – reduced by blinding
Observation - tendency to see what we want or expect to see – reduced by blinding
Attrition - unequal loss of participants from RCT arms – reduced by using intention to treat analysis
Reporting - findings influenced by direction of results – reduced by undertaking a comprehensive search strategy/Publication bias
Define power
Likelihood of detecting a significant difference when it exists
Therefore avoiding type 2 error
Depends on sample size
Define p value
Probability that the association detected has arisen by chance
Typically 0.05 is considered significant to reject null hypothesis
Depends on sample size
Define CI
Range of values within one can be 95% certain that the true population value lies
For ratios → if the value crosses 1 = not statistically significant
For absolute values → if the value crosses 0 = not statistically significant
Define incidence
Rate of new cases over a defined time period (cohort study)
Define prevalence
Proportion of the population with outcome at a specific point in time (cross-sectional study)
Define absolute risk reduction (ARR)
Risk of event in control (5%) - risk of of event in intervention (2%) = 3%
Define relative risk reduction (RRR)
Absolute risk reduction (3%)/ risk of event in control (5%) = 0.6%
Define number needed to treat (NNT)
1/ARR
Define odds ratio
Ratio of exposed to non-exposed in an outcome of interest
1.6 = 60% more likely to have been exposed if disease is present
Statistical tests
