SFP Academics appraisal Flashcards
What are the 5 components to critically appraising a research article
1) Background
2) Internal validity
3) External validity
4) Ethics
5) Funding
What are the components of background when critically appraising an article
Study Design
Research Question - what question is the study trying to answer?
Importance - why is this an important study? - what is the disease investigated, what is the study adding to the literature body
Core details [PICO]
- Population - sample size, patient group
- Intervention (or exposure in cohort study)
- Control (or compared in cohort study)
- Outcome - primary and secondary out comes
Conclusions - short summary of findings
Journal/funding
What does internal validity mean when critically appraising an article
How well is the study designed (i.e. its methodology) to answer the research question? - the extent to which the study measures what it sets out to measure/to what extent is free from bias
What are the components of internal validity to assess when critically appraising an article
Study design
Population
Intervention and control
Outcomes
Results
What are the components of study design in internal validity
Study design:
Where does it sit on the hierarchy of evidence?
Is this appropriate for the research question?
Mention some pros/cons of using this study design
What are the components of population in internal validity
Population - made of recruitment, sample size and randomisation:
Recruitment - is the population selected representative of true population from which the conclusions are drawn (check for selection bias) - look at:
- Inclusion and exclusion criteria - what have they narrowed the population to? (look at commonly excluded groups like cancer and steroid patients)
- Location - primary vs secondary care
- Consecutive vs non-consecutive
- Selection bias - systematic differences between baseline characteristics of the groups; randomisation aiming to distribute confounding factors evenly
Sample size - state that you would check a power calculation in the full text if relevant
- Power = probability of picking up a significant difference if there is one (i.e. the number needed to avoid a type II error which is the mistaken acceptance of the null hypothesis - false negative) - components are: 1) prevision and variance of measurements [sample size affected], 2) magnitude of difference are trying to detect, 3) how certain we want to be avoiding type 1 error (usually 0.8), 4) Type of stats (parametric better than non-parametric at finding differences)
Randomisation - how have the participants been distributed into the 2 groups:
- Baseline characteristics - have a look in results to see if any unequal distribution of confounders - provide relevant examples; want to see there is adjustment if so
- Confounder - in a triangular relationship between exposure and outcome but not on the causal pathway
What are the components of intervention and control in internal validity
Details of intervention/control (exposure/no exposure)
- Standardisation - variation in delivery
- Variability - additional care outside the intervention and control e.g. follow up appointments
- Performance bias - systematic differences between groups in care provided other than the intervention of interest
Blinding
- open label (no blinded), single (participant or researcher), double (participant and researcher), triple (participants, researcher and statistician)
- Who was blinded / who could have been (reduction of observer bias?)
- Risk of un-blinding (dose adjustment, noticeable effects e.g. ß blocker)
Control:
- Placebo or gold standard (at therapeutic dose?)
Other things to consider:
- Risk of cross-over: how accessible is the intervention?
- Standardization: variation in delivery
What are the components of outcomes in internal validity
Stated a priori?
- A priori - pre-specifying the end-points —> ensures are not bias in choosing outcomes that will give +ve results
What type are the outcomes?
- Clinical/safety/patient reported
- Surrogate endpoints - variable relatively easily measured and predicts a rate distant outcome of the intervention being tested but is not itself a direct measure of either harm or clinical benefit
- Composite outcome - combine 2 or more outcomes into single endpoint
How were they measured?
- Detection bias - systematic differences between groups in how outcomes are determined: Blinding, objective or subjective, clinical variability/inter-observer reliability (Kappa score)
What are the components of results in internal validity
Duration
- Is this long enough to detect an effect
- Did it follow original protocol
Attrition (drop out rate)
- Intention to treat analysis - all subjects randomised are included in analyses regardless of whether they completed the study - helps to preserve baseline comparability between 2 groups achieved through randomisation
Statistical validity of conclusions - restate conclusions
- P value - probability of an event happening by chance (wrongful rejection of null hypothesis = type 1 error)
- Confidence interval - provide an estimate or range within which the true answer will lie 95% of time
Robustness of results - sensitivity analysis
What are the 3 main bias types to discuss in internal validity and how can they be minimised
Selection bias (systematic differences between baseline characteristics of the groups) - randomisation minimises
Performance bias (systematic differences between groups in the care that is provided other than the intervention of interest) - minimised by blinding of participants and researchers + standardised protocols
Detection bias (systematic differences between groups in how outcomes are determined) - minimised by blinding (of outcome assessment) and objective outcomes
What is a type 1 error
False +ve - wrongful rejection of null hypothesis
What is type 2 error
False -ve, wrongful acceptance of the null hypothesis
What is the power
Probability of picking up significant difference is there is one (1 — type 2 error)
What is P value
Probability of event happening by chance (getting a type 1 error)
What is a confidence interval
Provides an estimate or range within which the true answer will lie 95% of the time)
What is external validity and its components
Can results from this study be applied to other populations?
Its generalisability
Components:
- Disease
- Population
- Intervention (STEP)
What are the components of disease in external validity
How relevant is this disease to my practice
- Epidemiology
- Natural history - aetiology/RF, genomics, severity of illness
What are the components of population in external validity
Is the sample population representative of my patient cohort?
- Severity of illness
- Socioeconomic
- Exclusion criteria
- Subgroup analysis
What are the components of intervention in external validity
Other than the clinical effectiveness what other factors would need to be considered for this intervention to be commissioned? (STEP)
- Safety - serious S/E, number needed to treat vs number needed to harm, MHRA Yellow card scheme
- Tolerability - withdrawal rates
- Efficacy - number needed to treat
- Price and practicability - availability, resource burden
What are the pros/cons of RCTs
Advantages:
- Can infer causality
- Adds to the body of work - in systematic reviews/meta analyses
Disadvantages:
- Expensive
- time consuming
- Prone to bias if imperfect or no blinding/randomisation
- Ethical consideration of giving different treatments
Quality of RCTs can be established by using CONSORT checklist
What are the pros/cons of a cohort study
Advantages:
- Can look at rare exposures
- Calculate RR
- Look at temporal relationships (he timing between a factor and an outcome which can be used to assign causality to a relationship)
- Aetiology and prognosis
Disadvantages:
- Can take a long time
- Hard if disease has a long latency time
- Expensive to initiate and maintain
- Bias issue if there’s a drop out - large numbers needed
Bradford hill criteria - can be used to determine if causation is present
What are the advantages/disadvantages of a case-control study
Advantages:
- Good for studying risk factors
- Good for studying rare disease
- Good for new diseases
- Useful for long time period between exposure and outcome
- Quick and cheap as low participant numbers are needed
Disadvantages:
- Can be difficult to match to controls
- Rely on recall - so subject to recall bias (systematic difference in accuracy of recollections retrieved by study participants)
- Temporal relationships are hard to judge
What is an odds ratio
Odds that the disease case was exposed/odds non-diseased was exposed
What are the pros/cons of a systematic reveiw/meta analsis
Advantages:
- Stronger power than smaller individual studies
Disadvantages:
- Publication bias
- Language bias - English
What are forest plots
Used in meta analysis
Individual studies are black squares
Horizontal line - is point estimate and 95% CI
Size of dot is weight of study in meta analysis
Vertical line indicates no effect of treatment (OR of 1)
Diamond at bottom is pooled results when its 95% CI
