SFP Academics appraisal

Question 1

What are the 5 components to critically appraising a research article

Answer 1

1) Background
2) Internal validity
3) External validity
4) Ethics
5) Funding

Question 2

What are the components of background when critically appraising an article

Answer 2

Study Design

Research Question - what question is the study trying to answer?

Importance - why is this an important study? - what is the disease investigated, what is the study adding to the literature body

Core details [PICO]
- Population - sample size, patient group
- Intervention (or exposure in cohort study)
- Control (or compared in cohort study)
- Outcome - primary and secondary out comes

Conclusions - short summary of findings

Journal/funding

Question 3

What does internal validity mean when critically appraising an article

Answer 3

How well is the study designed (i.e. its methodology) to answer the research question? - the extent to which the study measures what it sets out to measure/to what extent is free from bias

Question 4

What are the components of internal validity to assess when critically appraising an article

Answer 4

Study design
Population
Intervention and control
Outcomes
Results

Question 5

What are the components of study design in internal validity

Answer 5

Study design:
Where does it sit on the hierarchy of evidence?
Is this appropriate for the research question?
Mention some pros/cons of using this study design

Question 6

What are the components of population in internal validity

Answer 6

Population - made of recruitment, sample size and randomisation:

Recruitment - is the population selected representative of true population from which the conclusions are drawn (check for selection bias) - look at:
- Inclusion and exclusion criteria - what have they narrowed the population to? (look at commonly excluded groups like cancer and steroid patients)
- Location - primary vs secondary care
- Consecutive vs non-consecutive
- Selection bias - systematic differences between baseline characteristics of the groups; randomisation aiming to distribute confounding factors evenly

Sample size - state that you would check a power calculation in the full text if relevant
- Power = probability of picking up a significant difference if there is one (i.e. the number needed to avoid a type II error which is the mistaken acceptance of the null hypothesis - false negative) - components are: 1) prevision and variance of measurements [sample size affected], 2) magnitude of difference are trying to detect, 3) how certain we want to be avoiding type 1 error (usually 0.8), 4) Type of stats (parametric better than non-parametric at finding differences)

Randomisation - how have the participants been distributed into the 2 groups:
- Baseline characteristics - have a look in results to see if any unequal distribution of confounders - provide relevant examples; want to see there is adjustment if so
- Confounder - in a triangular relationship between exposure and outcome but not on the causal pathway

Question 7

What are the components of intervention and control in internal validity

Answer 7

Details of intervention/control (exposure/no exposure)
- Standardisation - variation in delivery
- Variability - additional care outside the intervention and control e.g. follow up appointments
- Performance bias - systematic differences between groups in care provided other than the intervention of interest

Blinding
- open label (no blinded), single (participant or researcher), double (participant and researcher), triple (participants, researcher and statistician)
- Who was blinded / who could have been (reduction of observer bias?)
- Risk of un-blinding (dose adjustment, noticeable effects e.g. ß blocker)

Control:
- Placebo or gold standard (at therapeutic dose?)

Other things to consider:
- Risk of cross-over: how accessible is the intervention?
- Standardization: variation in delivery

Question 8

What are the components of outcomes in internal validity

Answer 8

Stated a priori?
- A priori - pre-specifying the end-points —> ensures are not bias in choosing outcomes that will give +ve results

What type are the outcomes?
- Clinical/safety/patient reported
- Surrogate endpoints - variable relatively easily measured and predicts a rate distant outcome of the intervention being tested but is not itself a direct measure of either harm or clinical benefit
- Composite outcome - combine 2 or more outcomes into single endpoint

How were they measured?
- Detection bias - systematic differences between groups in how outcomes are determined: Blinding, objective or subjective, clinical variability/inter-observer reliability (Kappa score)

Question 9

What are the components of results in internal validity

Answer 9

Duration
- Is this long enough to detect an effect
- Did it follow original protocol

Attrition (drop out rate)
- Intention to treat analysis - all subjects randomised are included in analyses regardless of whether they completed the study - helps to preserve baseline comparability between 2 groups achieved through randomisation

Statistical validity of conclusions - restate conclusions
- P value - probability of an event happening by chance (wrongful rejection of null hypothesis = type 1 error)
- Confidence interval - provide an estimate or range within which the true answer will lie 95% of time

Robustness of results - sensitivity analysis

Question 10

What are the 3 main bias types to discuss in internal validity and how can they be minimised

Answer 10

Selection bias (systematic differences between baseline characteristics of the groups) - randomisation minimises

Performance bias (systematic differences between groups in the care that is provided other than the intervention of interest) - minimised by blinding of participants and researchers + standardised protocols

Detection bias (systematic differences between groups in how outcomes are determined) - minimised by blinding (of outcome assessment) and objective outcomes

Question 11

What is a type 1 error

Answer 11

False +ve - wrongful rejection of null hypothesis

Question 12

What is type 2 error

Answer 12

False -ve, wrongful acceptance of the null hypothesis

Question 13

What is the power

Answer 13

Probability of picking up significant difference is there is one (1 — type 2 error)

Question 14

What is P value

Answer 14

Probability of event happening by chance (getting a type 1 error)

Question 15

What is a confidence interval

Answer 15

Provides an estimate or range within which the true answer will lie 95% of the time)

Question 16

What is external validity and its components

Answer 16

Can results from this study be applied to other populations?
Its generalisability

Components:
- Disease
- Population
- Intervention (STEP)

Question 17

What are the components of disease in external validity

Answer 17

How relevant is this disease to my practice
- Epidemiology
- Natural history - aetiology/RF, genomics, severity of illness

Question 18

What are the components of population in external validity

Answer 18

Is the sample population representative of my patient cohort?
- Severity of illness
- Socioeconomic
- Exclusion criteria
- Subgroup analysis

Question 19

What are the components of intervention in external validity

Answer 19

Other than the clinical effectiveness what other factors would need to be considered for this intervention to be commissioned? (STEP)
- Safety - serious S/E, number needed to treat vs number needed to harm, MHRA Yellow card scheme
- Tolerability - withdrawal rates
- Efficacy - number needed to treat
- Price and practicability - availability, resource burden

Question 20

What are the pros/cons of RCTs

Answer 20

Advantages:
- Can infer causality
- Adds to the body of work - in systematic reviews/meta analyses

Disadvantages:
- Expensive
- time consuming
- Prone to bias if imperfect or no blinding/randomisation
- Ethical consideration of giving different treatments

Quality of RCTs can be established by using CONSORT checklist

Question 21

What are the pros/cons of a cohort study

Answer 21

Advantages:
- Can look at rare exposures
- Calculate RR
- Look at temporal relationships (he timing between a factor and an outcome which can be used to assign causality to a relationship)
- Aetiology and prognosis

Disadvantages:
- Can take a long time
- Hard if disease has a long latency time
- Expensive to initiate and maintain
- Bias issue if there’s a drop out - large numbers needed

Bradford hill criteria - can be used to determine if causation is present

Question 22

What are the advantages/disadvantages of a case-control study

Answer 22

Advantages:
- Good for studying risk factors
- Good for studying rare disease
- Good for new diseases
- Useful for long time period between exposure and outcome
- Quick and cheap as low participant numbers are needed

Disadvantages:
- Can be difficult to match to controls
- Rely on recall - so subject to recall bias (systematic difference in accuracy of recollections retrieved by study participants)
- Temporal relationships are hard to judge

Question 23

What is an odds ratio

Answer 23

Odds that the disease case was exposed/odds non-diseased was exposed

Question 24

What are the pros/cons of a systematic reveiw/meta analsis

Answer 24

Advantages:
- Stronger power than smaller individual studies

Disadvantages:
- Publication bias
- Language bias - English

Question 25

What are forest plots

Answer 25

Used in meta analysis

Individual studies are black squares
Horizontal line - is point estimate and 95% CI
Size of dot is weight of study in meta analysis
Vertical line indicates no effect of treatment (OR of 1)
Diamond at bottom is pooled results when its 95% CI