Lecture 14- Review of the evidence

Question 1

Epidemiological study designs

Answer 1

Question 2

Evidence-based healthcare

Answer 2

• Healthcare service and intervention should be based on best evidence
  
  • Need rigorous research
    
    • Primary e.g. RCT
    • Literature review of studies:
      
      • Narrative reviews
        
        • Implicit assumptions
        • Opaque
        • Methodology
        • Not reproducible biased, subjective
      • Systematic review
        
        • Explicit assumptions
        • Transparent methodology (e.g. search strategy)
        • Reproducible
        • Unbiased and objective
    • Decision analyses
      
      • Harm and benefits
      • Cost-effectiveness

Question 3

systematic rveiw

Answer 3

an overview of primary studies that used explicit and reproducible methods

Question 4

meta-analysis

Answer 4

a quantitative synthesis of the result of two or more primary studies that addressed the same hypothesis in the same way

Question 5

why are systematic such a credible source of evidence

Answer 5

• Clear focussed question
• Explicit statements about:
  
  • types of study
  • types of participants
  • types of interventions
  • types of outcome measures
• Systematic literature search
  
  • Methodology should be published
• Selection of material
• Appraisal
• Synthesis (possibly including a metanalysis

Question 6

purpose of a meta-analysis

Answer 6

To facilitate the synthesis of a large number of study results

• Systematically collate study results
• Reduce problems of interpretation due to variation in sampling
• To quantify effect sizes and their uncertainty as a pooled estimate

Question 7

quality criteria for meta-analysis

Answer 7

Should have a formal protocol (comprehensive search strategy and systematic method of reviewing each study for inclusion

Question 8

Example – binary outcomes (e.g. dead or alive after certain time period)

Answer 8

Question 9

Effect size calculated using Odds ratio

Answer 9

Question 10

Forest plot study numbers 1-7 matches studies from previous example

Answer 10

• Horizontal line= 95% confidence intervals
  
  • If it crosses 1- no significant difference in outcomes (5%)
• Square= effect
• Diamond= metanalyses estimates

Question 11

Interpretation of forest plot

Answer 11

• Individual odds ratios [squares] with their 95% CI [lines] are displayed for each study
• Size of square is in proportion to the weight given to the study
• The [diamond] is the pooled estimate with the centre indicating the pooled odds ratio [dotted line] and the width representing the pooled 95% CI
• The [solid line] is the null hypothesis OR
  
  • 1= no difference
• 6 out of the 7 RCTs had an OR > 1.00 indicating greater odds for survival amongst patients taking aspirin after MI
• Only 1 RCT (the largest) had a statistically significant result, but its OR was less than the other RCTs with an OR > 1.00
• Pooled estimate OR = 1.11 (95% CI: 1.04 to 1.19) leads to the conclusion that aspirin increases the chance of surviving after a MI (p<0.05)

Question 12

Meta-analysis problems

Answer 12

Question 13

(1) Heterogeneity between studies : modelling for variation

Two approaches to calculating the pooled estimate odds ratio and its 95% Cl:

Answer 13

Question 14

Heterogeneity

Answer 14

in meta-analysis refers to the variation in study outcomes between studies

Question 15

fixed effect model

Answer 15

Assumes there is only one true effect that every study is trying to estimate- variation in effects is due random variation

• Studies are weighted for uncertainty
• Smaller confidence interval studies will have greater weight

Question 16

random effect model

Answer 16

True mean effect= average of dashed vertical lines

• Each individual study measures different treatment effect
• Any difference between the true trial specific effect (dashed line) and study result is due to random error

Question 17

forest plot for same meta-analysis using fixed effect and random effects model

Answer 17

Difference: size of boxed= a little more equal in random effects model

• Don’t want to let big study dominant weight
• Want to see effect of smaller studies contributing more equally

Question 18

Example 2- Continuous outcome e.g. BP

Answer 18

Null value = 0 = no change in BP

• Shows significant changes in systolic blood pressure (top of graph)
  
  • Diamond doesn’t cross 0
• Insignificant change in diastolic bp (bottom of graph
  
  • Crosses 0

Question 19

Heterogeneity between studies: Analysing variation

Answer 19

• Random effects modelling can only account for variation but not explain it
• Sub-group analysis can help to explain heterogeneity which may provide further insight into the effect of a treatment or exposure
  
  • Study characteristics (e.g. year of publication, length of follow up, %female participants)
  • Participant profile – where data is analysed by types of participants (e.g. subgroups of males, females, adults, children)

Question 20

Variable quality of the studies: the issues

Answer 20

Question 21

Variable quality of the studies:the approaches

Answer 21

The approaches: 2 tend to be used

• Define a basic quality standard and only include studies satisfying this criteria e.g. Cochrane reviews used to include only RCTs
  
  • Score each study for its quality and then
    
    • incorporate the quality score into the weighting allocated to each study during the modelling, so that higher quality studies have a greater influence on the pooled estimate
    • use sub-group analyses to explore differences, e.g. high quality studies vs. low quality studies

Question 22

Assessing the quality of the studies

Answer 22

For RCTs, many scales available – e.g. CHEERs

Question 23

Publication bias and selection of studies

Answer 23

• Studies with statistically significant or ‘favourable’ results are more likely to be published than those studies with non- statistically significant or ‘unfavourable’ results
  
  • this applies particularly to smaller studies
• Any systematic review or meta-analysis can be flawed by such bias
  
  • publication bias leads to a biased selection of studies towards demonstration of effect

Question 24

methods of identification

Answer 24

• Check meta-analysis protocol for method of identification of studies
  
  • should include searching and identification of unpublished studies
• Plot results of identified studies against a measure of their size (e.g. inverse of standard error), i.e. a Funnel Plot
• Use a statistical test for publication bias – they tend to be weak statistical tests

Question 25

publication bias in selection of studies- funnel plots

Answer 25

• Larger studies tend to have less standard error (will be found higher up on the X-axis)
• Plots should look symmetrical
• Look at evidence of bias plot
  
  • Gap in studies in left lower quad
    
    • Shows that smaller studies with odds ratio below 1 haven’t been published

Question 26

forest plot example 3

Answer 26

Forest Plots- The clinical value of thrombolysis remains uncertain- 1987

• down the far left= trial
• shows odds ratio- depending on how far over the plot is on the control/ treatment side of the plot
• plots have confidence intervals (horizontal lines)
  
  • can make cumulative forest plots where data from different studies is synthesised- should decrease confidence limits
• if the confidence limits pass over 1= not significant
• if the confidence limits do not spread past 1, then the results are significant
• meta-analyses will have narrower confidence interval (diamond)
  
  • the wider the diamond the wider the confidence intervals

Can be use to look at trial on a specific topic over a long period of time e.g. thrombolysis

Ethical- e.g. is it ethical to give control, ten years after it has been proven to be the favourable treatment

Question 27

Categorical variable

Answer 27

Categorical variables contain a finite number of categories or distinct groups. Categorical data might not have a logical order. For example, categorical predictors include gender, material type, and payment method.

e.g. dead or alive

Question 28

Continuous variable

Answer 28

Continuous variables are numeric variables that have an infinite number of values between any two values. A continuous variable can be numeric or date/time. For example, the length of a part or the date and time a payment is received.

e.g. blood pressure, height etc