STATS Lec 20- Systematic reveiw and meta analysis

Question 1

Literature reviews

Answer 1

• Narrative
  
  • Provides written summary of information- e.g. introduction to a project
  • Usually done for a purpose
  • Usually involves some critical element
  • May be comprehensive and unbiased, often not
• Systematic review
  
  • A research method

Question 2

Why do a systematic review

Answer 2

• Fergusson et al (2005) effect of aprotinin on the outcome of cardiac surgery
• A CUMULATIVE systematic review of 64 RCT- first in 1987
• Meta-analysis of first 12 trials (1987-1992)- Statistically significant effect of treatment
• 52 further trials performed between 1992 and 202- would not have been required if meta-analysis had been done in 1992

Question 3

Classical versus cumulative meta-analysis

Smoking cessation with nicotine patch at 6 months

Answer 3

• 2nd box is a combination of all previous studies put together
• Showing clear benefit with tight confidence intervals

Question 4

Systematic review: A research method

Answer 4

• Process of identifying, interpreting and summarising information from studies in order to answer a structured question
• Doing an experiment with existing data rather than with your own data- Prof Li Wan Po

Question 5

Compare these two

Answer 5

• Narrative review
  
  • No defined method
  • Exploratory, creative
  • May be biased
  • Introduction to a project
• Systematic review
  
  • Rigorous method
  • Transparent and replicable
  • Aims to eliminate bias
  • Research method

Question 6

Meta-analysis

Answer 6

• Statistical- combination of the results from studies to obtain a more precise estimate of treatment effect
• Can be included as part of a systematic review- usually end part

Question 7

Key points- to do a systematic review you need to

Answer 7

• Have a good working knowledge and understanding of the field
• Understand the systematic review methodology
• Be able to follow the research method in a focussed, structured and transparent way
• Undertake a rigorous, systematic, comprehensive and exhaustive search for ALL relavent literature

Question 8

Stages of systematic review

Answer 8

1. Scope and Mapping- refine your research question
2. Plan search, develop protocol
3. Search and document- comprehensive and complete
4. Apply inclusion and exclusion criteria- refine search
5. Quality assessment- further exlude if necessary
6. Data extraction- compliation into summary tables
7. Synthesis- Meta-analysis if appropriate
8. Write-up

Question 9

search strategy: VITAL

Answer 9

• Databases
• Search terms and combinations
• Should be inclusive- identify all potentially relevant studies
• Limitations of search- recognise the effect on conclusions
• Unpublished data- relevance

Question 10

Inclusion/Exclusion criteria

Answer 10

• Include only studies relavent to question
• Consider
  
  • Study design
  • Population intervention
  • Outcomes
  • QUALITY

Question 11

Hierarchy of evidence a quality issue

Answer 11

1. Randomised controlled trial
2. Non-randomised trial
3. Cohort study (prospective)
4. Case-control study (retrospective)
5. Cross-sectional study
6. Surveillance data
7. Case report

• Systematic review and meta-analysis is often considered higher than RCT because it is many RCT summarised at once
• Further down, you move away from robust, reliable data and start to introduce bias

Question 12

Assessing quality: checklist

Answer 12

• Published checklists available
  
  • Modify to suit your question
  • What is a good quality/bad quality in your field
• How to measure ‘quality’
• Subjective measure
  
  • Must be transparent
• Quality of reporting vs. quality of the study

Question 13

DH NSF quality assessment scale (2008)

Question 14

Next stage: summarise results

Answer 14

• Descriptive summary and tables- show off data
• Measure of effect (+CI) for each study
  
  • Concept of effect size
  • Depends on question and data
• Test for heterogeneity
  
  • Can data be combined from different studies

Question 15

Outcome measures: effect size

Answer 15

• Varies depending on research question
  
  • Clinical
    
    • Odds ratio, RR, reduction in BP
    • See slides at the end
  • Economic
    
    • Cost/Benefit
  • Should be appropriate to your research question

Question 16

Further analysis: Meta-analysis

Answer 16

• Calculation of POOLED EFFECT SIZE
• Do not simply average data from a group of trials
• Give more weight to studies which tell us more
• Calculate a weighted average of the treatment effects from each study

Question 17

Can you combine data

Answer 17

• Are the same measurements made
  
  • Reduction in BP
  • Reduction in stroke incidence
  • Decreased mortality
• Can the data be converted to comparable measures
  
  • The number needed to treat NNT
  • Positive effects / negative effects (categorisation)
• Is the data from a homogeneous population
  
  • The same all the way through

Question 18

May not be homogeneous if

Answer 18

• The study designs are different
  
  • You need to have evaluated this
• The sample of patients/subjects varies between trials / studies
  
  • You need to have noted this for all studies
• Different outcome measures are used
  
  • Again, something to record for all studies
• The quality or size of the studies are different
  
  • Did you do a robust quality assessment

Question 19

Homogeneous

Question 20

If

Answer 20

• Your data does look suitable
• Meta-analysis
• Combine results from individual trials to produce summary outcome measure
  
  • Pooled effect size

Question 21

Forrest plot

Question 22

Meta-analysis models: Fixed-effects vs. random-effects

Answer 22

• Fixed-effects
  
  • Assumes that each study is measuring the same treatment effect
    
    • All variation due to that effect
  • Did the intervention provide benefit, on average, in the studies included
• Random-effects
  
  • Assumes that there is variation in treatment effects between trials
    
    • Reasons for variation can be investigated
  • Will the intervention provide benefit, on average

Question 23

Done by computer

Answer 23

• Is the data homogeneous
• Calculate a combined effect size

Question 24

What are the advantages

Answer 24

• Reduction of bias
• Improved precision of estimate
• Better understanding of seemingly heterogeneous results
• Hypothesis generation

Question 25

Draw conclusion

Answer 25

• Address the original question
• Consider the strength of evidence
• Clinical (or other real) significance

Question 26

Outcome measures: some notes to help

Answer 26

• RR
  
  • A proportion of events in the treatment group/proportion events in placebo group
  • e.g. 38/294 divided by 54/232 = 0.55
• Odds ratio (OR)
  
  • Odds in treatment group/odds in control group
  • e.g. 38/256 divided by 54/178 = 0.49

Question 27

Outcome measures

Answer 27

• Absolute risk reduction (ARR)
  
  • Proportion events in placebo group minus the proportion of events in the treatment group described as a %
  • e.g. (54/232 - 38/294) x 100
  • 10.3% in the treatment group
• A number needed to treat (NNT)
  
  • 1/ARR x 100
  • 9.7 i.e. approximately 10 people will have to be treated (for the time equivalent to trial duration) to prevent 1 event
  • Maybe useful in the report as easier to explain than OR etc

Question 28

Dichotomous vs continuous data

Answer 28

• Dichotomous- sick or not sick
• Continuous
  
  • e.g. BP, ChE level
  • Effect on public health measures in MANY ways
• Can dichotomise continuous data
  
  • e.g. ChE >5 or ChE <5
  • Positive, negative or no effect on public health