Reviews Of Evidence Pop Sci

Question 1

Describe the categories of epidemiological study designs?

Answer 1

Observational studies:

• descriptive
  1. Ecological studies
  2. Cross-sectional surveys
• analytical
  1. Case-control studies
  2. Cohort studies

Experimental:

• analytical
  1. Trials

Question 2

Hierarchy of scientific evidence

Answer 2

Bottom up:

Case reports, opinion papers and letters

animal trials and in vitro studies

Cross sectional studies

Case-control studies

Cohort studies

Randomised controlled trials

Meta-analyses and systematic reviews

~gets stronger as you move up

Question 3

What can evidence- based healthcare be based on?

Answer 3

Primary research studies e.g. RCT

Literature reviews of studies

• narrative reviews: implicit assumptions, opaque methodology, not reproducible -> biased, subjective
• systemic reviews: explicit assumptions, transparent methodology, reproducible -> unbiased, objective

Decision analyses (harm and benefits and cost-effectiveness)

Question 4

Describe what systematic reviews involve?

Answer 4

Clearly focused question

Explicit statements about:
Types of study, types of participants, types of interventions, types of outcome measures

Systematic literature search

Selection of the materials

Appraisal

Synthesis (possibly including a meta-analysis)

Extremely credible source of evidence (explicit, transparent and reproducible)

Question 5

Define systematic review

Answer 5

An overview of primary studies that used explicit and reproducible methods

Question 6

Define a meta-analysis

Answer 6

A quantitative synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way

Question 7

Purpose of meta-analysis

Answer 7

Facilitate synthesis of large number of study results

Systematically collate study results

Reduce problems of interpretation due to variations in sampling

Quantify effect sizes and their uncertainty as a pooled estimate

Question 8

What is the quality criteria for meta-analysis?

Answer 8

Should have a formal protocol specifying:

Compilation of complete set of studies

Identification of common variable or category definition

Standardised date extraction

Analysis allowing for sources of variation

Question 9

How to calculate odds ratio?

Answer 9

Slide 13

Question 10

How do you calculate a pooled estimate odds ratio for all studies? What are the two approaches to calculate this and it’s 95% CI?

Answer 10

Odds ratio and their 95% confidence intervals are calculated for all studies in meta-analysis

These combined -> pooled estimate odds ratio using a statistical computer programme

Studies are weighted according to their size and the uncertainty of their odds ratio (narrower confidence interval -> greater weight to result)

Either:
- fixed effect model - assumes studies are estimating exactly the same true effect size
OR
- random effects model - assumes that the studies are estimating similar, but not the same, true effect size

Question 11

Problems with meta-analysis

Answer 11

Heterogeneity between studies:
- modelling for variation 
Fixed effect model vs random effects model 
- analysing the variation 
Sub- group studies 

Variable quality of studies

Publication bias in selection of studies

Question 12

How do the odds ratio, 95% confidence interval and sighting of studies vary for fixed effect vs random effects?

Answer 12

Odds ratio/ point estimate:
Often similar but not always in both

95% CI:
Often wider in random effects

Weighting of studies: more equal in random effects model e.g. greater weighting towards small studies

Much debate over which is superior

Question 13

How can you explain variation between studies?

Answer 13

Random effects can only, account for variation not explain it

Sub- group analysis can help to explain heterogeneity which may provide further insight into effect of treatment or exposure

• study characteristics e.g. yr of publication, length of follow up
• participant profile e.g. where data is analysed by types of participants

Question 14

What issues are there with variable quality of studies?

Answer 14

Variable quality can be due to: (easier to harder to assess)

• poor study design
• poor design protocol
• poor protocol implantation

Some studies more prone to bias and confounding: (less susceptible to more)

• randomised controlled trails
• non-randomised controlled trials
• cohort studies
• case-control studies

Question 15

What are the two approaches to measure variable quality of the studies?

Answer 15

Define a basic quality standard and only include studies satisfying this criteria e.g. Cochrane reviews used to include only RCTs

Score each study for its quality and then

• incorporate quality score into weighting allocated to each study during modelling (higher quality = greater influence on pooled estimate)
• use sub-group analyses to explore differences e.g. high quality studies vs low

Question 16

When assessing quality of studies for RCTs what are the main components, who assess quality, should assessors be blinded to results?

Answer 16

Main components:

• allocation methods
• blinding and outcome assessment
• patient attrition
• appropriate statistical analysis

Who asses quality:

• > 1 assessor
• handling disagreements

Should assessors be blinded to results
- sometimes difficult

Question 17

Methods of identifying publication bias in selection of studies

Answer 17

• check meta-analysis protocol for method of identification of studies. It should include searching and identification of unpublished studies
• plot results of identified studies against a measure of their size e.g. inverse of standard error (funnel plot)
• use a statistical test for publication bias (tend to be weak statistical tests)

Question 18

How to produce and interpret a funnel plot

Answer 18

Plot some measure of study size e.g. standard error of estimate against measure of effect e.g. odds ratio

If no publication bias, the plot will be a balanced/ symmetrical funnel

Smaller studies can be expected to vary further from the central effect size

Publication bias likely if there are few small studies with results indicating small or negative measure of effect

Question 19

What are sources for evidence for systematic reviews?

Answer 19

• NHS centre for reviews and dissemination (a national centre for review, management and dissemination of research findings in Uk. Produces the database of abstracts of reviews of effectiveness, produces guidelines on the conduct of systematic reviews)
• NIHR health technology assessment programme (national programme of research, aims to produce high quality research info on costs, effectiveness and broader impact of health technologies, commissions reviews and modelling for national institute for clinical excellence, includes primary research and systematic reviews)

Question 20

What are examples of critical appraisal of systematic reviews?

Answer 20

• JAMA user’s guide to medical literature
• NHS critical appraisal skills programme
• preferred reporting items for systematic reviews and meta-analysis
• meta- analysis of observational studies in epidemiology

Question 21

Examples of evidence based medicine

Answer 21

• oxford centre for EBM
• students 4 best evidence
• EB decision making (6 step approach by Porzsolt F eat al)
• Uni of Leicester library