L17 - SysR and MeA

Question 1

What is SysR?

Answer 1

Clearly formulated question using systematic and explicit methods to collect and analyse data included in review

Question 2

What is MeA?

Answer 2

Using statistical technique to INTEGRATE results of some studies in SysR into a single quantitative estimate, or a summary effect size

Question 3

What are the purposes of MeA?

Answer 3

1. Increase stats power
2. Increase precision (due to increased n)
3. Deconflict studies, or generate new Hyp
4. Answer questions that are not posed by individual studies (e.g. effects on different race)

Question 4

Distinguish between a Narrative Review and SysR

Answer 4

Narrative Review:

1. Source and search: not specified and can be biased
2. Broad question
3. No specific study selection, may be biased
4. Varies in appraisal
5. Only provide qualitative summary

SysR:

1. Comprehensive sources using search strategy
2. Focused question
3. Uniformly applied I/E critera for studies selection
4. Rigorous critical apprasial
5. May be quantitative using MeA

Question 5

List the steps involved in conducting a SysR (and MeA)

Answer 5

1. Formulate review question and develop a SysR protocol
2. Search the literature and record studies included
3. Assess study quality with ≥2 independent reviewers
4. Abstract relevant data with ≥2 reviewers
5. Analyse data and interpret results. MeA if required
6. Report findings

Question 6

While assessing the quality of selected studies, what can be used?

Answer 6

Scales

• RCT: Jadad scale, Cochrane collaboration tool
• OS: Newcastle-ottawa Scale for CCS & Cohs

Question 7

What are some concepts used to help formulate review question and develop a SysR protocol?

Answer 7

1. PICO approach
2. Define I/E Criteria for study selection
3. Type of study design to be included

Guide: PRISMA-P

Question 8

Describe how would a reviewer search the literature for SysR

Answer 8

Comprehensive search using selected search terms while following reasonable limitations in various databases.

• Limitations like: Language
• Databases: Pubmed, NCBi, BMJ, etc.

Question 9

Considerations when abstracting data for SysR and MeA?

Answer 9

1. ≥2 reviewers

2. Formulate protocol for abstraction by assessing relevant study characterstics and quality

Question 10

What if there are discrepancy between the two reviewers while abstracting data?

Answer 10

1. Discuss till an agreement

2. Refer to 3rd investigator if discrepancy cannot be resolved

Question 11

What does the size of the box represent in a forest plot?

Answer 11

Weight of study. Larger box = more weight, usually due to higher sample size (and smaller CI) hence more contribution to the overall result

Question 12

What does the diamond box in the forest plot represents? Describe its various details.

Can it be statistically significant even if all of its separate studies used is not statistically significant?

Answer 12

Summary effect size from pooled result.

• Width: 95%CI of summary measure
• Mid-point of diamond: Summary effect measure

It can be statsig even if studies are not statsig

Question 13

Purpose of quantitative analyses (i.e. MeA) in SysR

Answer 13

1. Examine heterogeinity

2. Publication bias

Question 14

Distinguish between a forest plot for difference VS forest plot for measures of association

Answer 14

1. Line of non-significance
   - Differences: at 0, line of NO DIFFERENCE
   - MoA: at 1, line of NO ASSOC

Question 15

Is stratified analysis possible using a forest plot?

Answer 15

Yes, different summary effect measures can be obtained from stratifying studies (E.g. by RCT and OS)

Question 16

Describe the three types of heterogeneity among studies used in SysR

Answer 16

1. Clinical heterogeneity, due to different HCI practices
   (E.g. different dose, regimen, duration)
2. Methodological heterogeneity: due to differences in study design AND quality
3. Stats Heterogeneity

Question 17

How can stats heterogeneity (sHg) be measured?

Answer 17

1. Visual inspection of forest plot: the extent to which 95%CI overlaps
2. Stats Tests
   - Cochran’s Q test (Chisq for Hg): p<0.10 is statsig Hg

• I^2 (Isq) stats: Proportion of total variance due to between-study variability
  If 0% = no observed Hg, larger = increasing Hg

Question 18

What are two ways to analyse data in MeA and when to use them?

Answer 18

1. Random-effects model:
   - When there is concern on sHg
   - Can account for the variability between studies
2. Fixed-effects model:
   - When there is no statsig Hg among studies
   - No need to account for variability (since variability very little)

Question 19

Describe Publication Bias. What is its effect on the estimate made using MeA

Answer 19

Bias that arise during MeA due to publication of studies:

• Negative results less likely published
• Published studies usually +ve results
• Estimate in MeA may hence overestimate tx effect

Question 20

What are some ways to assess for publication bias

Answer 20

1. Funnel plot visual inspection
   - asymmetrical plot = bias
2. Stats test: Egger’s test for funnel plot asymmetry
   - If p < 0.05, plot is asymmetric, and there is publication bias

Question 21

What do the diagonal lines of funnel plot represents and what does it mean?

Answer 21

• 95%CI around the summary tx effect
• Meaning: expected distribution of studies should publication bias be absent (thus most points should cluster within diagonal lines)

Question 22

In the funnel plot, where can points that represent large studies usually be found at?

Answer 22

Near the narrow part of the diagonal lines (due to less error)

Question 23

Axes of funnel plot

Answer 23

Usually: Precision (y) against effect measure (x)

e.g. standard error against RiR

Question 24

What are some content you can put into findings of SysR and MeA?

Answer 24

1. Summary of key findings
2. Judge validity of findings and discuss
3. Judge generalisability
4. Possible further resh to be done
5. Possible impacts on clinical practice

Question 25

Guides tha help report SysR and MeA findings

Answer 25

1. PRISMA Statement

2. MOOSEGuidelines (for OS)