Secondary Research Flashcards
1
Q
What is secondary research?
A
Research conducted using data from other studies
2
Q
What is recruited instead of patients in secondary studies?
A
Eligible studies
3
Q
Types of secondary research
A
Narrative reviews
Systematic Reviews
4
Q
Who carries out narrative reviews?
A
Experts in field of study
5
Q
Structure of narrative reviews
A
Experts own opinion supported by selected research evidence
Broad, no specific clinical question
Subjective
6
Q
Risk of narrative reviews
A
Prone to bias
Literature not searched methodically
Cannot be replicated due to above
7
Q
Advantages of narrative reviews
A
Good introduction to topic
Stimulate interest and controversies
Collate existing data for new hypothesis
8
Q
Question in narrative reviews
A
Broad in scope
General
9
Q
Search methods in narrative reviews
A
Not usually specified
Comes from experts familiarity with literature
10
Q
Appraisal of individual studies in narrative reviews
A
Variable
Usually based on experts opinion
11
Q
Synthesis of results in narrative reviews
A
Usually only qualitative summary of studies
12
Q
Resources in narrative studies
A
Less time consuming but requires expert in field
13
Q
What are systematic reviews?
A
Follow rigorous steps in identifying relevant literature, appraising individual studies and analysing suitable data to synthesise a conclusion
14
Q
Characteristics of systematic reviews
A
Focused narrow question
Comprehensive and specified data collection
Uniform criteria for study collection
Quantitative synthesis of data (optional)
15
Q
What is meta-analysis?
A
Quantitative synthesis of individual study data in systematic reviews
16
Q
What is GIGO?
A
Garbage in, garbage out
17
Q
What does GIGO refer to?
A
If primary studies have poor quality, then even if they use sound methodology and statistical procedures, outcome will be meaningless
18
Q
How is quality of individual trials assessed
A
Nature of patient sample Outcome studied Length of follow-up Comparability of treatment Methodological factors
19
Q
What methodological factors are studied to assess quality of individual trials?
A
Adequacy of sample randomisation
Adequate concealment and control of intervention
Analysis with ITT
Objective, blinded outcome assessment
20
Q
What should inclusion criteria for studies in systematic reviews consider?
A
Types of study designs Types of subjects Types of publications Language restrictions Types of interventions Time frame for included studies
21
Q
What characteristics must a literature review have?
A
Use of multiple databases
Cross checking of reference list of each study retrieved by direct search
Hand searching for materials unidentified online
Approaching experts to common on any missing studies
Identifying grey literature
22
Q
What is grey literature?
A
Unpublished literature e.g. conference abstracts, presentations, posters
23
Q
What is meta-analysis most often used for?
A
Assess clinical effectiveness of healthcare interventions
24
Q
What data does meta-analysis use
A
Summary data or individual patient data
25
Why is summary data more often used?
Inability of investigators to supply patient data
| Increased costs & time
26
What does quality of a meta-analysis or systematic review depend on?
Comprehensive literature search
Clearly defined eligibility criteria
Clearly defined & strictly adhered protocol
Predetermined criteria related to quality of trials
Thorough sensitivity analysis
Discussion and analysis of statistical and clinical heterogeneity
27
Why is thorough sensitivity analysis needed in meta-analysis and systematic reviews?
As assessing quality of study can be difficult since information reported is inadequate for this purpose
28
Steps for conducting meta-analysis
Literature search
Establish criteria for inclusion and exclusion
Record data from individual studies
Statistical analysis of data
29
What do meta-analysis use for combining individual trial data?
Weight average of results
30
What is weighting?
Significance attached to each study based on sample size, precision, external validity and methodological quality
31
Is weighting dependent on outcome of a study?
No
32
Modes of analysis in meta-analysis
Fixed effects model
| Random effects model
33
What does fixed effects model assume?
All studies share same common treatment effect
34
What does random effects model assume?
Studies do not share common treatment efefct
35
What happens in fixed effect analysis?
Inference restricted to included set of studies
Assumes that only random error within studies can explain observed differences
Ignores between-study variations
36
When can fixed effect analysis be applied?
If heterogeneity can be safely excluded by testing for it
37
What happens in random effects analysis?
Assumes that each study shows a different effect which are normally distributed around true mean
This gives proportionally greater weight to smaller studies
38
Disadvantages of random effects analysis
Susceptible to publication bias
| Results in wider less precise confidence intervals
39
What is used in fixed effect analysis?
Mantel-Haenszel and Peto ratios
40
When is Mantel-Haenszel ratio useful?
When wide differences exist between studies in ratios of the size of two groups
41
What type of study designs is Mantel-Haenszel helpful for?
Cohort/case control
42
What is Peto ratio used for?
RCTs
43
Why is Peto ratio only used in RCTs?
Can produce bias results in unequal groups
44
When do both fixed and random effects analysis have similar confidence intervals?
In the absence of heterogeneity
45
How is heterogeneity calculated?
Q statistic
46
Advantage of random effects model
Wider confidence interval which may be more representative
47
Give examples of clinical heterogeneity
Diverse interventions
Differences in selection of patients
Severity of disease
Dose or duration of treatment
48
Is clinical heterogeneity measurable?
No
49
What is methodical heterogeneity?
Heterogeneity resulting from differential use of study methodology
50
Name the types of heterogeneity
Clinical
Methodological
Statistical
51
What is statistical heterogeneity?
Studies may report same outcome but results that are not consistent with each other
52
What is a homogenous sample?
Set of studies which have comparable outcomes without much variation
53
What is a heterogenous sample?
Studies with significant variation amongst them
54
What can be used to test for statistical heterogeneity?
```
Forest Plot
L'Abbe plot
Galbraith plot
Chi Square
I2 statistic
Cochrans Q
```
55
What is a L'Abbe plot?
Modified scatter plot where CER is plotted against EER from individual trials
56
How will data appear in L'Abbe plot?
Trials that report experimental treatment to be superior to control will be in upper left of plot
If experimental is no better than control, point will fall on line of equality
If control is better than experimental point will be in lower right of plot
57
Axis of a Galbraith plot?
Horizontal: 1/standard error of study effect estimate
Vertical: study effect estimate divided by its standard error (log odds ratio/SE)
58
What is 1/standard error equal to?
Precision
59
What is the standard normal deviate?
Study effect estimate divided by its standard error
60
What can statistical procedures test in meta-analysis re individual studies?
Whether results of study reflect single underlying effect (homogenous) or distribution effect (heterogenous)
61
Limitation of studying homogeneity/heterogeneity of single studies?
Statistical tests lack power to detect heterogeneity in most meta-analyses
62
What can Chi square test in meta-analysis?
Test of significance for heterogeneity - does not measure it
63
How can Chi square test studies in meta analysis for heterogeneity?
Q test
| I2 test
64
What is the rule re Chi square when used to test heterogeneity in meta-analyses?
Its value on average is equal to its degree of freedom, which is n-1 (number of studies -1)
65
What would Chi Square of 7 for a meta-analysis of 8 studies mean?
N-1 = 7
Chi square = 7
Thus, chi square of 7 or less means no significant heterogeneity
66
Disadvantages of chi square use for meta-analysis
Can only tell us heterogeneity
| Does not tell us anything about homogeneity
67
Why can Chi square not tell us anything about homogeneity?
Low power
68
How can one quantify heterogeneity?
Cochrans Q
69
How does one calculate Cochrans Q?
Weighted sum of squared difference between individual study effects and pooled effect across studies
70
What does I2 statistic do?
Describes percentage of variation across studies that are due to heterogeneity rather than chance
71
What does high P mean in chi square tests for meta-analyses?
High P suggests that heterogeneity is insignificant and that one can perform meta-analyses
72
If there is heterogeneity in meta-analysis, what can one do?
Study reasons behind it
| Random effects analysis
73
What must one consider when studying the reasons behind heterogeneity in meta-analysis?
Clinical differences
Methodological issues e.g. problems with randomisation
Early termination of trials
Use of absolute rather than relative measures of risk etc
Publication bias
74
What procedures can give suggestions of the causes behind heterogeneity?
Meta-regression analysis
| Subgroup analysis
75
What is publication bias?
Tendency for authors to submit or editors to publish only those studies that yield statistically significant results
76
What do methods that diagnose presence of publication bias work?
Based on the assumption that small studies are more susceptible to publication bias and may therefore show larger treatment effects
77
When are methods for publication bias diagnosis not feasible?
When all available studies are equally large i.e. have similar precision
If heterogenous studies
78
What happens if methods to diagnose publication bias are used in heterogenous studies?
May lead to false positives for publication bias
79
Methods used to diagnose publication bias
```
Funnel plots
Failsafe N
Linear regression
Correlation method
Cumulative meta-analysis
Trim-and-fill procedure
```
80
Most common method used to detect publication bias?
Funnel plot
81
What does a funnel plot show?
Relation between effect size and precision of individual studies in a meta-analysis
82
Reasoning behind funnel plots?
Small studies are more likely to remain unpublished if their results are not significant and larger studies get published regardless of producing asymmetrical funnel plots.
83
Axis of funnel plots
X: measure of effect size
Y: measure of precision
84
Measures of precision
Sample size
| Inverse of standard error (1/SE)
85
What can be used for measures of effect size?
OR
Log OR
Cohens d
86
Reasoning behind Failsafe N?
Meta-analyses can be wrong due to non-significant studies that are not published.
Inclusion of these studies could nullify the statistical significance of an observed mean effect.
87
Who developed Failsafe N?
Rosenthal and Cooper
88
What does Failsafe N do?
Calculates the number of zero-effect studies (no effect) that would be required to nullify the mean effect seen in a meta-analysis
89
What can be used to quantify publication bias?
```
Linear regression method
Correlation method (Tau)
```
90
What can cumulative meta-analysis be used for?
Assess potential impact of publication bias in tilting or nullifying an effect
91
How are cumulative meta-analysis carried out?
Studies sorted from largest to smallest
Meta-analysis is run with one study, then repeated with 2nd etc
Forest plot/blobbogram plotted; first row will show effect based on one study, 2nd row shows cumulative effect based on 2 studies etc.
92
Results of cumulative meta-analysis
If the point estimate stabilizes based on the initially added larger studies, then there is no evidence that the smaller studies are producing a biased overall effect.
If point estimate shifts when smaller studies are added, may be bias - and one can also see direction of bias
93
Who developed the trim-and-fill procedure?
Duval and Tweedie (2000)
94
What does trim-and-fill procedure do?
Assess whether effect would change if bias were removed
95
What happens in trim-and-fill procedure?
Type of sensitivity analysis where missing studies are imputed and added to the analysis, and then effect size is recomputed.
96
What is location bias?
Studies not located due to citation habits, databases used, keywords used or multiple replication of same data
97
What is inclusion bias?
Reviewers tendency to include studies they agree with
98
What is sensitivity analysis?
Analysis of the extent to which an outcome derived from research will change when hypothetical data are introduced
99
What does sensitivity analysis mainly apply to?
Economic analysis
| Meta-analysis
100
What are the nodal points in meta-analysis where hypothetical data could be introduced?
```
Significant heterogeneity (or absence) - impact of outcome?
Factors determining methodological quality
Publication bias
```
101
Name a sensitivity analysis method where publication bias can be examined
Failsafe N
102
What is the other name for a forest plot?
Blobbogram
103
What do forest plots present?
Effect (point estimate) from each individual study as a blob/square with horizontal line across the square
104
What is the square in forest blot?
Measured effect
105
What is the horizontal line in forest plot?
9% CI - indicates precision
106
What does size of square represent in forest plot?
Amount of information in that study - usually the sample size
107
What does length of horizontal line in forest plot represent?
Degree of uncertainty of estimated treatment effect for study
108
What is the mid vertical line in forest plots?
Represents null effect for difference in outcomes
109
What does it mean when a square lies on one side of the vertical line on a forest plot?
Outcome of that study was favoured based on the side of the line the square it is
110
What will be shown on a forest plot if an individual study result is statistically significant?
Horizontal lines representing uncertainty will lie entirely on the same size as the square and will not cross the vertical line
111
What does the diamond mean in forest plots?
Final synthesis of individual studies
112
What does the position of the diamond in forest plots depend on?
Whether the intervention rates favourably or not at the end of the meta-analysis
113
Things to note in a forest plot
Combined/pooled effect size
CI of individual point estimates and combined estimates
Weighting assigned to studies
Heterogeneity of results
114
What is the CI of individual point estimates on a forest plot?
Line width
115
What is the CI of the combined estimate?
Diamond width
116
What does the size of the squares refer to?
Weighting assigned to study
117
What would absence of heterogeneity look like on a forest plot?
Vertical linearity rectangles
118
Why are relative risk and odds ratio more advantageous than absolute risk for calculating effect size in meta-analysis?
Depend less on baseline risk
119
When are OR and RR appropriate to use as effect size measure?
When both variables are binary
120
What is conventionally used to show effect size when reporting treatment effects?
Odds ratio
121
What can occur if one assumes OR and RR are the same?
One may overestimate the effect of treatment - but this overestimation will be small if experimental and control event rates are <20%
122
At what point will OR be significantly greater than RR?
For frequent events i.e. >20%
123
Why is relative risk used in Cochrane reviews?
For frequent events, OR will be greater than RR, leading to overestimation of risk
124
What are the two effect size measures for continuous variables?
Simple difference between the mean values (DM)
| Standardized difference between the mean values (SMD)
125
What is DM?
Keeps original units and is the mean value of one group minus the mean value of the other group
126
How does one carry out SMD?
Dividing DM by pooled standard deviation of the two groups using a formula
127
Formula for SMD
SMD = (
128
What is SMD also known as?
Cohens D
| Hedges' g
129
What does SMD of 0 mean
Treatment and placebo have same effects
130
Advantages of Hedges' g?
Avoids a bias seen in Cohens d when there are small number of subjects
131
Statistical procedures for fixed effects
Peto
| Mantel-Haenszel
132
Statistical procedures for random effects
DerSimonian-Laired
133
Measured effects for fixed effects model
OR
Rate Ratios
Risk Ratios
134
Measured effects of fixed effects model
Approximates OR
135
Measured effects for random effects model
Ratios and rate differences
136
What is a Quorum statement?
Consensus statement on standard format for meta-analysis
137
What is a consort statement?
Consensus statement on standard format for RCTs
138
What is meta-regression analysis?
Technique of regression wherein regression model is applied to meta-analysis to analyse which characteristics of the studies actually contributed to overall effect size
