Research Methods and Analytical Procedures Flashcards
Hierarchy of Evidence (Weakest to strongest)?
Expert opinion
Case Reports
Cohort studies / Cross-sectional Studies
Randomised controlled trials
Systematic Reviews
Whats a narrative review?
While narrative reviews are often written by experts it is likely that his/her expertise, experience and beliefs will influence which studies are included in the review
Narrative reviews do NOT influence policy makers as they are NOT ‘good science’, even if the studies they include are individually very good science.
So falls under Expert opinion, it is cherry picking
What are systematic review?
Systematic reviews minimise the likelihood of such bias as the methods used to identify, select and critically appraise relevant primary research, is done in an open, explicit and systematic way.
Aim of a systematic review?
Summarise the findings of ALL studies that address a very specific research question and meet clear inclusion criteria (e.g. must be RCT, >1 mth). And, if possible, quantify the average effect or finding.
Identify gaps or under-researched areas in the literature (e.g. there maybe a shortage of trials in a particular age group, ethnicity, or sport).
Search term used when doing a systematic review? (example for assessing the impact of exercise referral schemes on physical activity and health outcomes
PICOS
Participants - (Has/risk of disease, old, obese)
Intervention type - (exercise referral)
Control/Compared to - (usual advice, diet)
Outcome - (BMI, BP, physical activity)
Study design - (RCT, cohort, cross-sectional)
PECOS for observational studies - P = population, and E = exposure
Types of systematic reviews?
A qualitative synthesis of the studies: these are quite descriptive. They report the number or proportion of studies that report a statistically significant difference – p<0.05 (systematic review)
A quantitative synthesis of the studies. An ‘average’ intervention effect is calculated from all of the intervention effects from the included studies (systematic review and/with meta-analysis)
Description of a Meta Analysis?
The general aim of a meta-analysis is to more powerfully estimate the ‘true’ intervention effect size as opposed to a less precise effect size derived in a single study.
To produce an ‘average’ intervention effect the units need to be same across all studies, or standardised:
[e.g. Diff of Means divided by Pooled SD].
If there is high heterogeneity (wide range) of effect sizes (e.g. some zero effect, some very positive) then producing a single summary estimate is not appropriate (low I2 value = low heterogeneity = effects were similar across studies). This is a score on the forest plot as well (0% is very low heterogeneity,
Notes about a forest plot for a meta analysis?
Is in chronological order In terms of publication date - done in this order as pilot studies normally show more effects than in real world
Square indicates relative risk of each study. Relative risk shows the percentage increase or decrease the study had
The wings represent the 95%CI, if wings cross the line of ‘no effect’ (relative risk ratio=1) then that study did not find a statistically significant effect.
Relative risk found by (the percentage of those who met the criteria who had the intervention) / the percentage of those who met the criteria who didn’t have the intervention)
Bigger studies will have a greater weighting when determining the ‘average intervention effect’. Instead of each having equal weighting represented by a larger rectangle
The kite/diamond shape at the bottom represents the ‘weighted average’ intervention effect and 95%CI. (the amount 1 represents how much % it increases it by) - and if it crosses the line then it is not significantly different (the size of the diamond is its confidence intervals) - can read of the p value (if lower than 0.05) to check significance as well
What differences could come up if you did a quantitative synthesis/ meta-analysis, compared to a qualitative synthesis?
If looked at studies qualitatively, could end up saying all the studies aren’t significant, but when you look at them qualitatively as a collective they actually make the result significantly
Standard criteria for assessing quality/ risk of bias in forest plots?
e.g. Newcastle-Ottawa risk of bias scale:
0-3 = very high risk
4-6 = high risk
7-9 = low risk
Or can do a Meta regression for high heterogeneity
If you add and minus 1 standard deviation to a normal distribution what percent of the data will you cover?
1 standard deviation = 68%
2 standard deviation = 95%
3 standard deviation = 99%
How to test if you data is normally distributed?
Look to see if curve is bell curved - not finite
Skewness (how the curve leans) should be between -1 and 1
Kurtois = (the peakedness or flatness of a distribution) , Kurtois should be between -1 and 2
Can still go ahead and analyse if values are just outside of boundaries
Is under descriptive statistics in SPSS, you look at the statistic values in the skewness and kurtois
Whats negative and positive skew?
Positive is when most of the results are bellow the middle
Negative skew is when most of the results are past the middle
What’s mesokurtic curve?
Kurtois is normal
What’s a leptokurtic curve?
Very peaked - positive kurtois
Whats a platykurtic curve?
Very flat - negative kurtois
T test definitions?
Test whether there is a difference between 2 means
t-tests are used to test if the difference between the 2 means is likely to be a real difference or one that occurred by chance (sampling variation)
p-value = probability that it occurred by chance
two types of t tests
Independent samples t tests?
Independent groups: Different people in each sample mean
Looks at differences between 2 different groups
Dependent / paired samples t test?
One group the sample people in each sample mean (except twin studies)
Look at differences within the same group in different conditions/ time-points (before/after)
Hypotheses types?
Null hypothesis = states there will be no difference between the groups
The alternative hypothesis = states there will be a difference between groups
Are most t tests 2 directional?
Yes, eg. there will be a significant difference between the lowering of blood pressure from drug a to drug b
Occasionally 1 tailed = Drug A will be better than Drug B, (but if Drug B is actually better than Drug A you cannot report it as that is not what you’ve tested, and this would have important implications on people’s health). If Drug B was a newer, much cheaper drug than Drug A, then the implications would be the same whether Drug B was better or same as Drug A ….. under both circumstances we’d recommend people used Drug B. (non-inferiority study)
What does the P value / significance level have to be below for it to be significant and in a t-test mean that the 2 means are significantly different?
p < 0.05 (expect when applying the Bonferroni correction)
This is when we believe the observed difference hasn’t happened due to random chance (95%)
13
Type 1 error:
You reject the null hypothesis - when the null hypothesis is true (you say there is a difference when there isn’t)
Due to random sampling
Type 2 error:
You fail to reject the null hypothesis when the null hypothesis is false (you say there isn’t a difference when there is)
Due to random sampling
Easy way to remember is boy who cried wolf did a type 1 error then a type 2