DATA ANALYSIS 3 Flashcards
The limitations of drawing conclusions based on p-values and statistical significance
t makes no assessment whatsoever regarding the meaning or
importance of the difference, or whether it is of biological or clinical significance.
What is the purpose statistical significance?
1) It assesses whether there is sufficient evidence to reasonably conclude whether the observations are due to genuine differences between the groups and not simply due to random variation.
2) The results are made on the assumption that the methods, techniques, etc. used to generate the data are appropriately designed and conducted. If the design of the experiments is flawed, any conclusions are going to be flawed.
3) The conventional alpha levels , e.g. 0.05, 0.01 are completely arbitrary. α = 0.05 was originally chosen
on the basis that 1/20 seemed like a reasonably infrequent occurrence. However, whether 1/20 is sufficiently infrequent to be judged as valid evidence to conclude something depends on the consequences of making false positive and false
negative errors.
Alternatives to p-values
confidence intervals
What are confidence intervals
A confidence interval of X% describes the size of the range of values that
would be required for the true mean of the difference to be contained within it X% of
the time. So, if a 95% confidence interval for a difference between groups was 1 to 5,
it would mean that if you repeated this procedure infinite times in different studies, you
would expect the true mean to be contained within this interval 95% of the time.
Why are confidence intervals preferred to p-values
They present more information; they not only report whether the difference was statistically significant or not relative to the chosen alpha level, but also enable an evaluation of whether the difference or effect is biologically/clinical significant (by
analysing whether a chosen threshold for determining biological/clinical significance is
contained within the confidence interval).
What happens to p value as effect size increases in parametric tests (t-tests)?
P value decreases as effect size increases for t-tests
How is the p-value affected by ranking in non-parametric data?
The smallest p-value
possible will be obtained in a situation where all of the largest values are contained in one
group and all of the smallest values are contained in the other group.
How does a Shapiro-Wilk test p-value indicate whether data is parametric or non-parametric?
Shapiro-Wilk test p-value < 0.05 = non-parametric
Shapiro-Wilk test p-value > 0.05 = parametric
What should you do if studies show statistical significance with no control group?
Do further analysis to evaluate the statistical significance of differences. If data does not adhere to a particular pattern do the Bonferroni post test
What should you do if studies show statistical significance with a control group?
Dunnett’s test as it has more statistical power (less likely to result in type II error)
What ANOVA should you use for paired and parametric data?
What ANOVA should you use for unpaired and parametric data?
repeated ANOVA
One Way ANOVA
What ANOVA should you use for unpaired and non-parametric data?
What ANOVA should you use for paired and non-parametric data?
Kruskal-Wallis
Friedman Test
If data is paired and categorical?
If data is unpaired and categorial?
If data is unpaired, categorial and includes a 2x2 contingency table?
If data is unpaired, categorial, includes a 2x2 contingency table and less than 10 samples in a cell?
McNemar’s test
Chi-squared Test
Chi-squared test
Fisher’s exact test
