Testing for differences Flashcards
What are some parametric tests?
Paired t-test
two-sample test
one-sample test
What are some non-parametric tests?
Mann whitney U
wicoxon
one-sample sign test
What is the quantifying difference?
Cohen’s d
what is the purpose of parametric T-tests?
testing for differences
hypotheses testing
can be one or two tailed tests
What are the assumptions of the T-tests?
continuous data
follows a normal distribution
variances are constant (standard deviations)
What happens in the assumptions of a T-test dont hold?
the statistical test is in doubt and the p-value may be wrong
Robust test can withstand some deviation
try to transform the data
Use a non-parametric test instead
When would you use the one sample t-test?
used to determine if mean of a sample is difference from an expected value
When would you use a paired t-test?
test for differences between two sets of paired observations
When would you used the two-sample t-test?
test whether the means of two independent sets of measurements are different
What are the assumptions of the paired t-test?
check that appropriate t-test assumptions are met.
What is the null hypothesis of the paired t-test?
The mean change or difference is zero
What is the rationale of the paired t-test?
calculate the difference between each pair of points, then determine if the mean of these values is different from zero
What is the method of using the paired t-test?
– Calculate the difference (d) between each pair of measurements
– Calculate the mean difference and standard error – Calculate the test statistic (|t|) and compare to the critical value
How do you interpret results compared to the critical value for the paired t-test?
– If |t| ≥ critical value, the null hypothesis is rejected, concluding the mean difference is “significantly” different from zero
– If |t| < critical value, there is not enough evidence to reject the null hypothesis, concluding the mean difference is not significantly different from zero
What are the assumptions of the two-sample t-test?
check appropriate t-test assumptions are met
samples are independent of each other
what is the null hypothesis to the two-sample t-test?
the difference between the two means is zero
What is the rationale with the two-sample t-test?
it compares the means from two independent samples, if the samples are not independent then the test will not be valid.
what is the null hypothesis for the one-sample t-test?
There is no difference between the sample population mean and an expected (given) value
What is the rationale for the one-sample t-test?
The test calculates how many standard errors the sample mean is away from the expected value (E)
How do you interpret the results for the one sample t-test?
The further away the mean is from the expected value, the larger the value of t, and the less probable it is
• The value of t can be positive or negative, it is the absolute value |t| of the difference that counts
• If |t| >critical value, then the difference is statistically significant
What is Cohen’s d looking at?
an effect size measurement that tells us the magnitude of an experimental treatment.
how do you calculate Cohen’s d?
𝑑 = (𝑀𝑔𝑟𝑜𝑢𝑝1−𝑀𝑔𝑟𝑜𝑢𝑝2) / 𝑆𝐷𝑝𝑜𝑜𝑙𝑒d
What are the results of cohens d?
d = 1, then groups’ means differ by 1 SD d = 0.5, the means differ by half an SD d = 2, means differ by 2 SDs
How do you interpret cohens d?
d <0.2, difference is minor, even if significant
d = 0.2, considered a “small” effect size
d = 0.5, considered a “medium” effect size
d = 0.8, considered a “large” effect size
What is absolute Cohen’s d value and what does it mean?
- Absolute Cohen’s d value = 1.3409 • Means differing by 1.3 standard deviations
- Indicates effect size is very large
- Concluded that difference in pH between control and treatment groups is very large and consistent enough to be very important
When would you use non-parametric tests?
- Used when underlying assumptions required for parametric tests don’t apply
- OR if data is not the correct type (i.e. categorical ordinal data, etc.)
- No assumptions are made about the underlying distributions of the sampled populations
What are the advantages of the non-parametric tests?
– Allow hypothesis testing where structure of population sampled is unknown
– Generally quicker and easier to use
– Can analyse data that consists of ranks (i.e. ordinal scale data) or classifications (i.e. nominal scale data)
What are the disadvantages of the non-parametric tests?
– Wastes data if you could apply a parametric method instead
– Not sensitive so important effects can be missed
– Some methods are laborious for large samples
What is the Mann-Whitney U test?
- Unpaired t-test equivalent
- Ranks of measurements used, not the measurements themselves
- Data ranked either from lowest to highest, or highest to lowest
- One of the most powerful non-parametric tests
- Can perform one- or two-tailed tests
What is the Wilcoxon Test?
- Paired t-test equivalent
- If the d values are not from a Normal distribution then the Wilcoxon test is applicable
- Procedure involves calculating the differences, d, as does the paired t-test
- Uses ranks of the differences
What is the one sample sign t-test?
- One sample t-test equivalent
- Uses ranks and not the actual measured values
- Compares against an expected value, E
- This is a special case of the Wilcoxon paired test but substitutes the expected value (E) for one of the two samples
