L06-L07 Parametric Tests

Question 1

Outline the procedure for hypothesis testing.

Answer 1

1) Define the problem.
- No. of groups & which samples were compared?
- Two groups: Independent or paired? Equal variances?
- Outcome of interest?
- Type of data? Normally distributed?
- Two-tailed or one-tailed test?
2) State null hypothesis H0 & alternative hypothesis H1
- H0: there is NO effect/difference
- H1: there is some effect/difference
3) Compute test statistic
4) Find the p-value for computed test statistic
5) Compare p-value for computed test statistic with given significance level (alpha) = 0.05 usually
6) State conclusion
- If p < alpha: Reject H0; result is statistically significant at given significance level
- If p >= alpha: Fail to reject H0; result is NOT statistically significant at given significance level

Question 2

State the assumptions when using parametric tests.

Answer 2

1) Samples are drawn from normally distributed populations.

2) Variances are the same in all samples compared

Question 3

State the purpose behind the hypothesis testing of paired samples t-test.

Answer 3

To test the H0 that the mean of the underlying population of differences in values of each pair is zero.

Question 4

State the assumptions when using paired samples t-tests.

Answer 4

1) Samples are randomly sampled from their populations.
2) The two underlying populations are paired, thus variances will be similar.
3) The population of differences in values for each pair is normally distributed

Question 5

E.g. of how to write conclusion of paired & independent samples t-test.

Answer 5

Paired:
At a significance level of 0.05, there is a statistically significant mean difference of 0.36 +- 0.41 mmol/L (p = 0.005) between the LDL cholesterol level after a 2-week diet with oat bran cereal (4.08 +- 1.06 mmol/L) and that after a 2-week diet with corn flakes (4.44 +- 0.97 mmol/L).

Independent:
At a significance level of 0.05, there is no statistically significant difference between the mean dissolution rates (measured by % dissolution after 15 min in this study) of the two formulations (57.45 +- 4.76% vs 61.33 +- 5.30%, p = 0.08).

Question 6

State the purpose behind the hypothesis testing of independent samples t-test.

Answer 6

To test the H0 that the two population means corresponding to the two random samples are equal. (i.e. no mean difference)

Question 7

State the assumptions when using independent samples t-tests.

Answer 7

1) Samples are randomly sampled from their populations.
2) The two underlying populations are independent.
3) The two underlying populations are normally distributed.
4) The two underlying populations have equal variances.
- If variances are not significantly different (i.e. p >= 0.05 for F test or Levene’s test for equality of variances), use the independent samples t-tests for equal variances.
- If variances are significantly different (i.e. p < 0.05 for F test or Levene’s test for equality of variances), use the independent samples t-tests for unequal variances.

Question 8

What statistical tests are used to test for equality of variances between two independent samples?

Answer 8

F test or Levene’s test

• F test is more restrictive since it requires normal populations & ONLY 2 groups can be compared.
• Levene’s test is more widely used since it is applicable whether or not the data are normally distributed & can also be used to test equality of variances when > 2 groups are being compared.

Question 9

What is the assumption behind the use of F test?

Answer 9

Populations from which samples are obtained must be normal.

Question 10

Explain the purpose behind the use of F / Levene’s test.

Answer 10

Designed to test if (at least) two population variances are equal by comparing the ratio of two sample variances.

• If two samples come from populations that have equal variances, the ratio of the sample variances will be close to 1.
• For independent sample t-tests: Numerator is the sample size of group with larger variance, thus all F values are non-negative
• F test is ALWAYS a one-tailed test!
• For one-way ANOVA: Numerator is the between-group variance, thus all F values are non-negative

Question 11

State the purpose behind the hypothesis testing of one-way ANOVA.

Answer 11

“One-way” indicates that there is one independent variable of interest.
ANOVA = analysis of variances
- Dependent on estimates of spread or dispersion

To test the H0 that all the population means corresponding to the random samples are equal.

• H0: All the means of the underlying populations are the same.
• H1: Not all the means of the underlying populations are the same OR The means of at least two of the underlying populations are different.

Question 12

State the assumptions when using one-way ANOVA.

Answer 12

1) Samples are randomly sampled from their populations.
2) The underlying populations are independent.
3) The underlying populations are normally distributed.
4) The underlying populations have equal variances.
- If variances are not significantly different (i.e. p >= 0.05 for Levene’s test for equality of variances), continue with one-way ANOVA.
- If variances are significantly different (i.e. p < 0.05 for Levene’s test for equality of variances), use Welch ANOVA instead.

Question 13

Why do we need to use one-way ANOVA for comparing more than 2 samples, rather than conducting all possible independent samples t-tests?

Answer 13

To control the overall probability of making a Type I error (i.e. false positive) on the predetermined significance level (alpha).
- i.e. to ensure the overall alpha = the predetermined level (e.g. 0.05)

Assuming we compare the means among 3 groups & we conduct 3 pairwise comparisons using independent samples t-tests, with alpha = 0.05 for each test:

• By multiplicative rule, the probability of failing to reject H0 in all 3 tests when H0 is indeed true (i.e. making the correct conclusion) = (1 - 0.05)^3 = 0.857
• Consequently, the overall probability of rejecting H0 in at least one of the t-tests when in fact, H0 is indeed true (i.e. making a Type I error) = 1 - 0.857 = 0.143 > 0.05

Question 14

How many sources of variation can occur when the means among more than two groups, which are randomly sampled from underlying populations with equal variances, are compared against? Name these sources of variation.

Answer 14

Total variation is made up of two components:

1) Within-group variation
- Variation of individual values around their population means

2) Between-group variation
- Variation of population means around the overall mean

Question 15

If between-group variability is large compared with within-group variability in one-way ANOVA, this means the underlying population means are _____.

Answer 15

different

Question 16

If between-group variability is small compared with within-group variability in one-way ANOVA, this means the underlying population means are _____.

Answer 16

NOT different

Question 17

E.g. of how to write conclusion of one-way ANOVA.

Answer 17

At a significance level of 0.05, NOT all the mean FEV1 of the patients from the 3 hospitals are the same.

OR

At a significance level of 0.05, the mean FEV1 of the patients in AT LEAST TWO of the hospitals are the different.

If p-value < 0.05, proceed w/ post-hoc tests (i.e. multiple comparisons procedures) to determine where the differences lie.

• At this stage, we only know that difference exists BUT we do not know where the difference are.
• If p-value >= 0.05, end conclusion here.

Question 18

Explain the purpose of post-hoc tests in one-way ANOVA.

Answer 18

Once we reject H0 & conclude that NOT all the means are the same, we only now that difference exists BUT we do not know where the differences lies.

Thus, post-hoc tests help us (1) identify the difference(s) (2) while controlling the overall probability of making a Type I error (i.e. false positive) on the predetermined significance level (alpha).

Question 19

How are post-hoc tests conducted in one-way ANOVA?

Answer 19

1) Post-hoc tests typically involve testing each pair of means individually.

2) Selection of appropriate post-hoc test depends on:
- Types of comparisons to be made (e.g. all possible pairwise tests [i.e. 1 vs 2, 1 vs 3 & 2 vs 3], or compare at least two treatment groups with a single control group [i.e. control vs 2 & control vs 3]?)
- Whether sample size is the same for all samples; NOT critical in one-way ANOVA due to more sophisticated tests that circumvent this issue

3) Different post-hoc comparisons vary in terms of how conservative each test is:
- Larger difference between means is required for statistical significance when a more conservative post-hoc comparison is used.
- Using a more conservative comparison reduces statistical power (1 - beta), translating to a greater chance for Type II error (i.e. false negative = beta), BUT the chance of a Type I error (i.e. false positive = alpha) can be controlled.

Question 20

Describe the uses, strengths & limitations of the different common types of post-hoc tests in one-way ANOVA.

Answer 20

1) Bonferroni adjustment
- Very simple approach & widely applicable to any type of statistical tests (e.g. parametric & non-parametric)
- Very conservative, hence statistical power is much reduced (i.e. greater chance for Type II error / false negative)

2) Scheffe’s procedure
- Most conservative (alongside Bonferroni adjustment), hence least statistical power (i.e. greatest chance for Type II error / false negative)
- Very flexible approach: can be applied for more complicated comparisons for a large number of groups

3) Tukey’s test
- More conservative than LSD test; reduced statistical power BUT less chance for Type I error / false positive
- Originally developed for equal sample sizes, but can be adapted to unequal sample sizes

4) Least significant difference (LSD) test
- NO correction to control significance level & thus highly DISCOURAGED to use!!
- Least conservative; results in more significant differences than would be expected according to alpha level (i.e. greater chance for Type I error / false positive)
- Less chance of missing real differences due to greater statistical power i.e. reduced chance for Type II error / false negative

5) Dunnett’s test
- Most suitable for intervention-placebo/control comparisons, BUT NOT comparing the others to each other (i.e. intervention 1 vs intervention 2)
- When appropriate, i.e. really interested only in comparisons of one group to each of the others, this approach is more powerful than methods performing all possible pairwise comparisons

Question 21

Describe how Bonferroni adjustment post-hoc analysis is conducted.

Answer 21

Signficance level for each pairwise comparison is set to alpha / m, where m is number of pairwise comparisons planned.
- E.g. for three pairwise comparison planned at overall alpha = 0.05, thus m = 3 & significance level for each pairwise comparison = 0.05 / 3 = 0.017

Subsequently conduct pairwise independent samples t-tests at significance level alpha/m = 0.017
- Usually, statistical software would have accounted for Bonferroni adjustment in post-hoc analysis & thus just to need check if p-value < 0.05 instead of adjusted alpha/m = 0.017

Question 22

In what situations will repeated measures ANOVA should be used?

Answer 22

Changes in a particular measure on the SAME group of subjects over different conditions e.g. time etc.