Week 4: T-Tests and ANOVA

Question 1

What is the purpose of a t-test?

Answer 1

To compare the means of two groups and determine if their difference represents a real difference in the population or occurred by chance.
T-tests are used to quantify how far apart the two means area. In other words, how many SEs is the observed ‘difference in the sample means’ away from zero?

Question 2

When should the t-distribution be used instead of the z-distribution?

Answer 2

When the sample size is small or the population variance (σ) is unknown

Question 3

What are the three types of t-test?

Answer 3

Paired t-test; Independent t-test (equal or unequal variance); One sample t-test

Question 4

What are degrees of freedom in a t-test?

Answer 4

The number of values in a calculation that a free to vary, typically N - 1 for a sample

Question 5

List the steps for hypothesis testing

Answer 5

1. Defne H0
2. Define H1
3. Choose a significance level (α)
4. Select and calculate the test statistic
5. Compare the test statistic to the critical value or p-value
6. Interpret results

Question 6

What is the H0 and H1 for a t-test?

Answer 6

H0 = μA - μB = 0 (the means of the two groups are the same)
H1 = 𝜇A ≠ 𝜇B (the 2 samples come from different populations)
​

Question 7

What assumptions are made for parametric tests like t-test and ANOVA?

Answer 7

Normality of data distribution; Homogeneity of variance (equal variance)

Question 8

What is ANOVA used for?

Answer 8

To compare the means of three or more groups (e.g., μ1, μ2, μ3) to determine if at least one mean differs significantly

Question 9

What is the F-statistic in ANOVA?

Answer 9

The ratio of between-group variance to within-group variance

Question 10

When is a non-parametric test preferred over a parametric?

Answer 10

1. When the sample size is small
2. When data are non-normal or contain outliers
3. For analysing ordinal or ranked data

Question 11

Define the critical value for a t-test

Answer 11

A threshold that the test statistic must exceed to reject H0 at a given α

Question 12

What is the pooled variance in an independent t-test?

Answer 12

A weighted average of the variances of the two groups, used when assuming equal variances

Question 13

What does a post-hoc test in ANOVA do?

Answer 13

Identifies which specific means differ after finding a significant F-test result (e.g., Tukey post-hoc test)

Question 14

What are the advantages of parametric tests?

Answer 14

1. Greater statistical power for detecting differences
2. Robust to violations of normality if the sample size is large

Question 15

What are advantages of non-parametric tests?

Answer 15

1. Valid for small sample sizes and non-normal data
2. Can handle ordinal data and outliers effectively
3. Assess the median which can be better for highly-skewed distributions

Question 16

What happens if multiple t-tests are used instead of ANOVA?

Answer 16

It increases the risk of Type I errors due to the inflation of the overall α
The probability of making one or more Type I errors is 1 - (1-.05)^3 = 0.14

Question 17

What is the difference between z and t distribution?

Answer 17

t is more spread out than Z (heavier tails)
The sample distribution of t depends on n (and has n-1 df). The larger the sample to estimate σ (df→∞), the closer is the t distribution to the normal distribution (z statistic)

Question 18

All other factors being equal, is a large ¯d more or less likely to occur by chance than a small mean difference?

Answer 18

Less

Question 19

Given the same ¯d, are groups with smaller SDs more or less likely to report a significant difference than groups with larger SDs?

Answer 19

More

Question 20

What’s the relationship between t-tests and sample sizes?

Answer 20

The more subjects, the more confident we can be that the differences we find did not occur by chance

Question 21

In a t-test, what do we need to assess how likely the difference between means is to be “real”?

Answer 21

• The mean difference, ¯d
• The SD for each group
• The number of subjects in each group

Question 22

When is there strong evidence against H0 for a t-test?

Answer 22

t_obs ≥ t_ critical at a certain α

Question 23

What is a paired (or dependent) t-test?

Answer 23

Difference between average scores of a single sample of individuals who is assessed at two different times or on two different measures. It can also compare average scores of individuals who are paired in terms of a particular characteristic.
In the population, the two means measured at different times might either:
- be the same (μt1= μt2 → μt1 – μt2 = δ = 0)
- be different (μt1 ≠ μt2 ; μt1 - μt2 < 0 : μt1-μt2 > 0)

Question 24

What is an independent t-test?

Answer 24

Compares the means of two samples selected independently of each other (i.e., subjects in the two groups are not the same). There are then two types of independent t-tests: equal/pooled variance and unequal/separate variance.
In the population, the two means measured among two different groups might either:
- be the same (μA = μB → μA – μB = δ = 0)
- be different (μA ≠ μB ; μA - μB < 0 ; μA - μB > 0

Question 25

What do we assume with t-tests?

Answer 25

No difference between the means.
Is the difference in the sampling means (¯d= x̄t1– x̄t2) significantly different from zero?

Question 26

What is the difference between one- and two-way ANOVA?

Answer 26

One-way: One characteristic/variable to define three or more groups of data
Two-way: Considers two independent variables

Question 27

What are the null and alternative HPs for ANOVA?

Answer 27

H0: μ1 = μ2 = … =μg (g ≥ 3)
H1: not all population means are identical or, at least two means differ from each other

Question 28

Why consider variances with ANOVA?

Answer 28

The “relative location: of the group means can be more easily identified by variance among the group means than comparing many group means directly (when # groups large)
The ANOVA method assesses the relative size of variance among group means (between-group variance) compared to the average variance within groups (within-group variance)

Question 29

What is the main interest of ANOVA? Why?

Answer 29

The ratio of between-group variance to within group variance. If the ratio is greater than expected by chance, we may think that at least one mean is different.
When between-group variances are the same, mean differences among groups seem more distinct in the distributions with smaller within-group variances compared to those with larger within-group variances

Question 30

What distribution is the F statistic known to follow?

Answer 30

F distribution with
df1=(#groups-1)
df2=(n-#groups)
To test our HP, we compare the F value calculated with the critical value at an α error level (0.05) in the relevant F table

Question 31

What does a larger F value imply?

Answer 31

The means of the groups are different from each other compared to the variation of the individual observations in each group
If the F value is larger than the critical value, there is strong evidence against H0 (differences between group means are larger than what would be expected by chance if the H0 was correct. At least one group mean differs from the others)

Question 32

What are the non-parametric alternatives to t-test and ANOVA?

Answer 32

T-test: Mann-Whitney U test; Wilcoxon T-test
ANOVA: Kruskal-Wallis test

Question 33

How do you decide whether to use a parametric or non-parametric test?

Answer 33

If the sample size is tiny, you may be forced to use a non-parametric test.
For larger samples, if the mean is a better measure of central tendency for the distribution of the data, a parametric test is usually better. If the median is a better measure, consider a non-parametric test (regardless of the larger sample size).
If some assumptions are violated, check both