Lecture 8: ANOVA

Question 1

ANOVA

Answer 1

Example of GLM, stands short for Analysis of Variance. It is a statistical method to compare means among three or more groups (as the Independent Sample T-test only suffices for 2 groups). The null hypothesis is that there are no differences in the means of the groups, while the alternative hypothesis states that there are likely differences in the means. We can do follow-up analyses to understand which groups differ significantly.

One-way ANOVA has one categorical predictor (e.g., socioeconomic status with the categories Low, Medium and High) and a continuous outcome. Because it has a categorical predictor, we only predict one value for each category, which is that category’s mean.

Question 2

Between-group sum of squares

Answer 2

In ANOVA, the regression sum of squares is called the between-group sum of squares. It measures the variability among the means of different groups being compared.

Question 3

Within-group sum of squares

Answer 3

In ANOVA, the sum of squared errors is called the within-group sum of squares. It measures the variability within each group and quantifies how many individual data points this group deviates from their respective group means.

Question 4

Eta squared

Answer 4

In ANOVA models, R2 (R squared) is called eta squared. R-squared and eta squared are exactly the same.

Eta squared values range from 0 to 1. A value from 0 means that none of the variability is explained by the group differences (the groups are all the same), while a value of 1 means that all the variability is due to group differences. Generally, the larger the eta squared value, the stronger the effect. It reflects the proportion of variance in the outcome variable that can be explained by the categorical predictor.

For eta squared, the rules of thumb differ from R-squared:

Small: 0.01
Medium: 0.06
Large: 0.14

Question 5

Formula for eta squared

Answer 5

The formula for eta squared is the ratio between the within-group sum of squares and the between-group sum of squares.

SSB / TSS

Where SSB is the sum of squares between groups (variability between group means) and SST is the total sum of squares (total variability in the data).

Question 6

F-distribution in ANOVA

Answer 6

The F-distribution has two degrees of freedom (df) parameters:

1. Numerator df: dfb = g - 1

Where dfb are the between degrees of freedom and g are the number of groups

2. Denominator df: dfw = n - g

Where dfw are the within degrees of freedom and n are the number of observations

The F-test in ANOVA helps determine whether the observed differences between group means are statistically significant. The numerator degrees of freedom reflect the number of groups being compared, while the denominator degrees of freedom reflect the total number of observations and the number of groups.

Question 7

Guidelines for R-squared

Answer 7

Small: 0.01
Medium: 0.06
Large: 0.138