Lecture 1 - ANOVA Flashcards
What is ANOVA?
A type of statistical model used to estimate the differences between means.
(ANOVA = Analysis of Variance)
- ANOVA statistic: F-statistic -> determines whether the difference between ANY TWO MEANS (doesn’t specify which ones) is significant (we’ll come back to this point later on again)
Why would we use ANOVA and not a t-test?
A t-test is used when we have two conditions of the Independent variable (IV), whereas ANOVA is used when the conditions are more than 2
Experiment example
Experimenters wanted to test the effect of puppy-treatment on happiness (IV: puppy-treatment, DV: stress). The IV has 3 conditions, 0minutes, 15minutes and 30minutes. They took a sample of 15 participants and conducted their research.
(This example is used throughout the book constantly to explain the concepts, so I’ll use it as well to give examples throughout the whole chapter, and I’ll do the same with the next chapters as well)
Comparing means using the Linear model
What are the two general methods to compare means?
- Variance-ratio method: good for simple situations, but not good for complex ones (just mentioned, don’t need to remember anything else)
- LINEAR MODEL: good for complex situations
What is the equation for the general linear model?
outcome(i) = model + error(i)
In the case of the puppy-treatment example:
happiness = bo(^) + b1(^)group(i) + e(i).
(For a better image of both, look at slide 1, Equations 1&2)
- This model states that we can estimate happiness from group membership and bo(^).
- ~ bo(^): amount of happiness group membership value is 0, in our case, when Dose of puppies is 0 minutes
- ~ b1(^): correlation between Happiness and Dose, in our case how the 15min condition or the 30min condition affect happiness
Dummy coding
Dummy coding
What is dummy coding?
When we use 0’s and 1’s to represent categories
Dummy coding
What is the process of dummy coding?
1) Count the number of groups you have and subtract 1 from them (0min, 15min, 30min: 3 groups, 3-1 = 2)
2) The amount of groups we have now (2) are equal to the number of dummy variables we’ll use (later steps)
3) Choose one baseline group to which all the other groups will be compared (This group is usally the control group)
- NOTE: if different conditions have unequal sample sizes, baseline must contain a large number of cases to ensure that b-estimates are reliable
4) Our baseline group is the 0min group. Assign 0 to all variables for this group (what this means is also explained more at the end of the flashcard)
5) 1st Dummy variable: assign 1 to one group, 0 to the other (E.g. 15min group, Variable 1 (LONG) = 0, Variable 2 (SHORT) = 1)
6) 2nd dummy variable: same process but the opposite (30min group, Variable 1 (LONG) = 1, Variable 2 (SHORT) = 0)
(Look at slide 2: There are two dummy variables, Dummy short and Dummy long. Each person has their group membership coded in terms of 0’s and 1’s. Somebody in the 0min condition, has 0 on both Dummy Variables. Somebody in the 15min condition, has 0 on the LONG variable and 1 on the SHORT variable. Someone in the 30min condition has 1 on the LONG variable and 0 on the SHORT variable)
Dummy coding
Based on dummy coding, how can we now re-write the linear model?
(Look at Slide 1, Equation 3)
(NOTE: There are two equations under Equation 3, the bottom one is missing the error term and has a hat (estimation hat) on top of happiness. When we try and predict (estimate) happiness through the model, we don’t actually collect data, so there can’t be any error, since we don’t have data. That’s why we omit the error term)
Dummy coding
Based on this dummy coding, how can we find the values of the b-estimates?
- If I’m in the 0min group: (Equation 4)
- If I’m in the 30min group: (Equation 5)
- If I’m in the 15min group: (Equation 6)
F-Statistic
F-statistic
What does a significant F-Statistic tell us?
A significant F-Statistic tells us that the group means are signficantly different. It also gives us an overall fit of a linear model to a set of observed data:
- F = (how good the model is)/ (how bad the model is)
- F = (explained variation)/ (unexplained variation)
(2 different general equations for the F-statistic)
!!!!!!! DOESN’T TELL US the difference between specific groups, or how large that difference is. It just tells us that at least any two groups differ signficantly from each other !!!!!!!
F-Statistic
((How do we determine if an F-statistic is significant or not?))
[not needed as a question necessarily]
E.g. assume we have an F = 5.12. To see if this F is significant we will compare it to the critical value for an F-distribution with the same degrees of freedom
- For df = 2, the critical value of F is 3.89 (where p = 0.05)
- For df = 12, the critical value of F is 6.93 (where p =0.01)
So F = 5.12 is critical at the 0.05 level, but not at the 0.01 level
(Not much was expanded on this, no need to fully understand or read it. If you want a better explanation just ask me because it’s a but more complex and not even mentioned in the book)
F-Statistic
What can tell us about the differences between specific means?
B(^)-values:
- e.g. say the mean happiness of the 0min group is 2.2, and the mean happiness of the 30min group is 5. The difference of means is 5 - 2.2 = 2.8. The value of 2.8 is a t-statistic with a p-value of 0.008<0.05, so the difference between the 0min group and the 30min group is significant
- Say the mean happiness of the 15min group is 3.2. Difference between 0min and 15min group: 3.2-2.2 = 1, p-value of t = 1 is 0.282>0.05, so the difference between the 0min group and the 15min group is NOT significant
F-Statistic
How can we apply all the above in Hypothesis Testing?
- Ho model: IV has no effect on the DV, the predicted value will always be that of the grand mean (always the same value)
- Ha model: IV has an effect on the DV, described by the equations mentioned previously
- ~ The bigger the coefficients (b1(^) and b2(^)) the greater teh deviation of this model from the Ho
- ~ If the differences between the groups are large enough (if b1(^) and b2(^) are large enough) then the Ha model is a better fit to the data than the Ho
Sum of Squares
Sum of Squares
What is the total sum of squares?
The total variance between each participant and the grand mean (the grand mean is the total mena of all participants’ scores, or the mean of all group means)
(See Equation 7)
- Used to find the total amount of variation within our results
- df = total sample size - 1 = 14
Sum of Squares
What is the Model Sum of Squares (SSm)?
The amount of variation observed that the model can explain (how much of the variation can be explained by the fact that different scores coem from different treatment conditions)
(See Equation 8)
- Differences between values predicted by the model (group means) and the grand mean
- df = amount of groups - 1 = 3 - 1 =2
Sum of Squares
What is the Residual Sum of Squares (SSr)?
The amount of variation that CAN’T be explained by the model
(See Equation 9)
More specifically, it represents how much a model deviates from the observed data.
- Differences between each point and the group to which that point corresponds
- df = dfT(total) - dfM(model) = 12
!!! See Slide 4 for another explanation of what difference each type of sum of squares represents !!!
Sum of Squares
What are mean squares?
Since SSr and SSm are both sums, their size depends on the number of scores (SSm: used number of 3 groups, SSr used the sum of 15 participants). To eliminate this inequality we calculate the average mean of squares, which are basically standardized sum of squares (so that they both have the same size approximately)
- MSm: SSm/dfm -> average amount of variation explained by the model
- MSr: SSr/dfr -. average amount of variation not explained by the model
Sum of Squares
How can you represent the F-Statistic in terms of mean sum of squares?
F-Statistic = explained variation/unexplained variation
So, F = MSm/MSr
Implications of this:
- If F < 1, then unsystematic variance (SSr)>systematic variance (SSm)
- If F>1, then unsystematic variance (SSr)<systematic variance (SSm)
Assumptions
Assumptions
What are the 3 assumptions when conducting an ANOVA?
- Independence
- Normality
- Homoscedasticity (Homogeneity of variances)
Assumptions
What is the independence Assumption?
It states that the observations within each group should be independent
Assumptions
What is the Normality assumption?
It states that the data within each group should be normally distributed
(So: THE SAMPLING DISTRIBUTION of the data must be normal)
Assumptions
What is the homoscedasticity (Homogeneity of Variances) assumption?
It states that the variance of data within each group should be equal
(In other words, the Residuals are equal in each group)
- Tested through the Brown-Forsythe F test or Welch’s F test
Assumptions
Is ANOVA robust?
- If the normality assumption is violated, F is affected a lot, even if the group sizes are equal. Therefore, F can only be robust if the distributions of each group are identical
- If the independence assumption is violated: big problem. If this assumption is violated and scores are correlated just a bit, then the Type I error rate will skyrocket
- If homoscedasticity is violated: Biased F-statistic, underestimated significance level, unreliabe results
Assumptions
What do we do if any assumptions are violated?
If any assumptions are violated then we just use a Welch Test, interpret the Welch F-Statistic.
OR we apply any other robust tests
Planned contrasts
Planned Contrasts
What is Contrast coding and why do we use it?
It’s a way of coding in which we assign weights to groups in variables in order to carry out planned contrasts.
Why we use it:
- In the puppy example, we have 2 dummy variables, which results in us using two t-tests (since we compare 0min and 15min group, and then 0min and 30 min group). This increases though the Type I error rate.
- Also, the variabes might not make all the comparisons we want to make (e.g. in puppy example, the 15min and 30min group are never compared)
Planned Contrasts
How do we break down the variation into components?
See Slide 5
(F statistic inly breaks down SSt into SSm and SSr. Planned Contrasts break down the SSm even further)
Planned Contrasts
What are some general notes on planned contrasts?
- If you have a control group, it’s because you want to compare it against any other experimental groups (See contrast 1 of Slide 5, Control vs 15min and 30min groups) -> Usually this is the first contrast all researchers make
- Each contrast may only have “2 pieces of variation” (e.g. control vs experimental, 15min vs 30min, control vs 15min etc.)
- ~ You always end up with k-1 contrasts (k is the number of condition you’re comparing. In our case, we’re comparing 3 conditions, so we have 2 contrasts in general)
- Once a group has been singled out, it can’t be used in another contrast. (E.g. once control group is used in the first contrast against experimental conditions in general it can’t be used again in other contrasts. Since in the first contrast we don’t mention specifically any of the two experimental conditions, we can use them in the next contrast. Since in the second contrast, we compare both experimental conditions, then we can’t use them in another contrast. Since we have no conditions left, the contrast ends there)
Planned Contrasts
What are some notes on making planned contrast tables?
The values we assign to variables are called weights. There are a few rules when it comes to assigning weights and getting the planned contrasts we want:
- Groups with negative weights are compared to groups with positive weights
- If you add up the weights the contrast should amount to 0. If all products sum up to 0, then the contrasts are called orthogonal
- ~ There can laos be contrasts that are related, called non-orthogonal (not discussed at all in this chapter)
- If a group isn’t in a contrast assign it a value of 0
(See Slide 6 for a a table for contrasts)
Planned Contrasts
What do b(^) values represent when doing planned contrasts?
In general, b(^) values represent the differences between the means that the contrasts were designed to test.
The linear model when doing planned contrasts is:
happiness(^) = b0(^) + b1(^)contrast1 + b2(^)contrast2
- b0(^) = grand mean = (X30min - X15min - X0min)/3
- b1(^) = See equation 10. Represents the difference between the two experimental conditions and the control group (0min)
- b2(^) = See equation 11
Post hoc procedures
Post Hoc Procedures
What are post hoc procedures?
Procedures that make pairwise comparisons: They compare all different combinations of treatment groups
- This way they control for Type I error rate by correcting α for each test, and without losing any power
- The Least-significance difference (LSD) pairwise comparison doesn’t control for Type I error rate
- Bonferroni’s and Tukey’s corrections both control the Type I error rate very well but lack the power (And they only make the correction when there are equal group varianecs and sample sizes).
- ~ Bonferroni is more powerful when the number of comparisons is small, Tukey is more powerful when testing large number of means
Post Hoc Procedures
Are Post Hoc procedures robust?
Most procedures perform well if there are small violations on the NORMALITY ASSUMPTION
Most procedures perform badly if group sizes aren’t equal and if the population variances are very different
Effect size
Effect sizes
R^2
R^2 = η^2 = SSm/SSr
Effect sizes
ω^2
ω^2 = [dfM + (MSm - MSr)]/ (SSt + MSr)
- value of 0.01 is small
- value of 0.06 is medium
- value of 0.14 is large
JASP
(See rest of powerpoint)
