Lecture 1 - ANOVA

Question 1

What is ANOVA?

Answer 1

A type of statistical model used to estimate the differences between means.
(ANOVA = Analysis of Variance)
- ANOVA statistic: F-statistic -> determines whether the difference between ANY TWO MEANS (doesn’t specify which ones) is significant (we’ll come back to this point later on again)

Question 2

Why would we use ANOVA and not a t-test?

Answer 2

A t-test is used when we have two conditions of the Independent variable (IV), whereas ANOVA is used when the conditions are more than 2

Question 3

Experiment example

Answer 3

Experimenters wanted to test the effect of puppy-treatment on happiness (IV: puppy-treatment, DV: stress). The IV has 3 conditions, 0minutes, 15minutes and 30minutes. They took a sample of 15 participants and conducted their research.
(This example is used throughout the book constantly to explain the concepts, so I’ll use it as well to give examples throughout the whole chapter, and I’ll do the same with the next chapters as well)

Question 4

Comparing means using the Linear model

Question 5

What are the two general methods to compare means?

Answer 5

• Variance-ratio method: good for simple situations, but not good for complex ones (just mentioned, don’t need to remember anything else)
• LINEAR MODEL: good for complex situations

Question 6

What is the equation for the general linear model?

Answer 6

outcome(i) = model + error(i)
In the case of the puppy-treatment example:
happiness = bo(^) + b1(^)group(i) + e(i).
(For a better image of both, look at slide 1, Equations 1&2)
- This model states that we can estimate happiness from group membership and bo(^).
- ~ bo(^): amount of happiness group membership value is 0, in our case, when Dose of puppies is 0 minutes
- ~ b1(^): correlation between Happiness and Dose, in our case how the 15min condition or the 30min condition affect happiness

Question 7

Dummy coding

Question 8

Dummy coding

What is dummy coding?

Answer 8

When we use 0’s and 1’s to represent categories

Question 9

Dummy coding

What is the process of dummy coding?

Answer 9

1) Count the number of groups you have and subtract 1 from them (0min, 15min, 30min: 3 groups, 3-1 = 2)
2) The amount of groups we have now (2) are equal to the number of dummy variables we’ll use (later steps)
3) Choose one baseline group to which all the other groups will be compared (This group is usally the control group)
- NOTE: if different conditions have unequal sample sizes, baseline must contain a large number of cases to ensure that b-estimates are reliable
4) Our baseline group is the 0min group. Assign 0 to all variables for this group (what this means is also explained more at the end of the flashcard)
5) 1st Dummy variable: assign 1 to one group, 0 to the other (E.g. 15min group, Variable 1 (LONG) = 0, Variable 2 (SHORT) = 1)
6) 2nd dummy variable: same process but the opposite (30min group, Variable 1 (LONG) = 1, Variable 2 (SHORT) = 0)
(Look at slide 2: There are two dummy variables, Dummy short and Dummy long. Each person has their group membership coded in terms of 0’s and 1’s. Somebody in the 0min condition, has 0 on both Dummy Variables. Somebody in the 15min condition, has 0 on the LONG variable and 1 on the SHORT variable. Someone in the 30min condition has 1 on the LONG variable and 0 on the SHORT variable)

Question 10

Dummy coding

Based on dummy coding, how can we now re-write the linear model?

Answer 10

(Look at Slide 1, Equation 3)
(NOTE: There are two equations under Equation 3, the bottom one is missing the error term and has a hat (estimation hat) on top of happiness. When we try and predict (estimate) happiness through the model, we don’t actually collect data, so there can’t be any error, since we don’t have data. That’s why we omit the error term)

Question 11

Dummy coding

Based on this dummy coding, how can we find the values of the b-estimates?

Answer 11

• If I’m in the 0min group: (Equation 4)
• If I’m in the 30min group: (Equation 5)
• If I’m in the 15min group: (Equation 6)

Question 12

F-Statistic

Question 13

F-statistic

What does a significant F-Statistic tell us?

Answer 13

A significant F-Statistic tells us that the group means are signficantly different. It also gives us an overall fit of a linear model to a set of observed data:
- F = (how good the model is)/ (how bad the model is)
- F = (explained variation)/ (unexplained variation)
(2 different general equations for the F-statistic)
!!!!!!! DOESN’T TELL US the difference between specific groups, or how large that difference is. It just tells us that at least any two groups differ signficantly from each other !!!!!!!

Question 14

F-Statistic

((How do we determine if an F-statistic is significant or not?))
[not needed as a question necessarily]

Answer 14

E.g. assume we have an F = 5.12. To see if this F is significant we will compare it to the critical value for an F-distribution with the same degrees of freedom
- For df = 2, the critical value of F is 3.89 (where p = 0.05)
- For df = 12, the critical value of F is 6.93 (where p =0.01)
So F = 5.12 is critical at the 0.05 level, but not at the 0.01 level
(Not much was expanded on this, no need to fully understand or read it. If you want a better explanation just ask me because it’s a but more complex and not even mentioned in the book)

Question 15

F-Statistic

What can tell us about the differences between specific means?

Answer 15

B(^)-values:
- e.g. say the mean happiness of the 0min group is 2.2, and the mean happiness of the 30min group is 5. The difference of means is 5 - 2.2 = 2.8. The value of 2.8 is a t-statistic with a p-value of 0.008<0.05, so the difference between the 0min group and the 30min group is significant
- Say the mean happiness of the 15min group is 3.2. Difference between 0min and 15min group: 3.2-2.2 = 1, p-value of t = 1 is 0.282>0.05, so the difference between the 0min group and the 15min group is NOT significant

Question 16

F-Statistic

How can we apply all the above in Hypothesis Testing?

Answer 16

• Ho model: IV has no effect on the DV, the predicted value will always be that of the grand mean (always the same value)
• Ha model: IV has an effect on the DV, described by the equations mentioned previously
• ~ The bigger the coefficients (b1(^) and b2(^)) the greater teh deviation of this model from the Ho
• ~ If the differences between the groups are large enough (if b1(^) and b2(^) are large enough) then the Ha model is a better fit to the data than the Ho

Question 17

Sum of Squares

Question 18

Sum of Squares

What is the total sum of squares?

Answer 18

The total variance between each participant and the grand mean (the grand mean is the total mena of all participants’ scores, or the mean of all group means)
(See Equation 7)
- Used to find the total amount of variation within our results
- df = total sample size - 1 = 14

Question 19

Sum of Squares

What is the Model Sum of Squares (SSm)?

Answer 19

The amount of variation observed that the model can explain (how much of the variation can be explained by the fact that different scores coem from different treatment conditions)
(See Equation 8)
- Differences between values predicted by the model (group means) and the grand mean
- df = amount of groups - 1 = 3 - 1 =2

Question 20

Sum of Squares

What is the Residual Sum of Squares (SSr)?

Answer 20

The amount of variation that CAN’T be explained by the model
(See Equation 9)
More specifically, it represents how much a model deviates from the observed data.
- Differences between each point and the group to which that point corresponds
- df = dfT(total) - dfM(model) = 12

!!! See Slide 4 for another explanation of what difference each type of sum of squares represents !!!

Question 21

Sum of Squares

What are mean squares?

Answer 21

Since SSr and SSm are both sums, their size depends on the number of scores (SSm: used number of 3 groups, SSr used the sum of 15 participants). To eliminate this inequality we calculate the average mean of squares, which are basically standardized sum of squares (so that they both have the same size approximately)
- MSm: SSm/dfm -> average amount of variation explained by the model
- MSr: SSr/dfr -. average amount of variation not explained by the model

Question 22

Sum of Squares

How can you represent the F-Statistic in terms of mean sum of squares?

Answer 22

F-Statistic = explained variation/unexplained variation
So, F = MSm/MSr
Implications of this:
- If F < 1, then unsystematic variance (SSr)>systematic variance (SSm)
- If F>1, then unsystematic variance (SSr)<systematic variance (SSm)

Question 23

Assumptions

Question 24

Assumptions

What are the 3 assumptions when conducting an ANOVA?

Answer 24

• Independence
• Normality
• Homoscedasticity (Homogeneity of variances)

Question 25

Assumptions

What is the independence Assumption?

Answer 25

It states that the observations within each group should be independent

Question 26

Assumptions

What is the Normality assumption?

Answer 26

It states that the data within each group should be normally distributed
(So: THE SAMPLING DISTRIBUTION of the data must be normal)

Question 27

Assumptions

What is the homoscedasticity (Homogeneity of Variances) assumption?

Answer 27

It states that the variance of data within each group should be equal
(In other words, the Residuals are equal in each group)
- Tested through the Brown-Forsythe F test or Welch’s F test

Question 28

Assumptions

Is ANOVA robust?

Answer 28

• If the normality assumption is violated, F is affected a lot, even if the group sizes are equal. Therefore, F can only be robust if the distributions of each group are identical
• If the independence assumption is violated: big problem. If this assumption is violated and scores are correlated just a bit, then the Type I error rate will skyrocket
• If homoscedasticity is violated: Biased F-statistic, underestimated significance level, unreliabe results

Question 29

Assumptions

What do we do if any assumptions are violated?

Answer 29

If any assumptions are violated then we just use a Welch Test, interpret the Welch F-Statistic.
OR we apply any other robust tests

Question 30

Planned contrasts

Question 31

Planned Contrasts

What is Contrast coding and why do we use it?

Answer 31

It’s a way of coding in which we assign weights to groups in variables in order to carry out planned contrasts.
Why we use it:
- In the puppy example, we have 2 dummy variables, which results in us using two t-tests (since we compare 0min and 15min group, and then 0min and 30 min group). This increases though the Type I error rate.
- Also, the variabes might not make all the comparisons we want to make (e.g. in puppy example, the 15min and 30min group are never compared)

Question 32

Planned Contrasts

How do we break down the variation into components?

Answer 32

See Slide 5
(F statistic inly breaks down SSt into SSm and SSr. Planned Contrasts break down the SSm even further)

Question 33

Planned Contrasts

What are some general notes on planned contrasts?

Answer 33

• If you have a control group, it’s because you want to compare it against any other experimental groups (See contrast 1 of Slide 5, Control vs 15min and 30min groups) -> Usually this is the first contrast all researchers make
• Each contrast may only have “2 pieces of variation” (e.g. control vs experimental, 15min vs 30min, control vs 15min etc.)
• ~ You always end up with k-1 contrasts (k is the number of condition you’re comparing. In our case, we’re comparing 3 conditions, so we have 2 contrasts in general)
• Once a group has been singled out, it can’t be used in another contrast. (E.g. once control group is used in the first contrast against experimental conditions in general it can’t be used again in other contrasts. Since in the first contrast we don’t mention specifically any of the two experimental conditions, we can use them in the next contrast. Since in the second contrast, we compare both experimental conditions, then we can’t use them in another contrast. Since we have no conditions left, the contrast ends there)

Question 34

Planned Contrasts

What are some notes on making planned contrast tables?

Answer 34

The values we assign to variables are called weights. There are a few rules when it comes to assigning weights and getting the planned contrasts we want:
- Groups with negative weights are compared to groups with positive weights
- If you add up the weights the contrast should amount to 0. If all products sum up to 0, then the contrasts are called orthogonal
- ~ There can laos be contrasts that are related, called non-orthogonal (not discussed at all in this chapter)
- If a group isn’t in a contrast assign it a value of 0
(See Slide 6 for a a table for contrasts)

Question 35

Planned Contrasts

What do b(^) values represent when doing planned contrasts?

Answer 35

In general, b(^) values represent the differences between the means that the contrasts were designed to test.
The linear model when doing planned contrasts is:
happiness(^) = b0(^) + b1(^)contrast1 + b2(^)contrast2
- b0(^) = grand mean = (X30min - X15min - X0min)/3
- b1(^) = See equation 10. Represents the difference between the two experimental conditions and the control group (0min)
- b2(^) = See equation 11

Question 36

Post hoc procedures

Question 37

Post Hoc Procedures

What are post hoc procedures?

Answer 37

Procedures that make pairwise comparisons: They compare all different combinations of treatment groups
- This way they control for Type I error rate by correcting α for each test, and without losing any power
- The Least-significance difference (LSD) pairwise comparison doesn’t control for Type I error rate
- Bonferroni’s and Tukey’s corrections both control the Type I error rate very well but lack the power (And they only make the correction when there are equal group varianecs and sample sizes).
- ~ Bonferroni is more powerful when the number of comparisons is small, Tukey is more powerful when testing large number of means

Question 38

Post Hoc Procedures

Are Post Hoc procedures robust?

Answer 38

Most procedures perform well if there are small violations on the NORMALITY ASSUMPTION
Most procedures perform badly if group sizes aren’t equal and if the population variances are very different

Question 39

Effect size

Question 40

Effect sizes

R^2

Answer 40

R^2 = η^2 = SSm/SSr

Question 41

Effect sizes

ω^2

Answer 41

ω^2 = [dfM + (MSm - MSr)]/ (SSt + MSr)
- value of 0.01 is small
- value of 0.06 is medium
- value of 0.14 is large

Question 42

JASP

Answer 42

(See rest of powerpoint)