lecture 17 - comparing multiple conditions - 1 way ANOVA

Question 1

Beyond the t-test

Answer 1

• The t-test is great if you want to compare 2 conditions. But often there is interest in more than 2 conditions. E.g.:
  
  • Several different drug doses vs placebo.
  • Mood as a function of season…
• Want to be able to ask the general question “are there any differences between the condition means?”
  
  • Which is not the same as asking “Are these two conditions different?” multiple times.
• So, need more than the t-test, and ANOVA (Analysis of Variance) is the answer!
  Well, one answer. ANOVA is a parametric method, and there are non-parametric ways to ask about comparing multiple conditions (though we don’t cover them on this course).

Question 2

But why analysis of variance? General concept

Answer 2

Two distributions
with different means
NB. The combined distribution has larger variance than the individual ones. - if you’ve got two different conditions and you put them together the variance gets bigger
Two distributions
with the same mean
NB. The variance of the combined distribution matches the individual distributions. - the height of the distribution changes but its range and variance doesn’t change - the variance is the same.
So, variance calculated
with, vs without, the
assumption of multiple conditions should only differ if there are actually differences between condition means.

Question 3

distribution graphs in notes

Answer 3

we analyse the variance as if there are different distributions and then we analyse the variance as if there is no difference between conditions. if null hypothesis is true then estimates of variance should be the same. if null hypothesis is false then two estimates of overall variance will be different.

Question 4

grand mean

Answer 4

looking at differences between each data point and overall mean. we take those differences square them and add them up.
if two conditions can calculate variance a different way. calculate the means of each condition and then look at the difference between those condition means and the grand mean - partitioning out the variability.

Question 5

equation

Answer 5

not expected to do ANOVA by hand

F = MSa/ MS s/a

F ratio the observed statistic we get out the test

MSa - observed variance due to factor A

MS s/a - expected variance without factor A estimated if no effect (Ho)
if the null hypothesis is true and there are no differences between conditions then those two estimates should be the same

sum squares
SS = ∑( Y - Ybar) ^2

mean squares
MS = SS/ df

variance
o^2 = ∑(Y - Y bar) ^2/ N-1

N- 1 = df

Question 6

graphical overview

Answer 6

if null hypothesis is false the mean square for our factor or the thing we are looking at should be bigger than the other one

Question 7

The ANOVA concept summarised

Answer 7

• “Model” the data with the assumption it comes from several conditions with different means.
• “Model” the data without this assumption (i.e. under the assumption of the null hypothesis or H0).
• If the null hypothesis (H0) is true, then the estimate of the variance should be same both ways.
  Note – not covering the way to make these variance calculations here (SPSS will do ANOVA for you) – but still worth knowing what SPSS (or any other stats program) is doing when calculating ANOVA results.

Question 8

Mean square and DFs in ANOVA

Answer 8

MSnumerator is for “effect” (factor under consideration)

dfnumerator is (a-1), where “a” is number of groups.

MSdenominator is for “error” (remaining variance)

dfdenominator is a × (n-1), where “n” is number in each group.

MSnumerator = SS / dfnumerator

MSdenominator = SS / dfdemoninator

F = MSnumerator/ MSdenominator
mean square for factor or effect under consideration.

F(dfn, dfd) = …

Question 9

The ANOVA table - need to be able to interpret

Answer 9

Typical ANOVA summary table from a computer program. Note, reporting result from experiment with 2 groups, and 6 subjects in each.

look at table in notes

F ratio = mean square of thing interested in / mean square of rest of variability

Question 10

reporting for the result

Answer 10

F(1,10) = 1.79, p > 0.05

F = type of statistic
1, 10 = both degrees of freedom
1.79 = value of statistic
p = significance level (p value)

Question 11

Relationship between ANOVA and t-test

Answer 11

• When an ANOVA has two conditions it is equivalent to a t-test. That is, the t-test is essentially a “subset” of ANOVA.
• But, how can this be given that, instead of t, an ANOVA produces F?
• Because - for a two-condition experiment, F = t2
  • And the resulting p value will be identical.
• So, in principle, if you have a 2-condition experiment you can report either ANOVA or a t-test. - can use ANOVA when over 2 conditions not a t-test

But, always look to see what you are asked to do in assignments etc.

Question 12

Additional notes between subjects ANOVA no1

Answer 12

• DO NOT need to be able to perform the variance calculations.
• DO need to be able to understand an ANOVA table and be able to report the results
  
  • Computing sessions will cover how to use SPSS etc. to do ANOVA.
• ANOVA “just” asks whether there are any differences between groups, it does not indicate what those differences might be.
  
  • Follow-up analyses are required for this, but that will be covered in year 2
    Like other tests, p-value is “probability of observed result IF the null hypothesis is assumed to be true”.

Question 13

Additional notes Between Subject ANOVA #2

Answer 13

• Have presented ANOVA here with same number of subjects per group. But between-subject ANOVA can be performed with unequal numbers per group (like the t-test).
• Like the t-test, ANOVA is “parametric”, and assumptions include: Data from Interval/ratio scales; Normal distributions; Variance equal between groups.
  
  • Actually, ANOVA is “robust” (i.e. p value probably not too wrong) if N is large & equal between groups even when assumptions broken. But still worth knowing about assumptions because “not too wrong” is still “not completely right”.
    So, ideally design experiments with equal numbers per group because it makes ANOVA less vulnerable to violation of assumptions.

Question 14

Within-Subject ANOVA

Answer 14

just like with the t-test, it is also possible to do ANOVA on within-subject data.
* Need to have data from each subject in every condition.
* Within-subject ANOVA will automatically exclude any subject that does not have data in every condition. So, design experiments to ensure there is data from every subject.
* There are alternatives to ANOVA that can cope with “missing data” – but we don’t cover them until 2nd year.
* Why do a different test for within-subject data?
Because this can “remove” variance due to the effect of “subject” from the MSerror – and so can improve power.

we are interested in the variance between groups

Question 15

examples of both within and between subject ANOVA in notes

Question 16

Additional notes Within Subject ANOVA

Answer 16

• Most notes on within-subject ANOVA apply here too.
• In particular, same assumptions as between-subject ANOVA, but “sphericity” assumption replaces the variance equal between groups (this assumption is covered next year).
• Typically, within-subject ANOVA more powerful than between (when subjects are generally consistent over repeated testing) – so can be a good design option to collect within-subject data.
  
  • But this is not always possible (imagine comparing mood as a function of nationality).
    Textbook goes into great detail on testing assumptions. Don’t get bogged down in this. But it is worth at least a “once-over” to get an idea of what can be done & why.

Question 17

One-way ANOVA summary

Answer 17

• Between-subject: Typical Application is controlled experiments with more than one condition (with each condition representing different levels of a single IV), each condition performed by different participants.
  
  • Like a “general” version of the between-subject t-test that can cope with more than 2 conditions.
• Within-subject: Typical Application is controlled experiments with more than one condition (with each condition representing different levels of a single IV), each performed by every participant.
  Like a “general” version of the within-subject t-test that can cope with more than 2 conditions.

Question 18

the ANOVA approach to repeated-measures design

Answer 18

Although you can conceptualize repeated measures as a linear model, there are other ways too. The way that people typically handle repeated measures in IBM SPSS Statistics is to use a repeated-measures ANOVA approach. Very crudely, this is a linear model like I have just described, but with some very restrictive constraints. In a nutshell, we have seen that the standard linear model assumes that residuals are independent (not related to each other – see Section 12.3), but as you can see from the models above, this assumption isn’t true for repeated-measures designs: the residuals are affected by both between-participant factors (which should be independent) and within-participant factors (which won’t be independent). There are (broadly) two solutions. One is to model this within-participant variability, which is what the models in the previous section do. The other is to apply additional assumptions that allow a simpler, less flexible model to be fit. The later approach is the one that is historically popular