Week 12 - ANOVA Flashcards
WHY are ANOVA tests done?
* ANOVA, which stands for Analysis of Variance, is a statistical test used to compare the means of three or more groups to determine if there are statistically significant differences among them. It allows researchers to examine the effect of one categorical independent variable (also known as a factor) on a continuous dependent variable.
- The primary purpose of conducting an ANOVA test is to determine whether the means of the groups are different enough to suggest that the variation in the dependent variable is not just due to random chance but rather due to the influence of the independent variable. ANOVA helps researchers understand whether there are meaningful differences between the groups and if the independent variable has a significant effect on the dependent variable.
- When ANOVA produces a significant result (meaning there are statistically significant differences among the group means), it tells us that at least one of the groups is different from the others. However, ANOVA does not tell us which specific groups are different from each other. To identify the specific group differences, researchers usually conduct post-hoc tests or planned comparisons.
*There are several software packages and statistical tools available that can automatically perform ANOVA calculations and provide the results. Some common software used for statistical analysis, including ANOVA, are:
- SPSS (Statistical Package for the Social Sciences)
- SAS (Statistical Analysis System)
- R(open-source programming language and software environment for statistical computing and graphics)
- Python (with libraries such as SciPy and StatsModels)
- Microsoft Excel (with the Data Analysis Toolpak)
How to calculate probability of getting a Type 1 error:
Experiment - students unscrambled anagrams to make words, to test if the colour red affected performance.
Probability of getting it right? 0.95 x 0.96 x 0.95 = 0.857
Probability of getting it wrong and getting a Type 1 error? 1- 0.857 = 0.143
TYPE 1 & TYPE 2 errors table:
Orange : Null Hypothesis states that all ‘means’ are the same
Green: 2 means are different (Ha)
Blue: ANOVA can test all means. to infinity.
What about the t-test:
the red group has 3.5 mean and the black group has a 5 mean.
T-test looking at how different the means are together with the amount of variability and far apart the means are from each other. And Calculate ‘s’, too. And pooled variance.
Can do the same with ANOVA’s (as the t-test):
Between-Group Variance: how far apart the means sit from each other – 3.5 to 4.5 to 5.5.
- Systematic Variance = the difference between the groups - the amount of variability due to our manipulation OR otherwise known as the independent variable. It’ systematic because we know the dependent variable is causing the variability.
Need to divide this by some variability and we can’t use each groups standard deviation, so need to do an average of the 3 groups variability:
Within-group Variance- the ‘spread’ of any one of the invidividual group distributions)…
* aka. the error variance or variability due to random fluctuations (we differ from day-to-day or we differ slightly from one another on task – it’s the variance that is random and can’t be explained by the manipulation of the indep variable.
z ,
WER
Less Likely to Reject Ho – the variability on these 3 distributions are smaller. And they overlap a lot more.
Likely to reject Ho – due to the systematic variance being bigger than the random variance (or random error – the differences within each group)
in these distribution, Likely to Reject the Ho – due to the red being an outlier and making the sytemic variance larger.
If the ratio is 1, then the between-group difference is the same as the within-group difference.
If the systemic difference is no different than the error variance, then it’s likely any difference between our groups is due to random error. This suggests that the Ho is true and that the groups don’t really differ.
But if the between-group variance is much larger than the within-group variance, then the ratio is a lot larger than one, then it’s inferred that the sample means are different from one another.
F-distribution table is now used to determine if the ratio is big enough to reject Ho.
How do we know what a big enough ratio is though? (after doing these calculations)?
Use the F-distribution table values: it looks like the graph below. A zero sits at the beginning and a value of 1 sits lined up with the peak.
Remember that F-distribution is based on the Ho (no difference between our groups or that the ratio of this systematic:error variance is going to be 1 (no difference)
Note that because it is a ratio, it uses variances, the value is always positive
Because it’s based on the Ho that there’s no difference between groups, the likelihood of getting a larger ratio (systematic : error variance) is small.
And we can quantify the F-statistic that represents a 5% probability for example, of having found that particular ratio by chance if the Ho were true (that they don’t differ)
The probabilities of the F-distribution are the square of the t-distribution for 2 samples (but is used for more than 2 samples)
Some researchers will use ANOVA when they only have 2 samples because it works just as well.
Why is it the square of the t-distribution? For the t-distributions, we use standard deviation. For F-distributions, we use variances — which are squares of the standard deviation.
So if we took the square root of the F-stat for 2 samples, it would match the t-stat exactly.
Recall a previous example where the between-group variance was not that difference from the systematic variance, the F-stat would sit at the blue star.
This indicates the ratio isn’t very extreme and the groups don’t differ from one another.
In this 2nd example, the between-group variance was a lot more than the within-group variance. Now the ratio is bigger or more extreme and the blue star will sit at the tail of the distribution. More likely to Reject Ho.
In the top distribution, the means are closer together than the bottom distribution (aka. less between-group difference in the top and larger between-group difference in the bottom) So a larger ratio for the bottom image.
In the 3rd distribution, the variability within each distribution is less. Less spread, smaller denominator, so larger ratio.
ASSUMPTIONS:
- Random selection
- Pop distribution is normal
- Homoscedasticity (homogeneity of variances)
A)
Next video:
Just to review the test that was done
STEP 1:
assumptions
STEP 2:STATE THE HULL and ALTERNATIVE
- State the Ha = at least 2 of the means are different
- State the Ho = All the means are the same, Ho: mu1 = mu2 = mu3
There is no 1-tail or 2-tail differentiation.
STEP 3:
- Comparison distribution based on Ho
- the between-group variance & within-group variance will be the same (or have a ratio of one)
- ^Expressed by the F-distributions
STEP #3:
the degrees of freedom
STEP #4:
Alpha = 0.1 = 1%
df-between = is at the top of the F-table
df-within = is at the side of the F-table
CRITICAL VALUE = 6.52
STEP #5 : THE LONGEST STEPS
Calculate the F-statistic by calculating a whole bunch of stuff first.
SS - between group
SS - within group
STEP 5: FINALLY, the F-statistic!
C) calculation error!
The F-statistic can’t be NEGATIVE
STEP 6: DECISION TIME
WITH CORRECT SYNTAX
D)
POST-HOC TESTS (3RD VIDEO)
R^2 = SS_between / SS_ total
- R2 represents the proportion of variance in the dependent variable that is accounted for by the independent variables in the ANOVA model. It measures the strength of the relationship between the independent variables and the dependent variable.
- R2 ranges from 0 to 1, with higher values indicating a larger effect size and a stronger relationship between the independent variables and the dependent variable.
- it is a ratio of the Between-groups SS to Total SS.
When R^2 is TOO LARGE, then w^2 can be used as the test of variance.
WILL MAKE EFFECT SIZE SMALLER THAN r^2.
POST-HOC TESTS: do this follow-up test when Reject the Ho. Do this as a follow up with a statiscally significant ANOVA.
- if we Fail to Reject the Ho, there’s no reason to do PostHoc tests (because we say they are all the same)
- TUKEY’S HSD (Honestly Significantly Different) = adjusting for multiple comparisons while using the test
- Need to find the standard error (denominator)
- BUT IF ‘n’ is unequal (like there’s aren’t perfectly 10 ppl in each of the 3 groups, let’s say, instead, there 5 ppl in one group, 7 in the second, etc. then need to us the ‘HARMONIC MEAN’ EQUATION(n’)
- Bonferroni
The Harmonic Mean: used if there are unequal groups
SO, overall for the Tukey’s HSD formula, either n OR n’ is used to calculate the standard error (Sm) in the denominator:
So, now the Sm (standard error with either n or n’) is calculated.
NOW, calculate the HSD ….FOR EACH GROUP.
NOW, FIND THE ‘critical value’ in the q-table.
Across: the # of groups
Down: df_within is down
Tukey’s HSD post-hoc tests are ALWAYS 2-tailed.
WHY?
- the Ha states that at least 2 of the means are different and it doesn’t specify which direction, so the post-hoc needs to test for both.
Q: is the HSD stat more extreme than the critical value?
YES, -6.67 is sitting very far to the LEFT SIDE.(red vs black)
YES, -7.11 is sitting very far to the LEFT SIDE. (red vs. green)
NO, 0.45 is sitting in the middle of the distribution. (black vs. green)
WHAT DOES THIS MEAN?
- people solved less puzzles when given red ink insteads of black or green ink.
- red DOES affect solving puzzles.
Bonferroni Post-Hoc Test:
a) because it’s HSD value that we want to compare to the critical value
ONE-WAY WITHIN-GROUPS ANOVA:
THE EXPERIMENT:
- exact same ppl in both conditions.
- same ppl viewed the negative, neutral, and positive images.
- there are still individual differences, such as client #2 below seems to have poorer memory than client #1 and #2.
- measure of variabilty is done because ppl are different and have different abilities anyways
- if the within-group variability is smaller, than test statistic will be larger, making it more likely we’ll reject the Ho
- this type of test is more sensitive to detecting differences in the conditions due to the indep variable – because a measure of variability is being removed.
WHY use any other test besides the 1-Way Within-Group Design if it’s so good?
Because of ‘order-effects’. Can be hard to counter-balance.
- research suggests as we age, we focus more on the positive stuff, called ‘positivity bias’.
- don’t want to generalize from our small college sample to the whole population because we need to look at more than uni students.
B) participants. BECAUSE the participants are the same for the before and after variables…!
