ANOVA Flashcards
When do we use ANOVA?
When we are comparing more than 2 groups
How do we calculate a one way BS ANOVA? (Not formula)
- Calculate within group variance (experimental error)
- Calculate between group variation (treatment effect)
- Then we can work out the F ratio which tells us if we have a significant TE
How do you calculate variance?
∑(Y - MeanY)^2/n-1
How do we calculate F?
F = TE + EE/EE
F = MS factor / MS error
What F would we expect if there is no TE?
F = 1 as… F = 0 + EE/EE
This is not possible however due to chance factors varying
How do we decide whether to accept or reject H0?
Sampling distribution - larger F ratio = less frequent it would occur if H0 were true
Probability value of 0.05 suggests that the F value occurs less than 5% of the time when H0 is true so its unlikely to occur and we reject H0
This probability error is referred to as alpha level and defines the risk of a type 1 error
What is experimental error?
A combination of individual differences, researcher error and chance factors
What are the assumptions of one way BS ANOVA?
Assumption of normality - dv is normally distributed (check on SPSS by splitting file and producing histograms showing normal curve)
Assumption of independence - one score is in no way related to any other score in any group
Assumption of equal variance - EE is approximately equal in each group (check using Fmax test)
What is the Fmax test?
Used to test assumption of equal variance is met in 1-way BS ANOVA
Fmax = largest variance / smallest variance
> 3 then alpha level must be changed from 5% to 2.5%
How do you report an ANOVA?
F(df for effect, df for error) = F value, probability level, mean square error
What is a WS (repeated measure) ANOVA?
An ANOVA where the same participant serves in each condition
What is the benefit of a WS design?
Minimises error variance - eliminated ID
SSresidual makes F ratio more sensitive so a significant difference is more likely to be found
What is SStotal made up of?
SSeffect = differences among means
SSerror = SSsubject (ID) and SSresidual (other error)
How are individual differences removed from WS design?
You subtract their mean performance from their scores in each condition which will give you a score that was a deviation from their usual performance i.e independent of the individual difference variable
Relationship between data remains the same
What are the assumptions of 1-way WS ANOVA?
Sphericity assumption - correlations between treatment conditions scores are equal (Mauchley’s test, if significant you have violated the assumption you need to use Greenhouse-Geisser or Huynh-Feldt to make F test more conservative using fewer df to calculate MS)
Homogeneity of covariance - variances of the differences between all combinations of pairs of conditions are equal
What does post-hoc testing show?
A significant F value informs you a significant difference exists among treatment groups post hoc is needed to determine which of the groups differ significantly
What are the types of post hoc comparisons?
Pairwise - 1 v 2, 2 v 3 and 1 v 3
Complex - av (1,2) v 3, av (2,3) v 1 and av (1,3) v 2
Why is post hoc testing better than running multiple ANOVAs?
Alpha level represents the possibility of making a type 1 error per comparison (alpha pc)
Post hoc is interested in a family of comparisons (alpha fw) which is the number of comparisons X alpha pc = 0.5 X n
Post hoc testing corrects this problem so alpha fw = alpha pc by reducing alpha level or increasing critical value of F
What are the types of post hoc?
Tukey (Ft) - only pairwise
Scheffe (Fs) - all comparisons
Dunnett (Fd) - control to each experimental group
Bonferroni (Fb) - adjust probability so its alpha/number of comparisons
What are the steps of post hoc?
- Calculate F comp for each comparison
- Calculate modified F value (Ft/s/d)
- If Fcomp > Ft/s/d then there is a significant difference
- Reject H0
What is a factorial design?
An ANOVA with more than 1 IV
What are the advantages of a factorial design?
Economy - more info for same amount of work
Experimental control and increased generality of results - by including a possible extraneous factor as an IV
Interaction - effect on IC rarely occurs in isolation
What are the F ratios calculated in a factorial design?
F(main effect A) = variability due to A / within V
F(main effect B) = variability due to B / within V
F(interaction of A X B) = variability due to interaction of A X B / within V
How to calculate main effect?
To find out separate effect of each IV on DV
You average scores and compare for each factor e.g average scores for a1 and a2
How can you visually tell if there is an interaction?
If the patterns of data are different
What is a mixed ANOVA model?
This type of design involves 2+ IVs where some are WS and some BS
What are the two perspectives variability can be examined from?
- Systemic (controlled) v unsystemic (error variance)
2. Whether source of variance is BS or WS
What is the difference between how F ratios are calculated in mixed ANOVA designs?
They are evaluated against the error term which reflects that component of study
E.g if A was BS and B WS —> Fa would use BS error term and Fb and Fab would use WS error
What does a significant interaction mean for main effects?
They may be misleading
What are the follow up tests for factorial and mixed ANOVAs?
Keppel’s follow up strategy:
If there is a significant interaction —> do simple effects test followed by simple comparisons (error term differs depending on simple effects you decide to do if its a mixed design)
If there is not a significant interaction —> do main comparisons on significant factor
What are simple effects tests?
Investigate the effect of 1 IV on DV at each level of the other IV
Used to understand nature of interaction
Can follow two strategies
ANOVA’s using pooled error term to calculate F
Why do you use the pooled error term to calculate F in simple effects and comparisons?
Based on all the data
Better estimate with more df
Captures total variance in data
What are the differences in follow up tests for mixed ANOVA?
Two strategies differ:
> for the effect of the WS IV on DV at each level of group (BS IV) = RM ANOVA using pooled error term and WS error
> for effect of the BS IV on DV at each level of WS IV = BS ANOVA using pooled error term AND within cell error (calculate yourself and is the average of variances from all cells in the design)
What is an effect size?
A measure of the magnitude of the experimental effect and the strength of association between 2 or more variables.
When IV’s have > 2 levels (ANOVAs) or are continuous, effect size estimates usually describe the proportion of variability accounted for by each IV
Why do we need effect sizes?
Meta-analyses
Compare similar studies with diff designs
Practical importance
Calculate power and estimate required sample size
Why is effect size more reliable than F?
It is independent of sample size so we can compare across studies
What are the families of effect sizes?
D family - standardised differences between means
R family - variations on the correlation coefficient
How is effect size calculated in general?
ES = SS effect / SS total
What are the different effect sizes?
Eta squared (n^2)
Partial eta squared (np^2)
Omega squared (w^2)
Partial omega squared (wp^2)
Eta squared
Proportion of the total variance in the DV that is attributed to an effect
n^2 = SS effect / SS total
Problem: the values for an effect are dependent on the number of other factors in the ANOVA and their magnitude
Partial eta squared
Resolves the n^2 problem by examining the effect of a given IV relative to only two variable components
In the factorial model rather than the total variance: that due to IV effect and that due to error variance
When there is 1 IV n^2 = np^2 but when there are more n^2 < np^2
np^2 = SS effect / SS effect + SS error
Omega squared
Is used to obtain a relatively unbiased estimate of the variance explained in the population by an effect
Unlike n^2 it takes random error into account (MS error)
w^2 = SS effect - (df effect X MS error) / SS total + MS error
w^2 < n^2 values
Partial omega squared
Similarly to partial eta squared it represents an unbiased stigmata of rage population proportion of variance in DV associated with an effect, after variability associated with all other effects has been removed from consideration
wp^2 = SS effect - (df effect X MS error) / SS effect + (N - df effect) X MS error
How do you interpret effect size?
Small > 0.01
Medium > 0.06
Large > 0.15 (0.14 cohen)
Why do we conduct significant testing?
To know if we should accept or reject H0
Significant testing errors?
Type 1 = rejecting the null hypothesis when it is true (alpha level indicated probability of this error)
Type 2 error = failing to reject the null hypothesis when it is false (there is an effect) (beta level (power 1-B) indicates probability of making this mistake)
What happens if you decrease alpha?
Power (1-B) decreases as B increases
What is power?
Ability to detect a difference when it exists
How can you increase power?
Decrease EE
Use more sensitive design (WS)
Increase sample size
Increase TE
What is the minimum power should be?
80% - Cohen (1977)
Balances a and B risk as type 1 error is more serious than type 2
a = 0.05, B = 0.2 and power = 0.8
What are the dangers of underpowered studies?
Sig effects not found when they do exist
Non-significant effects cant be interpreted
Inconsistent results
Problems for replication studies
What do you need to calculate power?
Sample size
Effect size (expected)
Alpha
Use SPSS
