Week 3: ANOVA assumptions, power and mean comparison Flashcards by Alicia Lowe

what is the underlying ANOVA equation?

Score = Grand mean + treatment effect + residual error (measurement and individual differences)

How well did you know this?

Not at all

Perfectly

Essentially, how does ANOVA break down variance? and what does the comparison of this breakdown produce?

Into 2 parts

Variance due to treatment
Error variance

Comparison of these two variances produces the F STATISTIC

How well did you know this?

Not at all

Perfectly

What are the assumptions of a between ANOVA?

Homogeneity of variance: SD for all groups is about the same
Normality: error is normally distributed
Independence of observation: truly between design

How well did you know this?

Not at all

Perfectly

How do you test the assumption of homogeneity of variance?

Levenes test
- if statistically significant the null is rejected. this means that there is a difference somewhere do homogeneity is violated

How well did you know this?

Not at all

Perfectly

What do you do if Levenes test is significant?

You can look up density plots to find the culprit

can be violated without grave consequences as long as sample sizes are equal

How well did you know this?

Not at all

Perfectly

What can you use to test the assumption of normality?

Shapiro WIlk
If significant, assumption of normality is violated

can also look at histograms, distribution plots, skewness and kurtosis

How well did you know this?

Not at all

Perfectly

What happens if normality is violated?

Typically its not a big issue

ANOVA still tends to be robust in terms of normality violations

How well did you know this?

Not at all

Perfectly

How do you report that you have used a non-parametric version of an ANOVA?

‘Similar results were found using non-parametric….’

How well did you know this?

Not at all

Perfectly

What are quantile-quantile plots?

They look at normality assumption.

Chop data up into how many scores are in each quantile and compare it against what we would expect in that quantile for a normal distribution.

Straight line = perfect
Deviations = bad

How well did you know this?

Not at all

Perfectly

What happens to the ANOVA if assumptions are violated?

If you arent confident, can use non-parametric versions of the test
This can then increase confidence in results

How well did you know this?

Not at all

Perfectly

What is skewness of data?

Deviations from typical bell curve of data distribution

- asymmetrical

How well did you know this?

Not at all

Perfectly

Explain positive and negative data skewness

Negative skew: Have a long tail towards low values in the data (more low scores than expected)

Positive skew: long tail towards high values (more high scores than expected)

How well did you know this?

Not at all

Perfectly

No skew value?

0 - perfectly symmetrical

How well did you know this?

Not at all

Perfectly

What values are considered moderately skewed?

Between -1 and -0.5
or
Between 0.5 and 1

How well did you know this?

Not at all

Perfectly

What values are considered highly skewed?

(-/+) 1+

How well did you know this?

Not at all

Perfectly

Skewness in Quantile-Quantile plots?

If data drops below the line, it is skewed to the left or negatively skewed

If data rises above the line, it is skewed to the right or is positively skewed

How well did you know this?

Not at all

Perfectly

What is kurtosis?

It is a measure of tailedness in data distribution curve

how light or heavy-tailed the data is

What is leptokurtic distribution (kurtosis)?

People scoring closer to the average (light tailed)

What is platykurtic distribution (kurtosis)?

Wider spread of scores so maybe more heavy tailed

What is the optimal kurtosis value?

It is reported in terms of how much excess skew there is - so the optimal would be 0!!!

How do you know when to reject normality in terms of kurtosis?

You are given a standard error as well as a skewness value - these can be used to determine a z-score

z score = Value of kurtosis/standard error

If z score is less than -2 or more than +2, reject normality

What is power?

The probability of correctly rejecting the null

finding a difference between means if it is there

What is power associated with?

Type 2 errors

when do type 2 errors typically occur?

when alpha level is too strict
or
outlier increases error variance so hard to see treatment effect clearly

What is the power equation?

power = 1-B The probability of finding a real difference = 1 - the probability of not finding a real difference

What increases power?

- Increase in magnitude of difference between means (effect size): makes it more obvious - Alpha level (more generous): increases power but also increases type 1 errors or false positives - variance in scores: less variance increases power as if too much variance, makes it harder to see effects clearly - larger sample size: gives more accurate means - more power as can see more clearly

A priori power estimate?

Estimate before the experiment is run | - helps to ensure have adequate numbers of participants to detect any real differences between treatments

Post hoc power estimate?

Estimate after data has been gathered Estimates the likelihood of being able to replicate a significant difference if repeated - value between 0 and 1 eg 0.82 - 82%

How can you obtain a post hoc power estimate?

G*power - REFER TO SLIDES Can provide you with a power estimate if you feed it means, average variance (MSE) and sample size - Tell it what kind of test you've done - gives an effect size that can then be used to find power

How do we find a priori power estimates?

Estimation of effect size (cohens d or f) - or you can use the effect size from other studies With this effect size you can compute a non-centrality parameter (used to find the power of an experiment for any effect size) Can then plan sample size etc (can use g*power again)

What happens when a statistically significant comparison has more than 2 means?

Need further tests to determine which of the means actually differ from each other - typically multiple comparisons

What are the two approaches to follow up tests?

A priori: chosen before data is collected | Post hoc: planned after data is examined

Errors in multiple comparison tests?

Major type 1 error rate risks

Per comparison error rates vs. family-wise error rates?

Per comparison: the probability of making a T1 error on any comparison - this is alpha Family-wise: the probability that a family of comparisons will contain at least 1 T1 error - per comparison rate stacks up - error rate per comparison X number of comparison

A priori comparisons?

Usually when have very specific hypothesis - creates a smaller number of comparisons so T1 probability is reduced - contrasts

How to do a priori comparisons in jamovi?

These are 'constrasts' and this is a tab within jamovi under the analysis set up

What are the different types of contrasts?

Deviation - each level to grand mean Simple - mean of one to each other one Difference - mean of level to previous Helmert - mean of level to all subsequent combined Repeated - mean of one to each subsequent individually

T1 error rates in contrast tests?

Need to do bonferonni adjustments | alpha/number of comparisons

Critics to bonferonni?

Claim it may be too conservative and we may make T2 errors instead and miss actual effects

What can we do here if we are worried that bonferonni is too conservative?

Linear step up!

What is linear step up?

Focuses more on false discovery rate (FDR) Rank comparisons according to their p values - divide this rank by the number of comparisons Then multiply this by FDR which is alpha Gives you a critical p value to determine significance against

What other post hoc tests may be appropriate when comparing means?

Tukey or Holm