Effect Size, Power, & ANOVA Flashcards
Effect Size
Cohens d
How big the effect is
Used to compute power
Small (.2)
Med (.5)
Large (.8)
Type 1 Error
Reject the null hypothesis when the null is actually true (alpha)
Threshold for alpha is usually .05
Type 2 Error
Accept the null hypothesis when the null is actually false (beta)
Threshold is usually .2
Willing to miss real findings 20%
Means power is .8 (80%) - odds of finding an effect if one exists
Power
Power depends on the degree of overlap between the sampling distributions
Power is the probability of correctly rejecting the null hypothesis if the null is indeed false
Power is the probability of correctly rejecting the null hypothesis if the null is indeed false
Power for Independent t-Test
Little n is number of participants in each group, not total N
Take the average of the two or the lower value, .71 or .725
Using alpha of .01 instead of .05, then power would decrease (the more conservative we are with type 1, the less chance of discovering an effect)
Convention is 80% power
Power for Dependent t-Test
Delta is used to get to power (is not the same as power)
Look at the table to find the delta value
Then determine what the power is
Usually use alpha 0.05
ANOVA
2+ sample means
E.g. placebo vs. drug 1 vs. drug 2 vs. drug 3
ANOVA Hypotheses
Null hypothesis H0: population means are equal
Alternative hypothesis H1: not all population means are equal - at least two of the means are different
ANOVA doesn’t tell you where the differences are
ANOVA Assumptions
Observations normally distributed within each population
Population variances are equal (homogeneity of variance or homoscedasticity)
Observations are independent
ANOVA is usually robust against violation of first two assumptions
Logic of ANOVA
As you increase sample size, it becomes more difficult to get a deviant mean
Everything is squared in anova because it is an analysis of variance
ANOVA Equation Variables
MS: mean square groups
MS: mean square error
MS error: pooled within group variance
Sample Size & Power
Bigger sample = more power
Less chance of making an error
Greater sample means you more accurately gage the universe, better reflect what’s out there
Relationship Between Alpha & Beta
Direct inverse relationship between alpha and beta
If you lower alpha threshold, then you raise your beta the same amount
As you make it harder to have a false positive, you make it easier to fail to find something that’s really there
Experiment/Family Wise Alpha
Odds of making at least one type 1 error in a set of comparisons
c = number of comparisons
Odds of making at least one type 1 error = 26.5%
