PSCH 443 - Final Flashcards
Basic Logic of NHST
- Belongs to null hypothesis significance testing - evaluates the probability of observing the data under the assumption that the null hypothesis is true.
- If we assume the null is true, we can generate a sampling distribution that characterizes the distribution of the sampling mean we expect to observe.
- By looking at the mean and the expected total variance, we can guesstimate the difference b/w an observed sample mean and the population mean when a sample error is made.
- We can use this difference to determine the likelihood of seeing an actual effect versus any difference being the result of sampling error alone.
Grand Mean (GM)
The combined mean of all the different means for each group or condition used in the ANOVA.
- We can take the average squared deviation of all the means
- Estimates the population variance
- Estimates the expected distribution over an infinite number of samples
Post Hoc Tests
- Used when we do not have a theoretical basis to expect any particular differences b/w groups or conditions
- More conservative measures of differences b/c they are not guided by theory
- Used when there is a significant omnibus F stat, but no specific differences b/w groups were originally predicted
Two categories of Post Hoc Tests
Fall into 2 broad categories:
- Adjusting type I error rate to accommodate multiple comparisons
- Calculating new and more conservative test statistic
When using Bonferroni correction:
- Calculate a new alpha
2. Takes the desired level of family wise error for an experiment (i.e., 0.05) and divides it by the # of comparisons
When using a Tukey HSD we:
- Calculate a new test statistic that represents the mean difference that must be reached in a comparison to be statistically significant
- Assumes we want to compare all means
- Uses 0.05 as an arbitrary cut-off
Planned Comparisons
- Planned b/c they should be guided by theory; # of planned comparisons is generally small relative to the # of conditions b/c this reduces family wise error by default
- Tests are only made b/w a few groups that have key differences as opposed to there being several tests across several conditions
- Uses the error term from the omnibus f test, or the Within Groups Mean Squares
The two types of planned comparisons:
2 types:
Pairwise – analyze simple differences b/w 2 means
Complex – analyze the difference b/w sets of means
What do we do in complex comparisons:
In complex comparisons, we need to come up w/ contrast weights. Contrast weights are sample means weighted by a coefficient
- Choose sensible comparisons
- Groups with positive weights will be compared to those with negative weights.
- The sum of the weights should always be zero.
- Groups not involved in a comparison always get a coefficient equal to zero
Family-wise Type I Error
Inflated probability of making a type I error based on greater # of tests performed
- Reflects that multiple tests are independent and have their own unique probability of committing a type I error (or of incorrectly rejecting the null)
- Sums the total of all tests performed
- Subtract them from 1 to standardize the probability of committing a type I error (or of incorrectly rejecting the null)
ANOVA as Regression
Can be understood as:
Systematic Variation + Unsystematic Error / Unsystematic Error
- Both try to explain variability although model estimation is different
- Focuses on categorical variables
- If mean differences are larger than what we expect due to chance (error), the value of the F stat should increase
- The systematic variance that our model explains = the effect of our IV on our DV
Partition of ANOVA
- SSt represents the overall variability we are trying to explain
- Partitioned into SSm (variance accounted for) and SSr (variance unaccounted for)
- Unsystematic variance cannot be explained for in any meaningful way using ANOVA models
Dummy Coding ANOVA for a regression analysis
- We enter all dummy codes in one block
- Comparison group is given a value of 0
- Other groups are given a value of 1 in each row
Orthogonal comparisons
- Info given by the comparisons is independent of other comparisons ran on the data
- Sum of weighted comparisons has to be equal to 0 to maintain independence
- Does not inflate familywise error b/c outcomes are treated independently so no test type I error probabilities are overlapping
Assumptions of ANOVA
- Normality
- The distributions of the residuals are normal - Homogeneity of variance
- Variances should be roughly equal across groups - Independence of observations
- The error term is the same across all values of the independent variables
- Spread is roughly the same across levels, so there is about equal random error
Eta-squared
The most generally accepted measure of effect size/statistical power
- Will be = to R2 in one-way ANOVA
- tends to overestimate the effect size in the population
- the inverse of type II error, or the likelihood that we will detect a significant effect when none exists
- smaller range for effect size = better chance of detecting the effect
Factorial ANOVA
Research designs that have more than one IV