Hypothesis Testing & Power Analyses Flashcards
Goal of a Hypothesis Test
To rule out chance as a plausible explanation for the results from a research study
Null Hypothesis (H0)
Assumes there is no treatment effect/difference between conditions/groups
First step in hypothesis testing
Alternative Hypothesis (H1)
What you are hoping to find
Hoping to be able to reject H0, in favour of H1
Statistical Inference procedure
- Make an assumption (H1)
1. State H0
2. Select statistical test and significance level
3. Select sample and collect the data
4. Find the region of rejection
5. Calculate the test statistic
6. Make the statistical decision
Central Limits Theorem
Distribution of means derived from sets of random samples taken from any population will trend toward normality
Greater the sample, greater the conformity to normal
p-value
The probability of observing a sample mean that is (at least) as far from the population mean as the value we have observed
Z-scores
0 represents mean of normal distribution
Each integer represents one SD away from mean
i.e. z = -1.5 corresponds to mean value minus 1.5*SD
Alpha level
Probability threshold, determined by what amount of risk (of Type II error) we deem acceptable
p-value below the threshold -> reject H0
Chosen prior to the experiment
Typically 0.05
Type I and Type II errors
Type I: incorrectly rejecting H0 (false positive)
Type II: incorrectly failing to reject H0 (false negative)
One-tailed vs Two-tailed tests
Depends on whether or not it matters which direction the effect will be in
Two-tailed test will produce a bigger p-value
e.g. if alpha = 0.05, then 0.025 is assigned to each side
Critical Value
Z-value defining the rejection region
Function of the chosen alpha level
Cohen’s d
Standardised measure of effect size:
Distance between two means, measured in SDs
Power
Probability of being able to reject H0 when it would be correct to do so
Depends on:
1) alpha level
2) (predicted) effect size
3) sample size
0.8 usually desired
Power Analysis
Calculating how big the sample needs to be for desired amount of power
e.g. small predicted effect + small sample –> low power
Solution? Increase sample size