Chapter 10: Null Hypothesis Significance Testing Flashcards
Why are p-values impacted by sample size? Provide a formulaic explanation.
Confidence intervals use the formula, CI = Mean ± 1.96(SE).
This contains standard error which uses the formula, SE = STD/SRT(N).
In the SE formula, as N gets larger, SE gets smaller. When SE gets smaller, so do the confidence intervals.
With small confidence intervals the probability of overlap decreases drastically, leading to an overestimation of significant difference.
This is why a large p-value can lead to unimportant, significant results.
Define Type II error and state the notation that typically describes it.
Failing to reject a null hypothesis given sufficient evidence for the alternative.
Notated by β or beta.
Describe the relationship between α and β.
“Although we know that as the probability of making a Type I error decreases, the probability of making a Type II error increases, the exact nature of the relationship is usually left for the researcher to make an educated guess.”
What is the formula for a confidence interval?
CI = Mean ± 1.96(SE)
What is the formula for standard error?
SE = STD/SRT(N)
How can you mathematically calculate statistical power?
1-β
Power is the probability that you will find an effect if it exists.
How do you correct for the family-wise error (two solutions from across the course)?
Post Hoc Testing: Use the Bonferroni correction.
Planned Contrasts: Using a dummy coding scheme to keep your contrasts independent of each other (not using the same data for each test).
What must we do when testing a non-directional hypothesis?
Divide our p-value by the two sides that we are testing.
What are the steps for calculating moderate overlap?
Use these two ranges for your calculation.
55-75 and 67-83.
1) Determine the distance in each range: 20 and 16.
2) Divide those distances by two: 10 and 8. These are the margin of error (MOE)
3) Add those together and divide by two: (10+8)/2 = 9. This is the average MOE.
4) Take the average MOE and divide it by two: 9/2 = 4.5. This is your moderate overlap.
5) Check if your confidence intervals overlap by more than this value. If so, your data are not significant.
Why do scientists choose to use a p-value of .05 in research?
No reason. This was an arbitrary number assigned by Fisher.
Calculate the probability of making a family-wise error while running seven independent tests. Also, what is a family-wise error?
Formula: 1-0.95^n
Answer: 1-0.95^7 = 0.30
There is roughly a ~30% chance of making a Type I error when running 7 independent tests on the same data.
How do we mathematically conceptualize a test statistic?
Effect / Error
OR
Parameter estimate / Standard error
Define Type I error and state the notation that typically describes it.
Falsely rejecting the null hypothesis.
Notated by α or alpha.
Statistically speaking, what is moderate overlap, and what does it tell us?
“It can be defined as half the length of the average margin of error (MOE).”
When two confidence intervals have moderate overlap then your p-value is greater than .05.
What is the interaction between a Bonferonni correction and statistical power?
Bonferonni corrections lower α , which lowers the tests statistical power.
