Statistical theory 3 Flashcards
What is null hypothesis significance testing called for frequentists?
orthodox hypothesis testing
or, null hypothesis significance testing (NHST)
What was Fisher’s thoughts on hypothesis testing?
Hypothesis testing is about trying to falsify a single hypothesis
What was Neyman’s theory on hypothesis testing?
Hypothesis testing is about choosing between two rival hypotheses
What does the null hypothesis predict?
There will be no effect
In null hypothesis testing, what is H1?
The alternative hypothesis
This one is looking for some effect (usually the one they want to find)
Do we always test the hypothesis or null hypothesis?
Null hypothesis
What is a statistical hypothesis?
Claims about population parameters
“A critical part of designing a study is being able to specify how your 1)_____ hypothesis (what you’re interested in) maps onto your 2)______ hypothesis (what you can actually test)”
1) Research
2) Statistical
How do we evaluate whether the null or alternative hypothesis is accurate?
By determining how likely our data would be if the null is true.
If they are unlikely ‘enough’, we reject the null
In hypothesis testing, what is a Type 1 error?
False positive
In hypothesis testing, what is a Type 2 error?
False negative
What type of error rate does null hypothesis testing control for?
Type 1 error rate
Why is null hypothesis significance testing designed the way it is?
To control for belief bias (avoid type 1 errors)
Forces a minimum evidentiary standard on you
What is the significance level?
If we would expect to see our data n% of the time (or less) if the null were true, we reject the null
The choice for n is arbitrary and is known as the significance level
What do you need to build your own statistical test?
Important for all types of statistical tests
A diagnostic test statistic T
Sampling distribution of T if the null is true
The observed T in your data
A rule that maps every value of T onto a decision (accept or reject H0)
What is a diagnostic test statistic (T )?
A single number that you can calculate only from your observations
As long as it’s just one number, it’s a test statistic
What are these all examples of?
- The mean of a set of observations*
- The standard deviation of a set of observations*
- The third-largest of a set of observations*
- A number of observations*
Diagnostic test statistics
Is the two largest numbers in the sample an example of a test statistic?
No
(multiple numbers)
How do you calculate sampling distribution if the null is true?
1st. Assume H0 is true
Figure out what the values of your test statistic you should expect
A test statistic is diagnostic if the null hypothesis and alternative predict _____ values
different
- e.g. Baby v cat cuteness: number of baby choices out of 100*
- H0: Should be around 50*
- H1: Should* not be around 50
In a binomial test statistic, when should we reject the null?
When 95% of the data accepts the H0 and the results lie outside of that.
Where is the ‘rejection region’ for null hypothesis testing?
Outside of the 95% probability range.
95% probability means that if the null hypothesis is true, there is a 5% chance of _____ rejecting it.
falsely
What is a ‘two-tailed’ test?
A directionless test
When the rejection region for the null hypothesis lies on both sides of the distribution