9 Hypothesis Testing Flashcards
What is hypothesis testing?
Hypothesis testing is trying to guess whether or not your hypothesis is true.
What are the two main types of hypotheses in hypothesis testing?
Null hypothesis and alternative hypothesis.
What does the null hypothesis (π»π»0) state?
There is no difference between the two groups.
What does the alternative hypothesis (π»π»1) state?
There is a statistically significant difference between the two groups.
What is the purpose of hypothesis testing?
To determine if the perceived difference between groups is due to variance or if it is statistically significant.
What is statistical significance?
The ability to say, with a very small margin of error, that there is an actual difference beyond pure chance between two groups.
List the general steps of the hypothesis testing process.
- Ask or receive a question
- Create a hypothesis
- Decide what analysis is appropriate
- Gather resources for analysis
- Run the analysis
- Answer the question
True or False: In hypothesis testing, both the null and alternative hypotheses can be true at the same time.
False.
What is a p-value?
A p-value is the probability that the difference between the groups is due to chance.
What does a p-value of 0.05 indicate?
There is a 5% chance that the observed difference is just random.
What is alpha in hypothesis testing?
Alpha, or the significance level, determines the cutoff for statistical significance.
If p-value < alpha, what decision do you make regarding the hypotheses?
Accept π»π»1 and reject π»π»0.
What does it mean when p-value > alpha?
Accept π»π»0 and reject π»π»1.
Fill in the blank: The null hypothesis states there is ______ significant difference.
NO
Fill in the blank: The alternative hypothesis states there ______ significant difference.
IS
What is the first step in the hypothesis testing process?
Ask or receive a question.
What is the significance of a larger sample size in hypothesis testing?
It can provide a more reliable estimate of the difference between groups.
What is the relationship between the null hypothesis and alternative hypothesis?
They are mutually exclusive; one is always true while the other is false.
What is the main goal of hypothesis testing?
To determine if there is a statistically significant difference between two groups.
What is the interpretation of a p-value less than 0.05?
The perceived difference is statistically significant.
What is a common mistake in hypothesis testing?
Not clearly stating which hypothesis is accepted and which is rejected.
What is the p-value in hypothesis testing?
The p-value indicates the probability that the observed difference is due to random chance.
What does it mean if the p-value is smaller than alpha?
It means we accept the alternative hypothesis (π»π»1) and reject the null hypothesis (π»π»0).
What is the default type of hypothesis test run by most analytical tools?
Two-tailed test.
In a two-tailed test with alpha 0.05, how is alpha distributed?
Half is at the far left of the distribution and half is at the far right.
What is a one-tailed test?
A test that puts all of the alpha on one side of the distribution.
What is type I error?
Falsely accepting the alternative hypothesis (π»π»1) and rejecting the null hypothesis (π»π»0).
What is type II error?
Falsely accepting the null hypothesis (π»π»0) and rejecting the alternative hypothesis (π»π»1).
What does alpha specifically describe?
The probability of a type I error.
What happens to beta as alpha decreases?
Beta increases, indicating a higher probability of type II error.
What is a common value for alpha in hypothesis testing?
0.05.
What is the relationship between type I and type II errors?
As the probability of type I error decreases, the probability of type II error increases.
What are the two parts of a good question in hypothesis testing?
- What two groups you want compared
- The metric you want to use to compare those groups.
What qualities should a good question have?
- The question is short
- The question is easy to understand
- The question is specific
- The question is something you can answer.
What should you do if you receive a vague question from your boss?
Ask for clarification to understand what they truly want to know.
What is the risk of answering a vague question with one massive visualization?
It may lead to misunderstandings and require more explanation.
What does a p-value of 0.04 indicate with an alpha of 0.05?
You should accept the alternative hypothesis and reject the null hypothesis.
What type of error occurs when you claim no difference when there actually is one?
Type II error.
Fill in the blank: A type I error is also known as a _______.
false positive.
Fill in the blank: A type II error is also known as a _______.
false negative.
What is the null hypothesis in a hypothesis test comparing two groups?
The null hypothesis states that the two groups are the same and there is no statistically significant difference between them.
When should you accept the alternative hypothesis in hypothesis testing?
You should accept the alternative hypothesis and reject the null hypothesis when your p-value is equal to or smaller than your alpha.
What type of error occurs when the null hypothesis is accepted incorrectly?
Type II error.
Which p-value indicates no difference between the new system and the old system when alpha is 0.05?
0.4.
What is a Type I error in hypothesis testing?
A Type I error occurs when you falsely accept the alternative hypothesis and reject the null hypothesis.
Fill in the blank: A p-value of _______ would lead you to believe there is no difference between the new system and the old system.
0.4.
True or False: A p-value smaller than 0.05 leads to the acceptance of the null hypothesis.
False.
What is indicated by a p-value larger than the alpha level in hypothesis testing?
We accept the null hypothesis and reject the alternative hypothesis.
What happens when you find a difference between the new system and the old system when there actually isnβt?
Type I error.
