Hypothesis Testing Flashcards
Two Types of Hypothesis Tests
- One sample Test
- Two sample Test
One sample Test
A one-sample test determines whether or not a population parameter, like a mean or proportion, is equal to a specific value
Two Sample Test
A two-sample test determines whether or not two population parameters, such as two means or two proportions, are equal to each other
Steps for performing a hypothesis test
- state the null hypothesis and the alternative hypothesis
- choose a significance level
- find the p-value
- reject or fail to reject the null hypothesis.
Null hypothesis
- is a statement that is assumed to be true unless there’s convincing evidence to the contrary.
- assumes that your observed data occurs by chance
- There is no effect in the population.
- Symbols: Equality =, ≤, ≥
- Phrases: no effect, no difference, no relationship, or no change
H0: μ = 300 (the mean weight of all produced granola bags is equal to 300 grams)
Alternative hypothesis
is a statement that contradicts the null hypothesis, and is accepted as true only if there’s convincing evidence for it.
- There is an effect in the population.
- Symbols: ≠, <, >
- Phrases: an effect, a difference, a relationship, a change
Ha: μ ≠ 300 (the mean weight of all produced granola bags is not equal to 300 grams)
significance level (α)
- significance level (α), represents the probability of making a Type I error.
- A significance level of five percent means you are willing to accept a five percent chance you are wrong when you reject the null hypothesis.
- typically use 5%
P-value
- P-value refers to the probability of observing results as or more extreme than those observed when the null hypothesis is true.
- lower p-value means there is stronger evidence for the alternative hypothesis
When to reject or fail to reject the null hypothesis?
- If your p-value is less than your significance level, you reject the null hypothesis.
- If your p-value is greater than your significance level, you fail to reject the null hypothesis.
2 Types of Hypothesis Test Errors
- Type I error
- Type II error.
- A statistically significant result cannot prove with 100 percent certainty that our hypothesis is correct.
- Because hypothesis testing is based on probability, there’s always a chance of drawing the wrong conclusion about the null hypothesis.
Type I error
- false positive
- occurs when you reject a null hypothesis that is actually true.
- In other words, you conclude that your result is statistically significant when in fact it occurred by chance.
- you incorrectly conclude that the medicine relieves cold symptoms when it’s actually ineffective.
The probability of making a Type I error
’- significance level (α), represents the probability of making a Type I error.
- α = 5% means you are willing to accept a 5% chance you are wrong when you reject the null hypothesis.
How to reduce Type 1 Error
A significance level of five percent means you are willing to accept a five percent chance you are wrong when you reject the null hypothesis.
To reduce your chance of making a Type I error, choose a lower significance level.
- from 5% to 1%
- reducing your risk of making a Type I error means you are more likely to make a Type II error, or false negative.
Type II error
- false negative
- This occurs when you fail to reject a null hypothesis, which is actually false.
- In other words, you conclude your result occurred by chance when it’s in fact statistically significant.
- ex. you incorrectly conclude that the medicine is ineffective when it actually relieves cold symptoms.
differences between the null hypothesis and the alternative hypothesis
- H0: typically assumes that there is no effect in the population, and that your observed data occurs by chance
- Ha: typically assumes that there is an effect in the population, and that your observed data does not occur by chance and is statistically significant.
Example
- H0: the program had no effect on sales revenue.
- Ha: the program** increased** sales revenue.
The probability of making a Type II error
is called beta (β), and beta is related to the power of a hypothesis test (power = 1- β). Power refers to the likelihood that a test can correctly detect a real effect when there is one.
How to reduce Type II Error
- by ensuring your test has enough power.
- In data work, power is usually set at 0.80 or 80%.
- The higher the statistical power, the lower the probability of making a Type II error.
- To increase power, you can** increase your sample size** or your significance level.
4 Outcomes of rejecting or failing to reject H0
- Reject the H0 when it’s actually true (Type I error)
- Reject the H0 when it’s actually false (Correct)
- Fail to reject the H0 when it’s actually true (Correct)
- Fail to reject the H0 when it’s actually false (Type II error)
Potential risks of Type I errors
A Type I error means rejecting a null hypothesis which is actually true. In general, making a Type I error often leads to implementing changes that are unnecessary and ineffective, and which waste valuable time and resources.
For example, if you make a Type I error in your clinical trial, the new medicine will be considered effective even though it’s actually ineffective. Based on this incorrect conclusion, an ineffective medication may be prescribed to a large number of people. Plus, other treatment options may be rejected in favor of the new medicine.
Potential risks of Type II errors
A Type II error means failing to reject a null hypothesis which is actually false. In general, making a Type II error may result in missed opportunities for positive change and innovation. A lack of innovation can be costly for people and organizations.
For example, if you make a Type II error in your clinical trial, the new medicine will be considered ineffective even though it’s actually effective. This means that a useful medication may not reach a large number of people who could benefit from it.
One-Sample Hypothesis Test Applications
- A data professional might conduct a one-sample hypothesis test to determine if a company’s average sales revenue is equal to a target value,
- a medical treatment’s average rate of success is equal to a set goal,
- or a stock portfolio’s average rate of return is equal to a market benchmark.
One Sample Z-Test Assumptions
- the data is a random sample of a normally-distributed population,
- the population standard deviation is known.
Test Statistics
The p-value is calculated from what’s called a test statistic.
In hypothesis testing, the test statistic is a value that shows how closely your observed data matches the distribution expected under the null hypothesis, so if you assume the null hypothesis is true and the mean delivery time is 40 minutes, the data for delivery times follows a normal distribution. The test statistic shows where your observed data, a sample mean delivery time of 38 minutes, will fall on that distribution.
Z-Score (Hypothesis Test)
- Since you’re conducting a z-test, your test statistic is a z-score.
- Recall that a z-score is a measure of how many standard deviations below or above the population mean a data point is.
- Z-scores tell you where your values lie on a normal distribution.
Z-Score: Left-Tailed Test
- For a normal distribution, the probability of getting a value less than your z-score of -2.82 is calculated by taking the area under the curve to the left of the z-score.
- This is called a left-tailed test because yourp-value is located on the left tail of the distribution.
- The area under this part of the curve is the same as your p-value
Z-Score: RIght-Tailed Test
- For a normal distribution, the probability of getting a value less than your z-score of 2.82 is calculated by taking the area under the curve to the left of the z-score.
- This is called a right-tailed test because yourp-value is located on the right tail of the distribution.
- The area under this part of the curve is the same as your p-value
