Hypothesis Testing Overview

Question 1

Hypothesis Testing

Answer 1

• Combines tried-and-tested analysis tools, real-world data, and framework that allows us to test our assumptions and beliefs.
• This way we can say how likely something is to be true or not true within a standard of accuracy

Question 2

When using Hypothesis testing we create:

Answer 2

A Null Hypothesis (H0)

Alternative Hypothesis (Ha)

• These hypotheses should ALWAYS be mutually excusive: if one if true, the other is false
• Once we have our null and alternative hypotheses, we test them with a sample of an entire population, check our results, and come up with a conclusion based on those results.

Question 3

Null Hypothesis (H0)

Answer 3

• The assumption that the experimental results are due to chance alone; nothing (from 6M) influenced our results
• A Null Hypothesis is never accepted; we simply fail to reject it.
• We are ALWAYS testing the Null
• A Null is what you would expect by chance alone
• A Null assumes things to be equal
• A Null Hypothesis is NOT your theory

Question 4

Alternative Hypothesis (Ha) or (H₁)

Answer 4

• We expect to find a particular outcome
• A hypothesis that disagrees with the null hypothesis
• The complementary hypothesis of the null hypothesis is an alternative hypothesis.
  
  • In other words, the alternative hypothesis shows that observations are the results of a real effect.
• An Alternative Hypothesis IS your theory
• Statement which is true if the Null Hypothesis is false
• The type of test (left, right, or two-tail) is based on the alternative hypothesis
• When the Null Hypothesis contains only an equal sign, the Alternative Hypothesis contains a “not equal to” sign.

Alternative Hypothesis for a Two Tailed Test

H0: µ_new = µ_current Ha: µ_new is not = µ_current

Question 5

Basic Hypothesis Testing Process

Answer 5

1. Identify the question
   
   1. Generally speaking, you’re looking to create a simple question that asks whether factor x affects scenario y, with an answer of ‘yes’ or ‘no.’
   2. Identify your Null and Alternative Hypothesis
2. Determine the significance
   
   1. Identify Sample Size and Confidence Level
3. Choose the test
4. Interpret the results
5. Make a decision

Question 6

Examples of Hypothesis Questions

Answer 6

Question 7

Identify your Null and Alternative Hypothesis

Answer 7

Question 8

When the Null Hypothesis Includes ONLY an equal sign

Answer 8

• The hypothesis test has two tails ( or rejection regions)
• The alternative hypothesis contains a “not equal to” sign
• It can be rejected by the test statistic being significantly large or small
• Statement of zero or no change. If the original claim includes equality (<=, =, or >=), it is the null hypothesis.
• If the original claim does not include equality () then the null hypothesis is the complement of the original claim.
• The null hypothesis always includes the equal sign.
• The decision is based on the null hypothesis.

Question 9

Hypothesis Testing Errors

Answer 9

Type 1 Error (Alpha)

Type 2 Error (Beta)

Question 10

Type 1 Error (Alpha Risk)

Aka

Type A Error

Answer 10

• Happens when our significance level is too large
• Rejecting the Null Hypothesis when it is true (saying false when true). Usually the more serious error.
• Type 1 error involves the Significance level.
  
  • For example, if alpha = 5%, then 5% of the time we will say there is a real difference between the null and alternate hypothesis (reject the null hypothesis) when there is no evidence of a difference
• Alpha - probability of committing a Type 1 error
  
  • The lower the alpha, the lower our chance of making a type 1 error
  • Always between 0 and 1

Question 11

Type II Error (Beta Risk)

Aka

Type B Error

Answer 11

• Failing to reject the null hypothesis when it is false (saying true when false)
• Beta - Probability of committing a Type II error

Question 12

Determine Significance

Answer 12

• Consists of two elements
  
  1. Sample Size
     
     1. Sample of the population
  2. Confidence Level
     
     1. How sure you need to be that the results you receive are actually statistically significant and that the conclusion based on them is correct
        
        1. Once you decide this, you can calculate the alpha level
           
           1. Alpha Level = (1 - confidence level)
           2. The standard confidence level used is 95%, or 0.95. Hence, the standard alpha level is 5%, or 0.05.

Question 13

P-Value

Answer 13

• The probability of the sample being studied could have drawn from the population due to change
• If P is low, Null must GO
• A p-value less than the alpha level decided upon in the Decide Significance step means that you can assume that your results are statistically significant
  
  • The null hypothesis can be rejected and the alternative hypothesis can be supported
• A p-value greater than the alpha level means you cannot assume that your results are statistically significant, and hence cannot reject the null hypothesis
  
  • The P-Value is integral in using a hypothesis test to make a decision
    
    • It reflects the possibility of falsely rejecting the null hypothesis when it really is true
  • If the P-Value is less than or equal to the agree upon significance level (alpha), then you reject the null and can support the alternate hypothesis
  • If the P-Value is greater than the significance level (alpha), then you cannot reject the null hypothesis (in states terms, you have to fail to reject the null) and thus you cannot support the alternate.
• P-Value is expressed as a number between 0 and 1
  
  • Once a p-value is calculated from analyzing test data, it is compared to the selected alpha level – if lower than the alpha level, the results are deemed to be statistically significant; if higher, the results are deemed to not be statistically significant.

Question 14

Conclusion

Answer 14

A statement which indicates the level of evidence (sufficient or insufficient), at what level of significance, and whether the original claim is rejected (null) or supported (alternative)

Question 15

Confidence Level

Answer 15

Also known as the confidence interval

This refers to how confident you can be that your conclusion is in face correct

The confidence level is easy to calculate: the alpha and confidence levels always add up to one

• 1 – α = confidence level

Question 16

Critical Region

Answer 16

• Set of all values which cause us to reject the null hypothesis H_0.
• Also known as rejection region

Question 17

Critical Value(s)

Answer 17

• The value(s) which separate the critical region from the non-critical region. The critical values are determined independently of the same sample statistics
• A critical value separates the rejection region from the non-rejection region

Question 18

Decision

Answer 18

• A statement based upon the null hypothesis.
• It is either
  
  • “reject the null hypothesis”
    
    • OR
  • “fail to reject the null hypothesis”
• We will NEVER accept the null hypothesis
• A p-value is the probability of getting a test statistic that is at least as extreme as the one found from the sample data

Question 19

Errors

Answer 19

• Type A or 1 Error: The null hypothesis is correct, but is incorrectly rejected.
• Type B or 2 Error: The null hypothesis is incorrect, but is not rejected.

Question 20

Left-Tailed Test

Answer 20

• If the alternative hypothesis H1 contains less-than inequality symbol (

Question 21

Pooled (vs Unpooled)

Answer 21

• The pooling refers to the way in which the standard error is estimated when calculating terms for a hypothesis test
• Pooled - the two proportions are averaged, and only one proportion is used to estimate the standard error.
  
  • ASQ favor pooled calculations
• Unpooled - the two proportions are used separately

Question 22

Right-Tailed Test

Answer 22

• If the Alternative Hypothesis H₁ contains the greater than inequality symbol (>), the hypothesis test is a right tailed test.
  
  • In hypothesis testing, when performing a right tailed test we reject the null hypothesis if the test statistics is larger than the critical value
  • Only when the test statistic is larger than the critical value will be able to reject the null.
    
    • Usually, we are hoping that we can reject the null because that means that our efforts are not in value
    • If you are testing the null hypothesis and you are hoping that you have no adversely affected the process, you would thne be hoping NOT to reject the null

Question 23

Significance Level

Aka

Alpha Level

Answer 23

• The probability of rejecting the null hypothesis when it is true
• alpha = 0.05 and alpha = 0.01 are common
• If no level of significance is given, use alpha = 0.05
• The level of significance is the complement of the level of confidence in estimation
• The significance level (denoted by Alpha) is the probability that the test statistic will fall in the critical region when the null hypothesis is actually true.

Question 24

Test Statistic

Answer 24

• Sample statistic used to decide whether to reject or fail to reject the null hypothesis

Question 25

Two Tailed Test

Answer 25

• A two tailed test is one with two rejection regions
• If the null hypothesis has an equal sign, then this is a two tailed test and you can use the test statistic to reject the null hypothesis if the test statistic is too large or too small

H0: µnew = µcurrent

Ha: µnew ≠ µcurrent

H0: µ_new = µ_current Ha: µ_new is not = µ_current

• Example, if the null hypothesis has an equal sign, then it is a 2 tailed test and you can use the test statistic to reject the null hypothesis if the test statistic is too large or too small