17 - Statistical Tests

Question 1

Hypothesis Test

Answer 1

• allows us to make a decision between 2 options
• provides a measure of the weight of evidence against the null hypothesis ⇒ p-value
  
  • ​p-value adds more force to your conclusions
• quantifies the 2 types of errors, Type 1 and Type 2, alpha and beta

Question 2

null and alternative hypotheses

Answer 2

null (H₀) = status quo

alternative (H₁) = converse of the null

Question 3

Type 1 Error

Answer 3

alpha

Reject H₀ incorrectly

false positive

innocent, but declare guilty

null true, but go with alternative

Question 4

Type 2 Error

Answer 4

beta, ß

Retain H₀ even when H_a is true

false negative

guilty, but say innocent

alternative true, but go with null

Question 5

test statistic

Answer 5

sample statistic that estimates the population parameter specified by the null and alternative hypotheses.

Question 6

The test statistic determines…

Answer 6

whether we reject H₀

“What is the chance of getting a test statistic this far from H₀ if H₀ is true?”

Question 7

Normal model for the sampling distribution of p^

Answer 7

p^ ~ N[p, p(1-p)/n]

Question 8

Significance Level

Answer 8

a; the chance of making a Type I error

Ex. if cut-off value for the test is 2, a=.05

if cut-off value for test is 1.645, a=.10

Question 9

t-test statistic

Answer 9

test statistic for testing a single mean against a hypothesized value

t-stat = (Xbar-µ₀)/(s/sqr(n))

n = sample size, s = sample SD

Xbar = sample mean

µ₀ = hypothesized value under the Null

**calculating how far away what we see (Xbar) is away from what we expect (µ₀), but on a standardized scale (the Empirical Rule scale)

Question 10

Assumptions for t-test statistic

Answer 10

1. SRS
2. sample size small compared to pop size (less than 10%)
3. sample size large enough for CLT to have kicked in

Question 11

p-value

Answer 11

measures credibility of Null

small p-values give evidence against the Null

• small p says it would be hard to replicate under Null → evidence against Null
  
  • since you should be able to replicate observed results if Null is true

also: the p-value is the smallest alpha-level at which H₀ can be rejected

Question 12

P-value mantra

Answer 12

If the p-value is less than 0.05, then reject H₀ at the a=0.05 level of significance

Question 13

Testing Process

Answer 13

• Compare the test-statistic to the cut-off value or compare the p-value to a
• If the test statistic exceeds the cut-off value or the p-value is less than a then reject H₀.

**cut-off value comes from looking up the appropriate quantile in the t-tables, but value is rounded to 2 for simplicity when the test is two-sided and a = 0.05

Question 14

Power of a test

Answer 14

1-ß (doing the right thing when the alternative is true)

Question 15

Z-test for population proportion

Answer 15

Z = (p^-p₀)/sqr[p₀(1-p₀)/n]

Question 16

One/Two-sided alternatives

Ex. We are better than the competition

We are worse than the competition

(how to determine w/JMP Data)

Answer 16

Achieve this by looking at the appropriate one-sided p-value (match the sign in the JMP output to the sign in the alternative hypothesis)

Ex. H₀ :p₀ ≤0.45 v. H₁ :p₀ >0.45

use Prob > t

Question 17

Calculating Beta and Power

Answer 17

1. given the alpha value → this is the area under the distrib
2. use alpha to find z value
3. convert z value to Xbar value
4. convert back to z value when given alternate µ value
5. calculate area under the curve w/z value; use standard normal table