9 Significance Tests

Question 1

What is the main point of doing a significance test?

Answer 1

To see if we have convincing evidence against a claim (H0) or in support of a counter claim (Ha).

Question 2

What do you do for the P step of PHANTOM?

Answer 2

Define your parameter(s). For proportions, “let p = the true prop of SOMETHING that would SOMETHING”

Question 3

What do you do for the H step of PHANTOM?

Answer 3

Write the Ho and the Ha using the parameter you just defined. The Ho should always be an equal sign and the Ha should use the same symbol and number but with a less, greater, or not equal sign.

Question 4

What do you do for the A step of PHANTOM?

Answer 4

Check Random, Normal (large counts. you have to use the number from the H0, not from the sample when doing np and n(1-p)), Independent: 10% condition

Question 5

What do you do for the N step of PHANTOM?

Answer 5

Name the test/procedure you are using: 1 (or 2) sample Z Test for proportions

Question 6

What do you do for the T step of PHANTOM?

Answer 6

List your test statistic (usually from the calculator. For proportions: z =

Question 7

What do you do for the O step of PHANTOM?

Answer 7

“obtain the p-value” State the p-value given by the calculator. Label it as: p-value =

Question 8

What do you do for the M step of PHANTOM?

Answer 8

Mantra: Since (p-value) is less (greater) than (alpha = .05 (or whatever alpha is), we (fail to) reject H0. We (don’t) have convincing evidence that (write out the Ha with context).

Question 9

Interpret the p-value.

Answer 9

You only need to use this cookie cutter when asked to interpret the p-value OR “what does .03 mean in the context of this problem?”
Assuming that __________ (H0 is true (with parameter written out)), there is a ___
(p-value) probability of getting a sample ______ (mean or proportion) of _______ (xbar or phat) or
_______ (more or less (depends on Ha)) just by chance in a random sample of ___ (n units)

Question 10

Basically, what’s a p-value?

Answer 10

The probability of getting evidence for the alternative hypothesis Hₐ as strong as or stronger than the observed evidence when the null hypothesis H₀ is true. The smaller the P-value, the stronger the evidence against H₀ and in favor of Hₐ provided by the data.

Question 11

What is a standardized test statistic?

Answer 11

Value that measures how far a sample statistic is from what we would expect if the null hypothesis H₀ were true, in standard deviation units. So…the z-score of your sample compared to the null.

Question 12

Formula for the test statistic in one sample proportion test

Answer 12

Note: use the p from the null on bottom, not the p-hat from the sample. This was the most missed MC last year.

Question 13

Formula for the test statistic in two sample proportion test

Answer 13

Note: use the combined p-hat for all the parts on bottom, not the 2 different p-hats.

Question 14

How do you find a p-value?

Answer 14

1 or 2 prop z-test. OR It’s just the probability of getting a z-score or more (or less) on a normal curve (normalcdf(lower: z-score, upper: 9999, mu=0, sigma = 1). OR approximate it using a simulation by counting how many dots are above (or below) the claim (H0).

Question 15

What symbols are never allowed in hypotheses?

Answer 15

p-hat or x-bar. Also, never use numbers from observed counts either. In the NBA bubble, don’t use the fact that the home teams won about 57% of the games anywhere in the problem except for finding the z-score (or typing it into the calculator to find the z-score)

Question 16

How do you tell which error Type could have happened?

Answer 16

Did you “fail to” reject? “Fail To” goes with Type 2. So if you didn’t “fail to” reject, then it must be a possible Type 1 error.

Question 17

How do you explain an error in context?

Answer 17

Type 1: We were convinced that (HA in context), BUT actually the (HA) is not true (context)
Ex: We were convinced that the water was unsafe, but actually it was safe.
Type 2: (fail to) We didn’t find evidence that (Ha in context), BUT actually (it was true with context).
Ex: We weren’t convinced that the water was unsafe, but it actually was unsafe

Question 18

How do you describe a consequence of an error?

Answer 18

Figure out what would happen if you reject and find enough evidence. Ex: switch to bottled water, sue the company for discrimination, use more coupons, etc. Then, Type 1: we switched but we shouldn’t have so now… Type 2: We didn’t switch but we should have and now…

Question 19

How do you find the probability of a Type 1 error?

Answer 19

Since a type 1 error is when you reject but shouldn’t, it will happen alpha % of the time (so usually 5%). This is because we say that alpha is our standard of rare enough. Things that are 5% rare still happen 5% of the time.

Question 20

How do you find the probability of a Type 2 error?

Answer 20

If they give you the power, just subtract it from 1. Otherwise, you can’t really calculate it because it depends on what the real parameter is (which we don’t know).

Question 21

How can you reduce Type 2 errors?

Answer 21

Reducing a type to error is the same as increasing power. So, increase n, increase alpha, or increase the distance of the parameter in question (we will be less likely to make a type 2 error if the real free throw % is only 50% vs someone who is 60%

Question 22

How do you increase the power of a test?

Answer 22

Increase n, increase alpha, or increase the distance of the parameter in question (There is more power in a test when the real free throw % is only 50% vs someone who is 60%).
More generally, power goes up if spread (standard error) goes down, but other than increasing sample size, we can’t control spread much.

Question 23

What is the power of a test?

Answer 23

It’s the probability that a test works. i.e. you put the guilty person in jail. It always depends on a specific value of the parameter.

Question 24

How do you interpret the power of a test?

Answer 24

If Ha is true (at a specific value in context) there is a _____(power) probability of finding convincing evidence to reject the null (context).
Ex. If Mr. G is really a 60% shooter, there is a 0.85 probability that Mr. Warner’s test will be convincing enough to kick him off the team.

Question 25

How do you calculate the power of a test?

Answer 25

You don’t have to other than: 1 minus the probability of a type 2 error