Statistics

Question 1

In an experiment with a high p-value, your data are highly ____ given a true null hypothesis.

Answer 1

likely

Question 2

In an experiment with a low p-value, your data are highly ____ given a true null hypothesis.

Answer 2

unlikely

Question 3

The p-value is the probability of…

Answer 3

obtaining an effect AT LEAST as extreme as the one observed assuming that the null hypothesis is true

Question 4

A study found a difference between two means with a p-value of 0.02. Interpret this p-value in terms of many repetitions of identical studies.

Answer 4

If you repeated the study many times, you would find differences at least as large as observed in this study 2% of the time.

Question 5

The p-value answers what question?

Answer 5

How likely are your data given that the null hypothesis is true?

Question 6

The probability of falsely rejecting a true null hypothesis is called…

Answer 6

Type I error = false alarm = false positive

Question 7

What two factors determine the probabilities of Type 1 and Type 2 errors?

Answer 7

The desired level of significance and the power of the test

Question 8

The probability of falsely accepting a false null hypothesis is called…

Answer 8

Type II error = missed detection = false negative

Question 9

Does type 1 error reject or accept the null hypothesis?

Answer 9

Type 1 error (odd number) rejects the null hypothesis, an “even” number

Question 10

Does type 2 error reject or accept the null hypothesis?

Answer 10

Type 2 error (even number) accepts the null hypothesis, another even number

Question 11

If you commit a type I error, what do you do to the null hypothesis?

Answer 11

Type I error = reject the null hypothesis even though it’s actually true = false positive

Question 12

If you commit a type II error, what do you do to the null hypothesis?

Answer 12

Type II error = accept the null hypothesis even though it’s actually false = false negative

Question 13

A false positive is also known as what kind of error?

Answer 13

Type I: false Positive has one vertical line

Question 14

A false negative is also known as what kind of error?

Answer 14

Type II: false Negative has two vertical lines

Question 15

What’s the formula for statistical power in terms of α and/or β?

Answer 15

power = 1 - β

Question 16

If an experiment’s probability of type II error increases, then the statistical power ____

Answer 16

decreases; power = the ability to correctly reject a false null hypothesis = 1 - β

Question 17

The likelihood that a study will detect an effect when there really is one to be detected is…

Answer 17

statistical power

Question 18

A study reports no effect when in fact there was one. What kind of error is this?

Answer 18

Type II error = β = false negative

Question 19

A study reports an effect when in fact there was no effect. What kind of error is this?

Answer 19

Type I error = α = false positive

Question 20

What four factors affect statistical power?

Answer 20

1. effect size
2. sample size
3. desired α (type I error)
4. the chosen or implied β, or equivalently, the statistical power 1 – β

Given any three of these, you can find the fourth.

Question 21

What are the two families of effect size indexes?

Answer 21

1. differences between groups (risk ratio, odds ratio, Cohen’s d, Glass’s delta, etc.)
2. measures of association (corr coeff r, r^2, Spearman’s rho, Cohen’s f, etc.)

Question 22

T or F: the p-value is the probability of getting a false positive.

Answer 22

False! p = probability of seeing at least that big an effect, assuming null is true. It is actually impossible to calculate the probability that the null hypothesis is true solely from sample statistics.

Question 23

In general, which is greater: the probability that the null hypothesis is true, or the p-value?

Answer 23

The probability that null is true tends to be greater than the p-value by a large margin.

Question 24

T or F: a confidence interval is a range of values that is likely to contain an unknown population parameter.

Answer 24

True

Question 25

If you draw a random sample many times, a certain percentage of the confidence intervals will contain the population mean. What is the name for this percentage?

Answer 25

The confidence level

Question 26

T or F: the confidence LEVEL is the probability that a specific confidence interval contains the population parameter.

Answer 26

False! For any given study, the confidence interval either contains or does not contain the population parameter of interest.

Question 27

Express confidence level in terms of α and/or β.

Answer 27

Confidence level = 1 – α

Question 28

If the confidence interval does not contain your null hypothesis value, what can you say about statistical significance?

Answer 28

If CI does not contain the H_0 value, the result is statistically significant.

Question 29

If p < α, what do you know about the confidence interval?

Answer 29

If p < α, the confidence interval will not contain the null hypothesis value.

Question 30

If 95% confidence intervals for two independent sample means overlap, could there be a statistically significant difference between them?

Answer 30

Yes! Non-overlapping CIs always imply a significant difference. But 95% CIs can sometimes overlap even when p < 0.05.

Question 31

What is the definition of positive predictive value?

Answer 31

PPV is the probability that a significant result represents a true effect. In terms of disease screening, PPV = P(disease | positive).

Question 32

In the context of disease screening, what is PPV expressed as a conditional probability?

Answer 32

PPV = P( disease | positive).

Question 33

In the context of disease screening, what is NPV expressed as a conditional probability?

Answer 33

NPV = P(healthy | negative).

Question 34

In the context of disease screening, what is the sensitivity expressed as a conditional probability?

Answer 34

Sensitivity = P(positive | disease).

Recall: sensitivity = true positive rate

Question 35

In the context of disease screening, what is specificity expressed as a conditional probability?

Answer 35

Specificity = P(negative | healthy).

Recall: specificity = true negative rate

Question 36

T or F: specificity is synonymous with negative predictive value (NPV).

Answer 36

False.

Specificity = P(negative | healthy), whereas
NPV = P(healthy | negative).

Question 37

T or F: sensitivity is the same thing as positive predictive value (PPV).

Answer 37

False.

Sensitivity = P(positive | disease), whereas
PPV = P(disease | positive).

Question 38

What does a ROC curve have on its x- and y-axes?

Answer 38

x-axis: FPR = 1 – specificity = α
y-axis: TPR = sensitivity

(Sensitivity = true positive rate, and
Specificity = true negative rate)

Question 39

What is another term for the sensitivity of a binary classifier?

Answer 39

True positive rate

Question 40

What is another term for the true positive rate of a binary classifier?

Answer 40

Sensitivity

Question 41

What is another term for the specificity of a binary classifier?

Answer 41

True negative rate

Question 42

What is another term for the true negative rate of a binary classifier?

Answer 42

Specificity

Question 43

Given the true positive rate, how do you calculate the false negative rate?

Answer 43

TPR = 1 – FNR

Think: true positives (TPR) + positives falsely labeled as negatives (FNR) = 1.

Question 44

How do you calculate the true positive rate if you know the false negative rate?

Answer 44

FNR = 1 – TPR

Think: true positives (TPR) + positives falsely labeled as negatives (FNR) = 1.

Question 45

What is the false positive rate in terms of specificity?

Answer 45

FPR = 1 – specificity.

FPR = 1 – TNR, since specificity = true negative rate.