Lecture 9 - Issues with Significance Testing

Question 0

What is the probability of a type 1 error?

Answer 0

Alpha

Question 1

Define power

Answer 1

The probability of finding an effect given that it actually exists,
1-ß = p(finding p<alpha given that H0 is false)
1-ß = p(true positive)

Question 2

What is the probability of a type II error?

Answer 2

ß

Question 3

What is the probability of a true negative?

Answer 3

1-alpha

Question 4

What (7) factors can affect power?

Answer 4

• the size of the effect (more variability = more power)
• the number of subjects (more subjects = more power)
• choice of alpha (lower alpha = more power)
• sources of variability (remove noise = more power)
• experimental design (individual differences reduce power)
• choice of test (tests that account for difference sizes = more powerful)
• tails of a test (one tailed = more power)

Question 5

Do the writers of SPSS feel that we should only run one- or two-tailed tests?

Answer 5

Two-tailed, as one-tailed tests are based on prior knowledge and must be chosen before obtaining results.

Question 6

What information is necessary to calculate power?

Answer 6

The size of the effect, the number of subjects and the choice of alpha

Question 7

What is family-wise error?

Answer 7

An inflated type I error due to multiple comparisons

Question 8

How do you correct for multiple comparisons?

Answer 8

Bonferroni correction - divides the alpha value by the number of statistical comparisons made
Domain-specific knowledge can also help