Power Analysis

Question 1

What is a Type I error?

Answer 1

When we accept a difference or relationship when in reality there isn’t one

Question 2

What is a Type II error?

Answer 2

When we conclude that there is no difference or relationship when in reality there is one

Question 3

What is the usual significance criterion?

Answer 3

α (the risk of making a Type I error) is usually set at .05

Question 4

What are the limitations of statistical significance testing?

Answer 4

Statistical probability can’t be used as a measure of the magnitude of the result as this may reflect either the effect size or the sample size.

Question 5

What is the effect size?

Answer 5

A measure of the magnitude of a result which is independent of sample size

Question 6

What is statistical power?

Answer 6

The power of a test is defined as the probability of avoiding making a type II error
The recommended level of power is .80, the probability of making a type II error is .20

Question 7

What is the ratio of chances of making a Type I : Type II error?

Answer 7

1:4 ratio

Question 8

What does statistical power depend on?

Answer 8

Effect size
Sample size
Precision of measures

Question 9

What do error bars represent?

Answer 9

The variability of the mean, they are the standard error of the mean

Question 10

What does having a larger sample mean?

Answer 10

Smaller standard errors, which means the larger the sample, the greater the power (due to smaller standard errors)
SO effects are easier to detect

Question 11

What do more reliable measures provide?

Answer 11

More precise estimates of the latent variable (less variance/noise)
Will also result in smaller standard errors

Question 12

What is a prospective power analysis (“a priori”)?

Answer 12

• Set significance criterion (.05)
• Set statistical power (.80)
• Estimate effect size (on the basis of previous research, conduct pilot study, decide on small/med/large effect size)
• Calculate required sample size

Question 13

What is a retrospective power analysis (“sensitivity”)?

Answer 13

• Set significance criterion (.05)
• Set statistical power (.80)
• Sample size is known
• Estimate minimum effect size that could be detected

Question 14

What is a retrospective power analysis (“post hoc”)?

Answer 14

• Calculate effect size
• Significance criterion is known
• Sample size is known
• Calculate statistical power of test

Question 15

What is a retrospective power analysis (“a priori”)?

Answer 15

• Calculate effect size
• Set significance criterion (.05)
• Set statistical power (.80)
• Estimate required sample size for the effect size

Question 16

How do you use G*Power3 to conduct a power analysis?

Answer 16

Tests –> Test class –> Design
Choose one of the five types of power analysis
Provide input parameters
Click ‘calculate’ to obtain output

Question 17

What must be included in the report of a power analysis?

Answer 17

All the inputs determining the results

• Effect size e.g. d or r or other
• Significance level (α)
• Required power
• Sample size

Question 18

How do you report an a priori prospective power analysis?

Answer 18

An a priori power analysis indicated that it would be necessary to recruit __ participants per group in order to detect an effect size of d = [d value], with alpha set at .05, at 80% power.

Question 19

How do you report a sensitivity retrospective power analysis?

Answer 19

A power analysis indicated that with a sample of __ in the experimental group and __ in the control group, it would be possible to detect a minimum effect size of d = __, with alpha set at .05, at 80% power.

Question 20

How do you report a post hoc retrospective power analysis?

Answer 20

The effect size for the experiment was calculated as d = __. A power analysis indicated that with a sample of __ in the experimental group and __ in the control group, and with alpha set at .05, the analysis only achieved __% power.

Question 21

How do you report an a priori retrospective power analysis?

Answer 21

The effect size for the experiment was calculated as d = __. A power analysis indicated that it would be necessary to recruit at least __ participants per group to detect this effect size, with alpha set at .05, at 80% power.