Issues with Significance Testing

Question 1

what is power

Answer 1

the probability of finding an effect assuming that one does exist

Question 2

whats a type 1 error

Answer 2

seeing an effect where there isnt one

Question 3

whats a type 2 error

Answer 3

not seeing an effect even though there is one

Question 4

what is power influenced by

Answer 4

size of effect
number of participants
sources of variability
experimental design
choice of test
tails of a test

Question 5

how does effect size influence power

Answer 5

its easier to measure/see an effect if it is a large one

Question 6

how does participant number influence power

Answer 6

if the effect is small we may need more participants to be sure an effect seen is real - we can be more confident in our data with more participants

Question 7

how do people choose what size alpha to use

Answer 7

its normally 0.05

we would use a small alpha such as 0.001 if the outcome of seeing an effect where there isnt one is bad

we would use a large alpha eg 0.2 if the outcome of not seeing an effect when there is one is bad - eg risk of death from drugs

Question 8

how do sources of variability influence power

Answer 8

sources of variability are things like confounding variables - things influencing participants other than whats meant to

Question 9

whats a one tailed test mean in terms of power

Answer 9

we hypothesise that subjects scores in condition A will be greater/less than condition B

(the p value is based on this one direction - easier to find an effect - better power)

Question 10

whats a two tailed test mean in terms of power

Answer 10

we hypothesise that subjects scores in condition A will be different to condition B

(the p value is based on both directions - harder to find an effect - less power)

Question 11

whats family wise error

Answer 11

the probability of making one or more false discoveries, or type I errors, among all the hypotheses when performing multiple hypotheses tests.