probability and significance

Question 1

probability and significance - the null hypothesis

Answer 1

• researcher begin investigation by writing a directional or non-directional hypothesis (alternative hypothesis)
• null hypothesis states no difference between conditions
• statistical test determines which hypothesis is true and wether we accept or reject the null hypothesis

Question 2

probability and significance - levels of significance and probability

Answer 2

• statistical tests work on basis of probability rather than certainty
• all tests employ a significance level, the point at which the researcher can claim to have discovered a large enough difference or correlation to claim an effect has been found (the point at which the null hypothesis is rejected and the alternative hypothesis is accepted)
• usual level of significance is 0.05 or 5%, means that the probability that the observed effect occurred when there is n effect in the population is equal to or less than 5%
• even when a researcher claims to have found a significant difference / correlation, there is still up to a 5% chance that it isn’t true for the target population in which it was found
• can never be 100% sure about a result as they haven’t tested all members of population under all possible circumstances

Question 3

use of statistical tables - calculated and critical values

Answer 3

• once statistical test is calculated, result is a number (calculated value)
• to check for significance, calculated value must be compared with a critical value, a number that tells us whether or not we can accept alternative hypothesis and reject null hypothesis
• each stats test has own table of critical values
• for some tests, calculated value must be equal to or greater than critical value, and for other tests, calculated value must be equal to or less than critical value

Question 4

use of statistical tables - using tables of critical values

Answer 4

three criteria for knowing which critical value to use -
- one or two tailed test? - one-tailed if hypothesis was directional, two-tailed if not, probability levels double when two-tailed tests are used as they are a more conservative prediction
- number of participants in study - usually appears as N on table, for some tests, degrees of freedom (df) are used instead
- level of significance (p) - 0.05 is standard level

Question 5

use of statistical tables - levels of significance

Answer 5

• 0.05 is standard level
• occasionally, a more stringent level may be used, such as 0.01, in studies where there may be a human cost, or in one-off studies that couldn’t be repeated in the future
• if there is a large difference between calculated and critical values, in the preferred direction, researcher will check more stringent values as the lower the p value is, the more statistically significant the result

Question 6

type I and type II errors

Answer 6

• it is possible that wrong hypothesis may be accepted

type I error - when null hypothesis is rejected and alternative accepted when it should be other way round, often referred to as optimistic error or false positive, researcher claims to have found a significant difference or correlation when one does not exist

type II error - when null hypothesis is accepted but it should’ve been alternative hypothesis, this is a pessimistic error or false negative

• more likely to make type I error when significance level is too lenient / too high
• type II error more likely to occur when significance level is too stringent / too low
• 5% level is favoured as it balances risk of making each error