Probability And Significance; Thpe 1 And Type 2 Errors Flashcards
“Level of Statistical Significance” defined
“The level at which the decision is made to reject the null hypothesis in favour of the experimental hypothesis. It states how sure we can be that the IV is having an effect on the DV and this is not due to chance.”
“Chance defined
“Something has no real cause, it just happens, e.g. by chance you are feeling happy today, there is no real cause that you can identify.”
What are significance levels
From the results gained from our experiment (for both the control and the experimental conditions) we would look for whether a real difference exists between the two sets of data, and how certain we are that there is a real difference. If the two sets of data are very similar, then a statistical test might indicate that chocolate makes no real difference to mood and we might accept the null hypothesis. However, if there is a probability that there is a real difference between the two conditions (and this can be proved by conducting statistical tests) then we would accept the experimental hypothesis and reject the null hypothesis.
What is probability
Probability is a numerical measure that determines whether our results are due to chance or whether there is a real difference that exists between the experimental and control conditions (and therefore we can accept the experimental hypothesis). If a real difference exists (that can be calculated statistically) we can say that results are significant, and the null hypothesis can be rejected (and we would accept the experimental hypothesis)
Which significance levels should I use
• The conventional and standard level of significance is expressed as:
p<0.05 (5% level). The 5% level of significance is mainly used in Psychology because:
• A) It is not too strict or too lenient, but is a middle, fair value of significance
• B) It minimises the chances of making a Type 1 or a Type 2 error (see later on).
• “p” stands for “probability” and the 0.05 (5%) value illustrates the level of significance that has been chosen (5% level of probability that results are due to chance/fluke, therefore 95% certainty that our results are showing a real difference between control and experimental conditions)
• This means that if the level of significance is achieved, then the probability of results being due to chance (or fluke) is 5% or less. This can also be expressed as p = 0.05, but we tend to use p<0.05 (5%)
• 5% significance levels are usually used when there is a directional one tailed hypothesis that has been clearly stated in the research
• Sometimes a 10% level of significance is selected, and this is expressed as; p<0.10 (10%), and this is often used when we allow a 10% margin of error, and we would be 90% certain that our results are really showing a significant difference
• Sometimes a very strict level of significance is selected at 1% which is expressed as: p<0.01 (which indicates there is a 1% probability that the results are due to chance). This is often used when research findings are critical and are very important e.g. when testing the effect of drugs on humans, we must make sure that results are not due to fluke but that a real difference occurs between the experimental and control conditions, and that is why we set a stricter significance level.
Type 1 and type 2 errors
When using probability and significance levels, sometimes mistakes can be made concerning the statistical data and its analysis.
What is a type 1 error
Is the incorrect rejection of a null hypothesis which is actually true. Researchers claim to have found a significant difference when there actually isn’t any (a false positive)
What is a type 2 error
Is the failure to reject the null hypothesis that is false. Researchers claim that there is no significant difference when there actually is one (a false negative)
