Research Methods - Inferential Testing Flashcards
The Spearman Rho’s Test
Scenario - In our healthy eating research, the researchers were interested in the relationship between age and attitude to healthy eating.
- To analyse the findings the psychologist used a Spearman’s Rank Order Correlation Coefficient, the scores are ranked, each column is ranked independently
The Spearman’s Rho test is used when the following conditions are met:
- Appropriate for variables that produces ordinal type of data (order)
- Exploring a relationship between co-variables
- A correlational design has been used.
The Spearman Rho’s Test - Steps
Step one: Data is categorised into a table of results
Step two: Data is then ranked the lowest given a rank of 1 and so on (each column ranked individually)
- If there is a tied score the rank scores are added and divided 10 = 3 + 4 / 2 = 3.5
Step three, four and five - The difference (d) between ranks needs to be calculated = Rank1 – Rank 2 and then squared (d2) and then all the values d2 are added together to give the sum Σd2
- (Negative signs disappear when you square the values.)
Step six – The FORMULA
Rs = 1 - 6Σd(squared) / n(n(squared)-1)
Step Seven: Level of significance - this requires looking at the critical values of the Spearman’s Rho test in the critical value table for this test.
- N = 9 Hypothesis two tailed, significant level = 0.05, 5%. (nb. The – or + simply tells you whether it is a positive or negative correlation)
Probability and significance
In the scientific process, in order to establish general laws and principles about human behaviour, the findings of an investigation need to be balanced against the probability of them occurring purely by chance.
- Probability (p) is a numerical measure of how likely something is to have occurred purely by chance, with 0 being 0% and 1 being 100%.
- Therefore, the closer to 0% the findings of a research investigation are, the greater the confidence the researcher has in claiming the results are due to manipulation of the independent variable or relationship between the co-variables and can therefore reject the null hypothesis (that there will not be a difference or relationship between the variables).
- When a researcher designs an investigation they need to decide on the level of probability that is appropriate to the aims of the investigation.
- For example, investigations involving health issues would require higher levels of confidence that chance will be less than 1%, due to the ethical concerns that are involved.
- For most conventional investigations a minimum level of 5% is chosen.
- Significance is statistical criteria that determines if differences and relationships are beyond the boundaries of chance
Statistical significance
In psychology for an investigation to be statistically significant, probability has to be less than 5%. This is expressed as p ≤ 0.05. The interpretation of this is that there is less than a 5% chance that the results occurred due to something other than the independent variable or purely by chance.
Type 1 Errors
Setting the levels of significance at 5% increases the chances of finding a significant result. Therefore, allowing the researcher to reject the null hypothesis. However, this level of significance increases the risk of finding a difference or relationship when in truth there is none. This is known as a Type 1 error
- This is a false positive
- The higher the p value, the more likely there is to be a Type 1 error
- Type 1 - the alternative/experimental hypothesis is wrongly accepted and the data is not significant
Type 2 Errors
On the other hand when higher levels of significance are set, for example 0.1% (p ≤ 0.01), findings that show a level of significance greater that 0.1% will mean that the researcher will have to accept the null hypothesis that there is no difference or relationship between the variables. This is the case even if the level of significance remains below 5%. However, any level of significance that is below 5% shows that there may be a difference or relationship and failure to accept this is known as a Type 2 error.
- This is a false negative
- The lower the P value, the higher the chance of a Type 2 error
- Type 2 - the alternative/experimental hypothesis is wrongly rejected and the data is actually significant
Ratio, interval data, nominal data and ordinal data
Continuous - Data that has a true zero, as in most measures of physical quantities - standardised units
Interval data - Data measured using units of equal length, such as counting correct answers on a test - standardised units with an arbitrary zero
Nominal data - Data is collected in separate categories, such as grouping according to favourite football team
Ordinal data - Data that is ordered in some way, such as people standing in different height order (rank order)
The Sign Test - the conditions
- In the Sign test if the observed value is equal to or less than the critical value, the null hypothesis can be rejected and the test is significant, expressed as p ≤ (the stated level of significance).
- If the observed value is greater than the critical value, the null hypothesis has to be accepted, the test is not significant. This would be expressed as: p > 0.05
- In the Sign Test we calculate an observed value is known as S
- We then have to compare the observed value with the critical values in the statistical table; this will determine if the findings are significant or not.
In order to do this, we need to know:
1. N = the number of participants
2. What is the stated level of acceptable significance i.e. p ≤ 0.05
3. Whether the hypothesis is directional (one-tailed) or non-directional (two-tailed)
Observed Value - The number produced after the various steps and calculations for a statistical test have been carried out.
Critical Value - A value taken from a statistical test table, which must be reached in order for results to be significant.
The Sign Test - what is a significant difference?
We need to determine if there is a significant difference.
- Simply finding a difference is not good enough, as any difference may simply be down to chance; to determine the level of probability that the difference is down to chance the psychologist carries out a statistical test, such as a sign test.
- Any level of significance equal to or below 0.05 (5%) allows the psychologist to accept the research hypothesis that any effect is due to the influence of the IV. This is expressed as p≤0.05
- Anything above 0.05 means that the research hypothesis is rejected and the null hypothesis, that there is no difference and any difference observed is down to chance, is accepted. Expressed at p≥0.05
Key Term
1. Significant Difference - A difference between two variables that is shown statistically not to be down to random chance
2. Probability - The likelihood that something is going to happen
3. Level of Significance - In psychology this is the level at which the research hypothesis can be accepted (less than 0.05) and the likelihood of something being down to chance rejected
The Sign Test - what is needed?
The sign test is used when the following conditions are met:
- Testing for a difference – burger or salad
- Repeated measures or matched pairs design – all 10 asked whether they like burger or salad
- Nominal data – data that is in categories such as yes and no answers
The Steps of the Sign Test
Example Hypothesis - there will be a difference between stress scores before and after a 6 week course in CBT
Step one: Data is categorised into a table of results
Step two: Positive and negative signs need to be added to the difference (row of direction)
Step three: This step requires the counting of each positive and negative sign assigned to each participant’s scores. Ignore numbers before the sign.
(+) TOTAL = 5
(-) TOTAL = 3
You can ignore the 0s (2 in this example)
Step four: Find the direction of the occurring sign, in this case it is minus which occurs 3 times
- This is the observed or calculated value of S = 3
- We also need to know the number of participants, in this case N=8 as we ignore the zeros
Step 5 of the Sign Test
Step five: Level of significance - this requires looking at the critical values of the sign test in the values table.
- Ask: is our test one or two tailed - this depends where you look in the table
- The level of significance is 0.05 unless you are told otherwise in the question
- N = number of participants whose scores were used.
- Any scores that are the same are ignored. In this example, number of participants scores used = 8 participants.
- The critical Sign test value for N8 = 0 for two tailed test
- The observed smallest Sign test value (S) for the research = 3
- In order for the study to be significant, the observed value has to be EQUAL TO or LESS THAN the critical value in the Sign test table.
- So in this example there is no sig difference b/w depression scores before and after CBT
How to find the critical value
- Identify the significance level, which is 0.05 unless told otherwise (general probability of accepted hypothesis) - P = 0.05
- Identify the number of participants included e.g. N=8
- In the sign test table, the square where the two values meet, and this is the critical value
- The calculated value is the smallest Sign test value
In order for a test to be significant, the calculated value of ‘S’ must be equal to or less than the critical value in the test table. (S> it is not significant, S< it is significant)
Statistical tests
- Sign test - used when a difference is predicted between two sets of data, the data is of least a nominal nature and RMD has been used
- Chi-squared - used when a difference is predicted between two sets of data, data is nominal and IGD has been used - possible to use this as a test of association of relationship
- Mann-Whitney - used when a difference is predicted between two sets of data, data is ordinal and an IGD has been used
- Wilcoxon signed-matched ranks - used when a difference is predicted to occur between two sets of data, the data is ordinal and an RMD or MPD has been used
- Independent / unrelated t-test - used when a difference is predicted between two sets of data, data is normally distributed and that data is of interval/ratio level and an IGD has been used
- Repeated / related t-test - same as above, but a RMD or MPD has been used
- Spearman’s Rho - used when a relationship (correlation) is predicted between two sets of data, the data is of at least ordinal level and the data are pairs of scores from the same person or event
- Pearson product moment - used when a relationship or correlation is predicted between two sets of data, data is normally distributed, the data is of at least interval/ratio level and the data are pairs of scores from the same person or event
Quantitative v Qualitative data
- Most investigations would prefer quantitative as it is more reliable
- Qualitative can be converted to quantitative using content analysis
- Coding units for each time something is mentioned
- For example, coding each time someone said they strongly disagree with uniform
- Thematic analysis to check for reporting themes
