Lecture 17 - Non-parametric tests Flashcards
1
Q
What is ‘parametric’?
A
- Nearly all the tests we’ve encountered so far are in the family of ‘parametric’ tests
- They are based on some commonly used parameters, such as standard deviation, mean, standard error, variance which assume a normal distribution
- If the data are not normally distributed, then those parameters might not be meaningful
- Ordinal scale = lines in right order but not got an equal step size
- Ordinal vs ratio/interval data = knowing whether the step size is equal (check afterwards with normal distribution e.g. ratings of attractiveness)
2
Q
What are interval scales and skews?
A
- Most data recorded on a ratio or interval scale are normally distributed
- When data are heavily skewed, the step size between points on the scale is probably not constant (i.e. the scale may be only ordinal)
- When data are ordinal, you may not have a normal distribution and parameters like SD, SEM, variance may no longer capture the data
- At the simplest level, one should probably be using a non-parametric test for ordinal data (or always run test of normality and base decision on results)
3
Q
What do most parametric tests have?
A
A non-parametric equivalent (usually based on ranking scores)
4
Q
What is a Mann-Whitney U test?
A
- 2 unpaired samples (between-groups design) = unpaired samples t-test equivalent
5
Q
What is the rationale behind a Mann-Whitney U test?
A
- If we arrange all the data into an ascending sequence of scores (can we see a surprising pattern in those ranks?)
- When the null hypothesis is true, we would expect the group labels then to be randomly distributed (roughly interspersed)
- When the null hypothesis is false, we would expect the scores of the two groups to be clustered at either end of the sequence (see clustering)
6
Q
Example of a Mann-Whitney U test
A
- Compare two groups of participants on learning lists of words and pictures
- Rank all scores (ignoring the IV) and sum them
- If the null hypothesis is true the rank totals should be the same
- If the null hypothesis is false the rank totals should be different
7
Q
What is the equation for U?
A
- Equation relative to number of participants (corrects for larger participant n having larger groups)
(Don’t need to know actual equation)
8
Q
How do we determine significance for a Mann-Whitney U test?
A
- As with other tests, this test-statistic allows us to determine significance, by comparing U to some critical value in a look-up table (or SPSS can do the test for us)
9
Q
How do you carry out the Mann-Whitney U test in SPSS?
A
- Make sure to have the data set to the right types of measure (nomina/ordinal/scale)
- ‘Task’ = categorical variable and only categorical variables can be used to assign groups
- ‘Score’ = can be set to either ordinal or scale
- Analyse -> nonparametric tests -> independent samples (automatically compare distributions across groups)
- Score in ‘test fields’
- Task in ‘groups’
- Error message comes up -> just have to press ‘ok’ and it goes away
10
Q
How would you report a Mann-Whitney U test?
A
- The median scores of subjects in the word condition and the picture condition were 10 and 14.5, respectively, with interquartile ranges of ___ and ___. A Mann-Whitney U test found the difference between these to be statistically significant (U=3.00, p<0.05)
- Use median score because mean/SD don’t make sense for reporting (don’t make sense for stats test) – test and reporting must match
11
Q
What is a Wilcoxon related-samples test?
A
- Wilcoxon signed-rank test
- 2 paired samples (repeated measures designs) = paired samples t-test equivalent
12
Q
What is the rationale behind a Wilcoxon related-samples test?
A
- Test is based on the difference between the 2 scores for each subject
- How often do pairs of scores go in one direction vs the other and size of differences compared to the other
- It uses the direction (which score is greater) and magnitude of the differences (how much greater)
- If H0 is true, then differences in one direction will be as large as the differences in the other
- That is what the test statistic will measure
- But it will use ranks of magnitudes rather than actual magnitudes so it can be used on skewed data
13
Q
What are tied ranks?
A
- Whenever we have tied ranks (in the unpaired or the paired test), we take the average of the range of ranks that the ties cover and allocate this to the value of the ties e.g. 4.5 and 4.5 not 4 and 5
- Do not change the other ranks!
14
Q
How do you conduct a Wilcoxon test in SPSS?
A
- Analyse -> non-parametric tests -> related samples (automatically compare observed data to hypothesised)
- Now got two columns per participant in data file (because repeated measures)
- Put variables in ‘test field’
15
Q
How do you report a Wilcoxon test?
A
- NB: SPSS give “Test Statistic (54.0, here) and also converts it to a “standardised test statistic” (a Z-score). You could report either one. Helps know whether it is extreme
- “A Wilcoxon signed ranks test showed that there was a significant difference in reaction times between the two conditions (W=54.0, p<0.01)”
16
Q
What is a Kruskal-Wallis test?
A
- Multiple levels with between-group designs = between-groups ANOVA equivalent
17
Q
What is the rationale behind a Kruskal-Wallis test?
A
- Take ranks, put them in order, and calculate the sum of ranks
- When the null hypothesis is true, we expect a random distribution of ranks across groups (average ranks of levels in the design should be equal)
- When the null hypothesis is false, we expect a systematic distribution of ranks across groups (average ranks of levels in the design should be different)
18
Q
Example Kruskal-Wallis test
A
- Experiment: we want to test the effects on learning of different types of reinforcement given to a child
- Three conditions:
- No reinforcement (independent learning)
- Positive (told ‘well done’ when correct)
- Negative (told ‘incorrect’ when incorrect)
19
Q
How do you conduct a Kruskal-Wallis test in SPSS?
A
- Analyse -> non-parametric tests -> k independent samples -> Kruskal-wallis
- ‘Score’ in test variable
- ‘Reinforcing group’ in grouping variable
- Won’t need to press ‘define range’ and select which conditions we want to include in the test
20
Q
How do you test differences with a Kruskal-Wallis test?
A
- Kruskal-Wallis only tells us that the 3 conditions differ, but not which ones
- We can use multiple Mann-Whitney tests and apply the Bonferroni correction (multiply the p value by the possible number of tests)
- Control vs Positive: U=13, p=0.4133 >0.05 | Control vs Negative: U=3.5, p=0.023 = 0.06 | Positive vs Negative: U=1.5, p=0.008*3 = 0.024
- “There was a significant […]. Mann-Whitney U tests on each pair of conditions, with a Bonferroni correction, revealed that this resulted from the positive reinforcer being significantly more effective than negative (U=1.5, p=0.024)”
21
Q
What is a Friedman test?
A
- Multiple levels with repeated-measures design = repeated measures ANOVA equivalent
22
Q
Example Friedman test
A
- Experiment: same as before but run over several days in a within-subject design manner
- Rank within each subject
- Sum within conditions
- If difference within conditions, sums are going to be different
23
Q
How do you conduct a Friedman test in SPSS?
A
- Analyse -> nonparametric tests -> k related samples -> select Friedman
- Drag test variables you need into box
24
Q
How do you test differences for a Friedman test?
A
- As with Kruskall-Wallis, the Friedman test does not tell us which conditions differ:
- Is praise is better than no reinforcement?
- Is criticism worse than no reinforcement?
- Is praise better than criticism?
- We can use multiple Bonferroni-corrected Wilcoxon tests
25
Q
What are some potential problems with non-parametric tests?
A
- Assumption-free(r) tests = non-parametric tests
- Concern = throwing away data by converting scores to ranks
- Size of the value may be important
- Possibly reducing power by throwing away information about participants
