L8: Nonparametric testing

Question 1

When should you use nonparametric tests?

Answer 1

• When assumptions are violated
  
  • e.g. strong non-normality
• When the variable is ordinal
  
  • e.g. when playing Mario Kart
• When unsure about outliers

Question 2

Parametric vs Nonparametric
Differences?

Answer 2

Look at figure 1.
Important note: Nonparametric tests aren’t capable of handling non-random samples.

(Last row isn’t as important because we now have computers that can handle everything.)

Question 3

Whats the most common solution to analyse weirdly distributed data?

Answer 3

Ranking the data
All nonparametric tests involve ranking to overcome distributional problems.

Question 4

What is ranking?

Answer 4

A way of handling data that allows you to deal with extreme data.
You assign ranks to the data, going from lowest score to the highest score.
This means that you can handle outliers well.

Look at figure 2 to see what I mean
The last column has the same ranking despite very different scores on each variable.

Question 5

How do you deal with ties when ranking individuals?

Answer 5

Find the mean ranking of the individuals that have tied.
Look at figure 3 for an example.

Question 6

What is the general procedure for nonparametric tests?

Answer 6

1. Assumption: Independent random samples
2. Hypothesis
   
   1. H0: equal population distributions (implies equal mean ranking)
   2. HA: Unequal mean ranking (two sided)
   3. HA:Higher mean ranking for one group
3. Test statistic is difference between mean or sum of ranking
4. Standardize test statistic to normal sampling distribution
5. Calculate P-value one or two sided
6. Conclude to reject H0 if p < alpha

Question 7

What is the Wilcoxon rank-sum test

Answer 7

Nonparametric version of independent 2 sample t-test
Also known as the Mann-Whitney U test

Question 8

Whats the main expectation of a Wilcoxon rank-sum test

Answer 8

By ranking all values and then summing the ranks per group, one would expect under the null hypothesis that the sum of ranks is approximately equal

Question 9

The next few flashcards are how to perform a rank-sum test. It’s kind of confusing so stay with me.
(flip the card for good news)

Answer 9

JASP does this all for you, so just try and comprehend how the test works, rather than memorise it.

Question 10

After summing the ranks per group, How do you find W?

Answer 10

Pick the group with the lowest sum rank score.
If the group sizes are unequal, W is the group with the smallest amount of people.

Question 11

JASP produces a value called U. It is W - W.min
W.min = the minimum value of W.
How do you find W.min?

Answer 11

The minimum value depends on the sample size.
If each group has 10 participants, the lowest possible sum ranked score would 1+2+3…+10 = 55.

So, W.min = sum(1:n), with n being your group size.

Question 12

How do you find the Mean W under the null hypothesis?

Answer 12

Look at figure 4.
It depends on your sample size. H0 assumes that the sum rank score will be equal, so do some maths trickery, and shabang.

Question 13

Besides the formula in figure 4, how else could you find the Mean W?

Answer 13

You’ve found W.min, if you find the maximum value of W, you can average those two values and get the mean.
To find the max value of W, do the same as W.min, but with the highest ranks (11:20 in the case of 2 groups of 10.)

Question 14

Almost there, calculate the standard error of W, (so standard deviation thingymabob)

Answer 14

Look at Figure 5.
(two points.
1. I don’t know why the bottom is 12, johnny didnt say, the book didnt say, it just is
2. You don’t need to know the formula, i’m just going step by step)

Question 15

Alas, you have the mean W, the SE of W, and the W of your data (being the lowest sum rank score)
What do you do now?
Hint, block 1 stats is coming to haunt you

Answer 15

Thats right! Calculate the Z-score!
Figure 6 shows you it.
Its the same as if you were doing it with the mean value of a t-test, but instead of the mean, its W
Pretty nifty, right?
(end my suffering)

Question 16

What do you do with your prized Z-score?

Answer 16

Calculate the p-value to see if your data is significant.
Make sure to specify if its one sided or two sided based on your hypothesis.
If its significant, you’re done!

Question 17

Hol’up, did you seriously forget to calculate the effect size… -.-
Do better.
Whats the name of the effect size you use. How do you calculate it
(IMPORTANT)

Answer 17

Rank-biserial correlation
Formula 7
(You don’t need to know how to calculate it, but remember its name and what type of number it is)

It formats your W as a correlation.
Therefore, the closer to -1 or 1 it is, the higher the effect is.

Question 18

How can you convert your rank biserial correlation into a percentage of support?

Answer 18

((1+rbs)/2) x 100% , e.g. an effect size of 0.29 = 65% of the ranked data support the idea that …

Question 19

How robust is W?
Are nonparametric tests good?

Answer 19

Look at figure 8
Shits robust as fuck
No matter how you fuck with the data, it dont change the W.

Question 20

Tips and tricks for JASP when doing a rank-sum test

Answer 20

Use a raincloud plot, they are useful, they show you the differences between groups clearly.
Q-Q plots allow you to see if the data is distributed weirdly.
Look at the repeated p-value, is it significant?

Question 21

How to write the results for rank-sum tests

Answer 21

Report only the test statistic and its significance, and include effect size and exact values of p
E.g.
Depression levels in ecstasy users did not differ significantly from alcohol users the day after the drugs were taken, U = 64.50, p =0.29, rbs = 0.29, 95% CI [-0.22, -0.67]. However, by Wednesday, ecstasy users were significantly more depressed than alcohol users, U = 96, p <.001, rbs = 0.92, 95% CI [0.79, 0.97].

Question 22

Wilcoxon Rank-sum test Notation summary

Answer 22

W: Base test statistic - Lowest summed rank score
W.min: Lowest possible summed rank score - Dependent on sample size
W-hat: Mean W under H0
SEw-hat: Standard error of W
U: W - W.min
rbs: rank biserial correlation - effect size

Question 23

What is the Wilcoxon signed-rank test

Answer 23

A nonparametric alternative to the paired samples t-test.
It assigns + or - signs to the difference between two repeated measures.
By ranking the absolute differences and summing these ranks for the positive group, the null hypothesis is tested that both positive and negative differences are equal.
The test statistic is T+ (the sum of positive ranks)

Question 24

What does a high sum of positive ranks indicate?

Answer 24

It indicates that most of the individuals’ second measure score decreased from the first measure.
T+: Score decreased at second measure
T-: Score increased at second measure

Question 25

Why do you calculate both the difference scores and the absolute difference scores.

Answer 25

You use the absolute diff. scores to rank the individuals.
You use the normal diff. scores to see whether its a ‘+’ or a ‘-‘.

Question 26

Why do you remove scores that have no difference?

Answer 26

It won’t affect our sum positive score, but it will affect the sample size, making the mean T larger, weakening your effect.
By removing them, you only analyse the proportion of affected scores.

Question 27

What are all the values needed to standardize T?

Answer 27

Look at figures 9 to 11.

Question 28

Whats the name of the effect size?

Answer 28

Matched rank biserial correlation - Figure 12
It’s similar to the rank biserial correlation, but here it is literally the effect expressed as a proportion of the total of all the ranks.

Again, its a correlation, so interpret it as you normally would
You can convert it into a percentage of support if you please (look at flashcard 18)

Question 29

How do you write the results of a signed-rank test?

Answer 29

Report the test statistic (denoted by the letter T+), its exact significance and an effect size.
E.g.
For ecstasy users, depression levels were significantly higher on Wednesday than on Sunday, T+ = 0, p = 0.014, rsb = 1. For alcohol users, no significant difference was found between depression levels on Wednesday and Sunday, T = 8, p = 0.052, rbs = -0.71, 95% CI [-0.92, -0.18].

Question 30

JASP

Answer 30

Perform a paired samples t-test as usual.
Then just click the Wilicoxon signed-rank option under Tests.

Question 31

Wilcoxon signed-rank test Notation summary

Answer 31

T+: Sum of positive rank scores
T-: Sum of negative rank scores
T-hat: T mean - middle point for T, expectation under H0
SEt: Standard error of T
r: Matched rank-biserial correlation

Question 32

What is a Kruskal-Wallis test?

Answer 32

Independent >2 samples
Nonparametric version of independent one-way (Between-subjects) ANOVA
It essentially subtracts the expected mean ranking from the calculated observed mean ranking which is Chi^2 distributed
You rank them exactly the same as Wilcoxon

The test statistic is ‘H’

Question 33

What values do you need to calculate H?

Answer 33

Look at figure 13.
N: Total sample size
ni: sample size per group
k: number of groups
Ri: rank sums per group

Not needed for H itself, but to use it
Degrees of freedom: k-1

Question 34

What follow up would you do after you’ve tested H?

Answer 34

A Dunn’s Post Hoc test
Since you have multiple groups, you need to assess which group causes a difference, etc.
JASP performs pairwise Wilcoxon rank-sum tests for you, so it gives rbs as an effect size

Question 35

JASP info for Kruskal-Wallis test

Answer 35

You can look at the Q-Q plot, see if the line is weird.
Run a regular ANOVA, then go all the way down to the nonparametric test.
Enable effect size, look at ε

Enable Dunn’s Post Hoc test, it compares all possible pairs of groups with a p-value that is corrected so that the error rate across all tests remains at 5%.
When doing post-hoc tests, look at the effect size (rbs) to avoid issues.
Also never look at regular p, look at Bonferroni’s p or Holm’s p. They’ve been drinking water, they’ve got clear p.

Question 36

Writing the results from a Kruskal-Wallis test

Answer 36

Report the test statistic (denoted by H), its degrees of freedom, and its significance.
You can report just the main test, but since it doesn’t tell you which conditions are different, include the follow-up results.
E.g.
Testosterone levels were significantly affected by eating soya meals, H(3)= 8.66, p = 0.034.

Pairwise comparisons with Holm adjusted p-values showed that there were no significant differences between testosterone levels when people ate 7 soya meals per week compared to 4 meals (p = 0.133, rbs = 0.44), 1 meal (p = 0.133, rbs = 0.39) , or no meals (p = 0.058, rbs = 0.48). There were also no significant differences in testosterone levels between those eating 4 soya meals per week and those eating 1 meal (p = 1.00, rbs = 0.02) and no meals (p = 1.00, rbs = 0.06). Finally, there were no significant differences in testosterone levels between those eating 1 soya meal per week and those eating none (p = 1.00, rbs = 0.04).

This is very long because it discusses 4 pairwise tests, i doubt we’d do the same

Question 37

Kruskal-Wallis test Notation summary

Answer 37

H: Test statistic
N: Total sample size
ni: sample size per group
k: number of groups
Ri: rank sums per group

ε: Effect size for main test
rbs: Effect size for Post hoc tests

Question 38

What is Friedman’s ANOVA?

Answer 38

Paired >2 samples
Nonparametric version of repeated one-way ANOVA
Compares differences between several related groups
Like Kruskal-Wallis, this test subtracts the expected mean ranking from the calculated observed mean ranking, which is also chi^2 distributed.

Fr: Test statistic

Question 39

How do you rank people in Friedman’s ANOVA?

Answer 39

Rank within each participant
Sum the ranks for each of the conditions

Question 40

What do you need to calculate Fr

Answer 40

Look at figure 14.
N = total number of subjects
k = number of groups
Ri = rank sums for each group
df = k-1
Test for significance with Chi-square distribution

Question 41

What effect size do you look at for the main test?

Answer 41

Look at Kendall’s W (coefficient of concordance).
It quantifies the similarity between participants’ scores.
It has a limited range: From 0 (no similarity between participants) to 1 (complete similarity between participants).

Question 42

What post hoc test do you do?

Answer 42

Conover tests (JASP performs the Wilcoxon signed-rank tests for you to compare the groups)

Look at the effect size (rbs) to see how different each group is

Question 43

How can you see the sum of ranks for each group?

Answer 43

Go to the Conover test table.
Wi is the sum of the first group being compared, Wj is the sum of the second group being compared.
E.g. If t0 and t1 are being compared, Wi is t0’s sum, and Wj is t1’s sum

Question 44

How do you write the results for Friedman’s ANOVA?

Answer 44

Report the test statistic (Fr), its degrees of freedom, and its significance.
e.g.
The weight of participant did not significantly change over the two months of the diet, Fr (2) = 0.20, p = 0.91. Wilcoxon tests were used to follow up this finding. It appeared that weight didn’t significantly change from the start of the diet to one month, T = 0.21, rbs = -0.02, from the start of the diet to two months, T = 0.43, rbs = -0.09, or form one month to two months, T = 0.21, rbs = -0.06.

Question 45

Friedman’s ANOVA Notation summary

Answer 45

Fr: Test statistic
N = total number of subjects
k = number of groups
Ri = rank sums for each group
df = k-1