RMB: Independent t-test & Mann-Whitney WEEK 7

Question 1

Conditions to use Independent t-test

Answer 1

• Unrelated/independent/between groups > Looks at difference between two distinct groups. e.g. is there a difference between male and female scores on a numerical ability test?
• Two conditions to be compared
• Interval or ratio data needed
• Design must be unrelated (i.e. independent groups/between subjects
• Parametric assumptions must be met (see previous notes)
• Independent t-test compares the means of 2 independent groups

Question 2

Example of independent t-test

Answer 2

• Context: Imagine that we wanted to test the prediction that women will score higher on a verbal reasoning test than men. We perform a verbal reasoning test and find a difference in the means with men scoring higher than women.
• Two possibilities: 1. any difference is result of sampling variability (i.e. null hypothesis is true and two sets of scores come from the same population). 2. OR It may be that our difference reflects a real difference between the two underlying populations
• We want to know if our sample is part of a larger sample (where there is a significant difference) or if our sample is part of the general population (non-significant difference)
• Procedure: Perform an independent t-test if conditions are met using SPSS > Levene’s test for homogeneity of variance must be non-significant to use the independent t-test. If the variance test IS significant, then equal variance not assumed output could be used if everything else holds up like it is normally distributed (if homogeneity of variance is not there then P increases making it harder for it to be significant), but ideally use a non-parametric test instead > both are fine if you justify why

Question 3

Report independent t-test in APA

Answer 3

• To report an independent t-test, you need the P value, T value and df and also the mean + SD of values E.g. Descriptive statistics
• e.g. “There was a significant difference between the groups (t(14)=2.409, p=0.03). The female group scored higher (M = 114.3, SD = 10.6) than the male group (M=103.1, SD = 7.6).”

Question 4

Mann-Whitney Test conditions

Answer 4

• Non-parametric test > The Mann Whitney U test is the non-parametric equivalent of the independent samples t-test.
• Use if: You want to draw inferences regarding group differences, Groups are independent of one another – participants only contribute to one data sample, Your data is non-parametric + Ordinal level data or above (interval/ratio)

Question 5

Mann-Whitney assumptions

Answer 5

• Initially assume null hyp so both samples are from the same population
• The Mann-Whitney test assumes that, under the alternative hypothesis the probability of an observation from one population (X) exceeding an observation from the second population (Y) (after exclusion of ties) is more than 0.5 > probability of x being greater than y (50%) is the same as the probability of y being greater than x (other 50%)
• if null hyp is true, if you take a random person from X and Y, 50% of the time X will score higher than Y and 50% of the time Y will score higher than X > same pop
  -The alternative may also be stated in terms of a one-sided test, for example: P(X > Y) + 0.5 P(X = Y) > 0.5. Or P(X=Y) <0.05 > aka X is greater than Y or Y is greater than X > not equal
• the probability of X=Y needs to be less than 0.5 to be significant difference > chance of X + Y being from the same pop is less than 0.05
• The test involves the calculation of a statistic, usually called U, whose distribution under the null hypothesis is known. In the case of small samples, the distribution is tabulated, but for sample sizes above 20 there is a good approximation using the normal distribution.
  SPSS uses virtual tables to find P for U depending on N

Question 6

Example of Mann-Whitney Test

Answer 6

• Context: Same rationale as the above for independent t-test
• Procedure: If the sample is under 20 then the exact significance p value can be used otherwise use the normal significance value. Z score can be helpful in finding effect size
• Reporting in APA: To report an independent Mann Whitney U test you need the U value and ideally the Z value and the p value as well as the median
  Median
  Male 104.5 Female 111.5
• A Mann-Whitney U test was undertaken. The female group (Mdn = 111.5) did not score significantly higher than the male group (Mdn = 104.5) in a verbal reasoning test (U=15, Z=-1.8, p=0.072).
  -There can be instances where the Levene’s test is met but the K-S test is significant (over 0.05) > here we should really use a non-parametric test although sometimes do continue to use it
  -BUT when writing up a report you must ALWAYS justify why you have used a certain test!
  E.g.: A Mann-Whitney test was used for analysis. This test was utilised because a comparison was being made between two independent groups and it was not the case that both samples were normally distributed (K-S test: p<0.05 for male group)

Question 7

Calculating effect size

Answer 7

• R is effect size and N is total number of observations
• R = Z score divided by the square root of N > R = Z/√N
• E.g. R = -1.8/√16 (square root 16 first, = 4) > =-1.8/4 = -0.45
• Types of effect sizes: Large R = > 0.5 (bigger than 0.5). Medium R = > 0.3 (bigger than 0.3). Small R = < 0.3 (less than 0.3)
• Our effect size here is a medium effect size because it is bigger than 0.3 but smaller than 0.5
  Ignore the sign (e.g. +0.4/-0.4) when calculating effect size