Lecture 8: Non-parametric tests Flashcards
What does the chi-square tests?
whether there is a relationship between two categorical variables.
What leads to chi-squared test?
Q: What sort of measurement? A: Categorical (in this case counts or frequencies)
Q:How many predictor variables? A: One
Q: What type of predictor variable? A: Categorical
Q: How many levels of the categorical predictor? A: Not relevant
Q: Same or Different participants for each predictor level? A: Different
This leads us to and Chi-square test for independence of groups
IV and DV in chi-squared tests - (2)
One categorical DV (because of frequencies)
with one categorical IV with different participants at each predictor level
In chi-square since we are using categorical variables we can not use
mean or any similar statistic hence cannot use any parametric tests
In chi-square test when measuring categorical variables we are interested in
frequencies (number of items that fall into combination of categories)
What does chi-square compare?
observed frequencies from the data with frequencies which would be expected if there was no relationship between the two variables.
In chi-square test, participants is allocated to one and only one category such as - (3)
pass or fail,
pregnant or not pregnant,
win, draw or lose
What is assumptions of chi-square test? - (3)
Data values that are a simple random sample from the population of interest.
Two categorical or nominal variables. Don’t use the independence test with continuous variables that define the category combinations. However, the counts for the combinations of the two categorical variables will be continuous.
For each combination of the levels of the two variables, we need at least five expected values. When we have fewer than five for any one combination, the test results are not reliable
Since each participant is allocated to one category in chi-squared test each individual therefore
contributes to the frequency or count with which a category occurs
In chi-squared categorical outcomes, the null hypothesis is set
up on the basis of expected frequencies, four all four variable combinations, based on the idea that the variable of interest has no effect on frequencies
Example of scenario using chi-square
We have a list of movie genres; this is our first variable. Our second variable is whether or not the patrons of those genres bought snacks at the theatre. Our idea (or, in statistical terms, our null hypothesis) is that the type of movie and whether or not people bought snacks are unrelated. The owner of the movie theatre wants to estimate how many snacks to buy. If movie type and snack purchases are unrelated, estimating will be simpler than if the movie types impact snack sales.
We have a list of movie genres; this is our first variable. Our second variable is whether or not the patrons of those genres bought snacks at the theatre. Our idea (or, in statistical terms, our null hypothesis) is that the type of movie and whether or not people bought snacks are unrelated. The owner of the movie theatre wants to estimate how many snacks to buy. If movie type and snack purchases are unrelated, estimating will be simpler than if the movie types impact snack sales.
Is the Chi-square test of independence an appropriate method to evaluate the relationship between movie type and snack purchases? - (3)
We have a simple random sample of 600 people who saw a movie at our theatre. We meet this requirement.
Our variables are the movie type and whether or not snacks were purchased. Both variables are categorical.
But last requirement is for more than five expected values for each combination of the two variables. To confirm this, we need to know the total counts for each type of movie and the total counts for whether snacks were bought or not. = check later
We have a list of movie genres; this is our first variable. Our second variable is whether or not the patrons of those genres bought snacks at the theatre. Our idea (or, in statistical terms, our null hypothesis) is that the type of movie and whether or not people bought snacks are unrelated. The owner of the movie theatre wants to estimate how many snacks to buy. If movie type and snack purchases are unrelated, estimating will be simpler than if the movie types impact snack sales.
Diagram of contigency table in Chi-square and calculating row totals and colum and grand total - (7)
50 + 125 + 90 +45 = 310
75 + 175 + 30 + 10 = 290
50 + 75 = 125
125 + 175 = 300
90 + 30 = 120
45 + 10 = 55
310 + 290 = 600
We have a list of movie genres; this is our first variable. Our second variable is whether or not the patrons of those genres bought snacks at the theatre. Our idea (or, in statistical terms, our null hypothesis) is that the type of movie and whether or not people bought snacks are unrelated. The owner of the movie theatre wants to estimate how many snacks to buy. If movie type and snack purchases are unrelated, estimating will be simpler than if the movie types impact snack sales.
Diagram of contigency table in Chi-square of calculating eexpected counts
e.g., for action and snacks it would be column total (310) * row total (125) divided by grand total of 600 = 65
How to calculate chi-square test statistic? - (4)
- Calculate the difference from actual and expected for each Movie-Snacks combination.
- square that difference.
- Divide by the expected value for the combination.
- We add up these values for each Movie-Snacks combination. This gives us our test statistic.
Example of calculating chi-square from table
For this it would be 65.03
How to understand your test statistic from chi-squared? - (5) if you have test statistic of 65.03
- Set your significance level = .05
- Calculate the test statistic -> 65.03
- Find your critical value from chi-squared distribution table based on df & significance level
- Degrees of freedom: df (r – 1) x (C-1)
For the movie example this is; Df = (4-1) x (2-1) = 3 -> 7.815 - compare test statistic with critical level
65.03 > 7.82 so reject the idea that movie type and snack purchases are independent
Example of research question and hypothesis and sig level of chi-square test of independence- (4)
Research question:
Does the area of psychology that a person prefers depend on whether they would select a cat or a dog as a pet?
Hypotheses:
H0: The area of interest in psychology and type of pet preferred are independent of each other.
H1: The area of interest in psychology and type of pet preferred are not independent of each other. That is the primary area of interest in psychology depends on whether you prefer a cat or a dog.
Significance level: α = .05
Does the area of psychology that a person prefers depend on whether they would select a cat or a dog as a pet? - chi-square test of independence
Chi-square example we need to check the assumptions below - (2)
Independence
Each item or entity contributes to only one cell of the contingency table.
The expected frequencies should be greater than 5.
In larger contingency tables up to 20% of expected frequencies can be below 5, but there is a loss of statistical power.
Even in larger contingency tables no expected frequencies should be below 1.
Does the area of psychology that a person prefers depend on whether they would select a cat or a dog as a pet? - chi-square test of independence
Chi-square example we need to check the assumptions of The expected frequencies should be greater than 5.
What does it show? - (4)
Here we see that all the expected counts in the cat group and one expected count in the dog group are below 5.
We also have one in the cat group that is below 1.
So, SPSS has flagged that we have 60% of the expected counts falling below 5.
So assmption of expected frequencies greater than 5 is not assumed
If chi-square assumption that The expected frequencies should be greater than 5 is not satisfied then do - chi-square test of independence
We should use Fisher’s Exact Test which can correct for this.
Does the area of psychology that a person prefers depend on whether they would select a cat or a dog as a pet? - - chi-square test of independence
If assumptions were met (expected frequencies greater than 5) then.. report - (2)
A chi-square independence test was performed to examine whether there was a relationship between their area of studies in psychology and their preference for cats or dogs.
The relationship between these variables was not significant, χ²(4, N = 46) = 1.46, p = .834, so we fail to reject H0.
Are directional hypotheses possible with chi-square?
A.Yes, but only when you have a 2 × 2 design.
B.Yes, but only when there are 12 or more degrees of freedom.
C.Directional hypotheses are never possible with the chi-squared test.
D.Yes, but only when your sample is greater than 200.
A = only when you have 2 variables to compare and can’t do non-directional in chi-square have to use loglinear or goodness of fit tests
Example situations you can do chi-square directional and not possible - (5)
If we are just comparing pet preferences between males and females, we can make a directional hypothesis (2 x 2 – male/female, cats/dogs).
Males prefer cats or females prefer dogs.
However, when we start adding variables to the design it gets complicated.
If we wanted to compare drink preferences at different times of the day for students/lecturers, we couldn’t form a directional hypothesis.
This is because we have 3 main effects and several interactions to consider. We need to use loglinear analyses to do this.
Table scenario in which cats can be trained to dance more effectively with food or affection at reward - chi-squared test
Table scenario in which cats can be trained to dance more effectively with food or affection at reward - chi-squared test
what are the four categories? - (4)
- could they dance - yes
- could they dance - no
- food as reward
- affection as reward
Table scenario in which cats can be trained to dance more effectively with food or affection at reward - chi-squared test
highlight the frequencies for four categories
Table scenario in which cats can be trained to dance more effectively with food or affection at reward - chi-squared test
what do the rows give?
Row totals give frequencies of dancing and non-dancing cats
Table scenario in which cats can be trained to dance more effectively with food or affection at reward - chi-squared test
what do the columns give? - (2)
The column totals give frequencies of food and affection as reward
These are the numbers in each group
Loglinear analysis is a …. of chi-square
extension
Chi-square only analyses two variables at a time, whilst log-linear models
can determine complex interactions in multidimensional contingency tables with more than two categorical variables.
Loglinear is appropriate when
there’s no clear distinction between response and explanatory variables
think of
Think of chi-square like t-tests (2 groups) and log-linear like ANOVA (more than 2 groups).
Example of RQ, hypothesis and sig level of loglinear - (3)
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Hypotheses:
H0: Treatment, type of animal and improvements are independent of each other.
H1: Treatment, type of animal and improvements are associated with each other.
Significance level: α = .05
Assumptions of log linear - (2)
Independence
Expected counts > 5
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Checking assumption of expected counts - (3):
Here we have 3 things we are comparing: animal (cat/dog), treatment (yes/no) and improvement (yes/no) all of which are categorical.
We look and see that all of the expected counts are above 5.
So met assumption of independence and expected counts
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Here we have 3 things we are comparing: animal (cat/dog), treatment (yes/no) and improvement (yes/no) all of which are categorical.
In loglinear model selection it begins with - (2)
all terms present (all main effects and all possible interactions
main effects: Animal, Treatment and Improvement
interactions: Animal * Treatment, Animal * Improvement, Treatment * Improvement and Treatment* Animal* Improvement
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Here we have 3 things we are comparing: animal (cat/dog), treatment (yes/no) and improvement (yes/no) all of which are categorical.
In loglinear model selection after including all main effects and interactions then it - (4)
Remove a term and compares the new model with the one in which the term was present.
Starts with the highest-order interaction (including max number of variables/categories)
Uses the likelihood ratio to ‘compare’ models below:
If the new model is no worse than the old, then the term is removed and the next highest-order interactions are examined, and so on.
Model selection of loglinear - what does it show?
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Here we have 3 things we are comparing: animal (cat/dog), treatment (yes/no) and improvement (yes/no) all of which are categorical. - (3)
We can see that the model selection worked in a way that it first tried to remove the 3-way interaction.
However, we can see here that it * affected the fit of the model, so it was left in.
Since removing the highest-order interaction made a * difference to the fit of the model, we get a final model that is the saturated model (it contains all main effects and interactions).
Loglinear SPSS K way and Higher order effects what does it show?
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Here we have 3 things we are comparing: animal (cat/dog), treatment (yes/no) and improvement (yes/no) all of which are categorical. - (2)
we are using the likelihood ratio here because that’s how we compare the models to find the best fit
. We see that all main effects and interactions are significantly contributing to explaining the variance in the data
loglinear what does K represent and what does K = 1,2 and 3 represent? - (4)
K represents the level of the terms.
For example, K=1 would be the main effects,
K=2 would be our 2-way interactions and
K=3 is our 3-way interaction.
Loglinear SPSS - what does parameter estimates show?
Research question: Is the new treatment associated with improvements in health in cats and dogs?
Here we have 3 things we are comparing: animal (cat/dog), treatment (yes/no) and improvement (yes/no) all of which are categorical. - (3)
There is a significant three-way interaction between animal, treatment and improvement, as well as two significant two-way interactions between animal and improvement and treatment and improvement (p < .001)
a * 3-way interaction between animal, treatment and improvement as well as two * 2-way interaction between animal/improvement and treatment/improvement.
Like our post-hoc tests, this is telling us where the * differences are.
Loglinear after seeing statistical tests we go to raw data showing that…
Based on the raw data, there seems to be indication that the cats responded better to treatment than dogs, this should be followed up by chi-square tests separately for cats and dogs to determine whether the association between treatment and improvement is present in both cats and dogs
When conducting a loglinear analysis, if our model is a good fit of the data then the goodness-of-fit statistic for the final model should be:
A. Significant (p should be smaller than .05)
B. Non-significant (p should be bigger than .05)
C. Less than 5 but greater than 1
D. Greater than 5
B
The goodness of fit tests in log linear tests
hypothesis that frequencies predicted by model (expected frequencies) are sig different from actual frequencies in data (obsevered)
A significant goodness of fit result mean
our model was significantly different from our data (i.e., the model is a bad fit to the data).
A recent story in the media has claimed that women who eat breakfast every day are more likely to have boy babies than girl babies. Imagine you conducted a study to investigate this in women from two different age groups (18–30 and 31–43 years).
Looking at the output tables below, which of the following sentences best describes the results? = chi-square
A. Women who ate breakfast were significantly more likely to give birth to baby boys than girls.
B. There was a significant two-way interaction between eating breakfast and age group of the mother.
C. Whether or not a woman eats breakfast significantly affects the gender of her baby at any age.
D. The model is a poor fit of the data.
C
Chi square and log linear are both
non-parametric methods
Logic behind Wilcoxon’s rank sum test, what does SPSS do? - (3)
Step 1: Get some not normally distributed data
Step 2: Rank it (regardless of group)
Step 3: Significance testing
Does one of the groups have more of the higher ranking scores than the other?
For Mann-Whitney, the statistic U use the
sum of ranks for group 1, R1, as follows
Example of table where comparing 2 independent conditions of Wilcoxon rank sum or Mann Whitney U test
Here we have data for two groups; one taking alcohol, the other ecstasy. The scores for a measure of depression. Scores were obtained on two days; Sunday and Wednesday. The drugs were administered on Saturday.
Example of table where comparing 2 independent conditions of Wilcoxon rank sum or Mann Whitney U test
Here we have data for two groups; one taking alcohol, the other ecstasy. The scores for a measure of depression. Scores were obtained on two days; Sunday and Wednesday. The drugs were administered on Saturday.
Two steps for both statistics:
Rank all the data on the the basis of the scores irrespective of the group
compute the sum of ranks of each group - (5)
The graphic here shows how we can list the scores in order and as a result assign each score a rank.
When scores tie, we give them the average of the ranks.
If we ensure we keep track of the group the scores came from we can relatively easily add the ranks up for each group.
Note, that if there was little difference between the groups the sums of their ranks would be similar, as they are for the data shown her for Sunday.
However, the sum of ranks differ considerably for the data obtained on Wednesday.
For Wilcoxon sum of ranks = comparing 2 independent groups the W s statistic,
the group sizes of n1 and n2 the mean of W2 is given:
For Wilcoxon sum of ranks = comparing 2 independent groups the W s statistic,
the standard error of Ws is given
For Wilcoxon sum of ranks = comparing 2 independent groups the z score of Ws can be calculated
For Mann Whitney , the statistic U use the the sum of ranks for group 1, R1, as follows
For Mann Whitney , the statistic U use the the sum of ranks for group 1, R1, as follows
Specificy equation - (2)
The first terms involving n1 and n2 actually compute the maximum possible sum of ranks for group 1.
U is zero when all those in group one have scores that exceed the scores of those in group 2.
In Mann Whitney U there is a standardised test statistic which is z score that can allow you to compute
effect size so r = z / square root of N (number of pps
Example of Wilcoxon signed rank test carrying out steps - (9)
The table shown here has the Depression Scores taken on Sunday and Wednesday for those taking ecstasy on Saturday.
Data for Sunday are in the first column and Wednesday in the second column.
The third column shows the difference between scores obtained on Sunday and Wednesday.
NOte some could be negative, some positive. In this example however the difference is always positive apart from two values when the difference is zero.
The fourth column notes the sign of the difference or notes it is going to be excluded because the difference was zero.
The fifth column ranks the differences in terms of their size, but not sign.
The sixth and seventh column list the ranks that were for positive and negative differences, respectively.
It is these two columns that are summed to get the relevant statistics, called T+ and T-.
Because T+ and T- are not independent, we take only the T+ value.
For Wilcoxon signed rank test the group size n the mean T is given
For Wilcoxon signed rank test thestandard error fo T is given:
For Wilcoxon signed rank test compute z score of T by
For Kruskal-Wallis, the statistic H is as follows
For Friedman, the statistic F is as follows
K = conditions
N = number of pps
Example when using chi-square test - (3)
- In this example, they wanted to look at whether attendance at lectures had an impact on their exam performance on whether they passed or failed
- Attendence was coded as 1 if participants generally attended lectures , barirng illness, and 2 if they did not attend
- Exam was scored as 1 = Pass and 2 = Fail
- In this example, they wanted to look at whether attendance at lectures had an impact on their exam performance on whether they passed or failed - chi square
What does it show? - (4)
- Attendence, Attended Lectures , Count = this is people who attended lecture and number of people who passed was 84 and people who failed was 29
- % Within attendence give same info so 74.3% passed and 25.7% failed when attended lectures
-Going to didn’t attend lectures, 22 people passed and 35 failed and below is in percentages: - Easier using percentages writing up
- In this example, they wanted to look at whether attendance at lectures had an impact on their exam performance on whether they passed or failed - chi square
What does it show? - (2)
- At top row, pearson chi-squared which chi-square statistic which was 20.617, DF which is 1 and p-value was 0.000
- 0 cells have a count less than 5 met assumption of chi-square test that expected counts greater than 5
- In this example, they wanted to look at whether attendance at lectures had an impact on their exam performance on whether they passed or failed - chi square
What does it this effect size show? - (2)
- x^2 (1) = 20.62, p < 0.001
- Cramer’s V = 0.35 , indicating a medium effect size
Example of calculating odds ratio for chi-square - (3)
odds of passing/failing for students who attended lecture = no. of students who attended and passed (84) / no. of students who attedned and failed (29) = 2.897
odds of passing/failing for students who did not attend = no. of students who did not attend lectures and passed (22) / no of students who did not attended and fail (35)
Odds ratio = odds of P/F of attended/ odds of P/F of not attended = 2.897/ 0.629 = 4.606 saying for an individual who attended lectures lead them to be more likely to pass exam
Example research scenario of Mann Whitney - (4)
- Independent sample design
- One IV, two conditions = existing vs new medication
- One DV (symptoms) but this time on ordinal (scale from 1 to 5) and got combination of non-normally distributed data and small sample size (very problematic for t-tests)
- Mann Whitney U Test
Example of using Mann Whitney U = skew
What does this Mann Whitney U show?
- Independent sample design
- One IV, two conditions = existing vs new medication
- One DV (symptoms) but this time on ordinal (scale from 1 to 5) and got combination of non-normally distributed data and small sample size (very problematic for t-tests)
- Mann Whitney U Test
- This box summarises the p-value ( p = 0.026) and tells you whether to accept or reject the null hypothesis.
What does this output show of Mann Whitney U? - (3)
- Mann Whitney U test statistic is 166.000 to report and also people report standardised test statistic 2.292 which is z score so handy to report as know if its above +/-1.96 then p-value we get out of test is significant
- P-value of exact significant is p = 0.026
- This is significant difference between the 2 groups
- Next we would want to look at the median scores to see which group is scoring highest and lowest after sig Mann Whitney U test
What does this output show? - (3)
- Independent sample design
- One IV, two conditions = existing vs new medication
- One DV (symptoms) but this time on ordinal (scale from 1 to 5) and got combination of non-normally distributed data and small sample size (very problematic for t-tests)
- Mann Whitney U Test
- For existing treatment, median score was 3.
And new treatment the median score was 4.
It suggests new treatment was more effective in reducing symptons than the existing treatment
Example research scenario of Friedman ANOVA - (5)
- Again we got ordinal data for DV not sure distances between levels is going to be the same
- Related design
- One IV, 3 conditions
- One DV (level reached to video game)
- Friedman’s ANOVA = more than 2 groups in related design
What does this Friedman ANOVA output show?
- We got total sample size which is 30 and test statistic which is 21.788, DF = 2 and p value is 0.000 so significant difference between the 3 groups
A psychologist was interested in whether there was a gender difference in the use of email. She hypothesized that because women are generally better communicators than men, they would spend longer using email than their male counterparts. To test this hypothesis, the researcher sat by the computers in her research methods laboratory and when someone started using email, she noted whether they were male or female and then timed how long they spent using email (in minutes). How should she analyse the differences in males and females (use the output below to help you decide)?
A. Mann–Whitney test
B. Paired t-test
C.Wilcoxon signed-rank test
D. Independent t-test
A.Mann Whitney Test
