L4 - Nonparametrics Flashcards
When we compare means, what type of data do we use?
Metric data (numeric)
What is a parametric test?
Parametric tests assume a normal distribution of values, or a “bell-shaped curve.”
What is a non-parametric test?
A non parametric test (sometimes called a distribution free test) does not assume anything about the underlying distribution
(for example, that the data comes from a normal distribution).
It usually means that you know the population data does not have a normal distribution.
“Analysing the difference between indigenous and non-indigenous smokers”
What type of data is this?
Categorical
How many people are using services, how many people do “x”.
Count type data - difference between two categories
What is a binomial test useful for?
Useful for tests with binary outcomes
What is a “proportion difference test”?
T-tests for proportions.
E.g. are men more likely to smoke than women
Are men more likely to report abuse than women etc
What is a “t-test”?
A statistical test to see whether there is a significant difference between two means.
What do non-parametric tests for related samples analyse (e.g. McNemar’s test and Cochran’s Q)?
Whether the probability of a binary outcome changes over time
e.g. pass or fail test; 40% fail on first try; only 20% fail on second try
What is a chi-square test used for?
A chi-square test is used to determine whether there is a statistically significant difference between the expected frequencies and the observed frequencies in one or more categories of a contingency table.
What are the two types of chi-square tests?
- Goodness of fit test (or one-way test - 1 independent variable)
- Contingency table/cross-tab (or two way analysis - 2 independent variables)
What does a ‘Goodness of Fit Test’ analyse?
Tests whether how well an observed distribution of scores matches a distribution expected by chance
Why it’s called a “goodness of fit test” - how good does the IV fit the distribution expected by chance
How many IV’s does a goodness of fit test have?
1
What does a distribution expected by chance look like (in a bar-graph)?
A rectangle
each variable is the same
Which test would you use to test - “8 starting gates on a race-track: are there more wins in some starting gates?”
Why?
Goodness of fit test.
There is only one IV (starting gates) and you are looking to see if there is any difference from chance.
If you observe three vending machines, and you observe 120 people. If based on chance, how many people would use one of those vending machines?
40
120/3 = 40
Explain the “chi-squared calculation” χ2 = Σ ( O – E )2 / E
Sum of [observed - expected frequency per level]; square the differences; and divide by the expected frequency. This is the Chi-Squared value χ2
χ2 = Σ ( O – E )2 / E
How do you know if a “chi-squared calculation” (χ2) is significant?
It must be greater than the critical value
If your value is greater than the critical value, it is significant
What is the critical value based on?
- The probability level you want to test at
- Degrees of freedom
In a goodness of fit test, what is degrees of freedom?
The number of levels (variables) minus 1
Df if 3 variables = 3-1 = 2
If the chi-squared calculation in a “goodness of fit test” is greater than the critical value, how would we interpret the results?
The observed distribution of scores differs significantly from what would be expected by chance.
E.g. some lanes are used disproportionately more than others in a way that differs from chance
What does a 2-way chi-squared test look at?
This is the ‘familiar test’ where one tries to determine whether there is an ASSOCIATION between 2 categorical /ordinal variables
It is a test of independence.
Are the variables independent (no effect) or interdependent (related/ an effect)
A 2-way chi-squared test is trying to determine if two variables are _____ or _____ of one another.
Independent (no effect on each other) or Interdependent (related to each other)
What test would we use to determine; “Is the person’s SES and ethnic group related?”
2-way chi-squared test
Is SES and ethnic groups independent or interdependent variables?
Other examples - “are smokers more likely to be drinkers” etc.
What is the 2-way chi-squared test formula?
[Row-total / Total N] * Column total
E.g. for Mach 1; [row total (180) - grand total (290)] times Mach 1 column total (90)
How is degrees of freedom calculated for a 2-way chi-squared test?
Df = (rows – 1)*(columns - 1)
For above example; 2 rows minus 1 = 1; 3 columns minus 1 = 2; 1*2 = Df = 2
When is a 2-way chi-square test significant?
If it is greater than the critical value
What does it mean if a 2-way chi-squared test is significant when interpreting results?
The two variables are NOT independent
They are interdependent
One assumption of a chi-squared test is minimum expected frequency
What % of cells can have a minimum expected frequency (MEF) of less than 5 in a 2-way chi-squared test?
No more than 25% of cells
MEF = number of values in a category
What test should you use instead of a 2-way chi-squared test if minimum expected frequency (MEF) is violated?
Kolmogorov-Smirnov test
For a 2-way chi-squared test what is the effect size statistic?
Kramer’s V
How do you determine the source of effect in a 2-way chi-squared test?
Inspection of standardised residuals
Source of effect can be discerned by comparing expected vs. observed frequencies in cells (standardised residuals).
Values of 2+ are large
χ2(df= 2, N = 50) = 18.69, p < .01. Is an example APA format of what type of test?
How would you write it up if it was a choice between genders and machines (1-3)?
2-way chi-squared test
“There was a significant association between gender and choice of machine”
“A greater proportion of females preferred machine 1, whereas males preferred machine 3”
In a chi-squared test, what is the standardised residual?
What is it’s formula?
A standardised score representing the residual leftovers. It is used to see which variable has the most effect.
Formula for standardised residual = O - E / √E
How are standardised residuals interpreted?
The standardized residual can be interpreted as any standard score.
The mean of the standardized residual is 0 and the standard deviation is 1.
Anything 2+ is high.
Standardized residuals are calculated for each cell in the design.
Why are standard residuals helpful for understanding the results of a chi-square?
They are useful in helping to interpret chi-square tables by providing information about which cells contribute to a significant chi-square.
Higher scores contribute more to a significant chi-square
What is a Kappa Test used for?
To examine inter-rater reliability in situations involving 2 independent raters
What test would you use to determine: “2 clinical psychologists are asked to assess 50 clients as having or not having clinical depression”. Why?
Kappa test
Examining the inter-rater reliability involving 2 independent raters
How do you calculate Kappa values?
K = [Po – Pe] / 1 – Pe
Where Po = Proportion of cases where there is agreement
Pe = Proportion of cases expected to agree by chance
Calculations are similar to chi-sq
In week 4 handout
What is a good and very good Kappa value (K)?
.5 = good
.7 = very good
.3 = fair - anything lower is bad
When does K (Kappa) tend to be unreliable?
If the base-rate for a phenomenon is very high or very low in the population
E.g. <5% or >95% in the population of interest
I.e If the vast majority are yes or no
When is a Fisher Exact Test used?
Used when chi-squared test is invalid due to violation of MEF assumption
When you have very small samples
How is a fisher exact test interpreted if the results are significant?
Same as a 2-way chi-square test.
The two variables are NOT independent
They are interdependent
What are binomial tests used for?
Binomial tests are used to test for the probability of binary outcomes
e.g., proportion of L vs. R turns, good vs. bad in a batch, clinical vs. non-clinical cases, left vs right-handers
What is a binomial test attempting to figure out?
Does the probability obtained for one event (e.g., Right) differ from what is expected by chance?
What is the most common binomial test?
The Z-test
Or z-approximation to binomial
What is a z-test.
Z-test, tests the means of 2 population distributions are different, when we already know the population variance.
Must have sample size (N > 30)
A statistical test that assumes a normal distribution for the null hypothesis.
What is the formula for a z-test?
Z = X – Mean / SE
X = observations of binary outcome
M (mean) = Number expected by chance
SE = standard error = √n.p.(1-p), where p = probability of the event
N = number of tests
Observed count (number of observations)
Expected or Mean = number * probability
In given no other information = .5
(person equally likely to turn left or right)
100 people, Mean = 100 x .5 = 50
What test would you use to determine if people tend to ignore a stop sign?
Binomial
2 answers - ignores or stops (50-50)
What test would you use to determine if “drug use is increasing in the office”?
Is it significant?
Binomal
An increase of 5% from 5-10% is therefore significant
What is a proportion difference (PD) test?
A parametric test that examines whether two samples are drawn from populations with the same proportions of a given characteristic
i. e. It tests the hypothesis that 2 sample proportions are, or are not, drawn from the same population.
* Similar to a t-test (t-test for proportions)*
* One of the most important and useful tests*
What test should you use to calculate whether “males are more likely to smoke than females” or “are port supporters likely to drive Fords than Crows supporters”?
Proportion difference test
Looking to see whether the distribution of proportions for the DV of the two different populations are different
Why should you use a proportion difference test instead of a chi-squared test?
Easier to interpret (you get a Z score and a test statistic)
Difference between proportions is clear
Can calculate a confidence interval
How do you calculate a proportion difference test?
P’ = the overall probability of the event or quality (e.g., smoking in the whole sample)
If E1 = instances in sample 1 (n1) and;
E2 = instances in sample 2 (n2)
Then p’ = [E1+E2] / [n1 + n2]
When can a proportion difference test be used?
It can be used instead of the non-parametric tests described above when the sample size is sufficient.
Usually when the two sample sizes n1 and n2, are such that n1 or n2.p, or n1 or n2.(1-p) are all > 5.
Using this proportion difference test data. Calculate the Z-value to see whether the difference in values is significant
How would you interpret/report the results?
To report this you would say “there was a significant difference in smoking between males and females.
How would you measure repeated non-parametric tests?
Use a Cochran’s Q to measure whether there is a difference across time, and then a McNemar test to test differences between specific time points
What does a McNemar test examine?
Examines whether the probability of a given event is more or less likely in the same sample over time
When would you use a McNemar Test?
If you have two related samples with repeated measures in categorical data analysis
What does Cochran’s Q measure?
Is the probability of an event less likely in the same sample of people when assessed at a different point in time or in relation to a different choice
What values do both a McNemar test and a Cochran’s Q give you?
A test statistic and associated probability value
How would you measure (what tests) whether clients using a CBT program get better over time?
You would use a Cochran’s Q to measure whether there is a difference across time when using CBT.
Then you would use the McNemar test to compare individual time points (see which time points are most significant)
Why wouldn’t you use a chi-squared test for measuring whether clients using a CBT program get better over time?
- Can’t do a Time x Outcome chi-squared test because the data are measured at different points in time
- The data are non-independent (a fundamental assumption of chi-squared test) as they are collected from the same people
If results for running a repeated measure non-parametric test using Cochran’s and McNemar test are significant. How would you interpret the results?
What is a unstandardised residual?
What is it’s formula?
In a 2-way chi-square test; the standardised residual is the simple difference of the observed and expected values
Unstandardized residual = O - E
