Testing methods Flashcards
When is Cohen’s W used?
Also explain how the effect sizes are distributed
To test the effect size of a Chi squared goodness of fit test
0.10: small, 0.30: medium, 0.5: large
What are the assumptions of a Chi-squared Goodness of fit test?
- The variable is nominal
- Observations are independent
- All groups have >= 5 observations
What is a Chi squared goodness of fit test?
Explain: use, formula, degrees of freedom
- Tests whether a sample is from a certain distribution
- ∑((O - E)^2 / E)
- Categories - 1
What is a Chi squared test of association?
Explain: Use, formula, degrees of freedom
- Tests whether two categorical variables are independent
2.∑((O - E)^2 / E)
Creating expected: (row amount * column amount) / N - (r-1) * (c-1)
When is Cramer’s V used?
To calculate the effect size of a Chi squared test of independence
Effect size:
0 – 0.15: very weak, 0.15 – 0.20: weak, 0.20 – 0.25: moderate; 0.25 – 0.30:
moderately strong, 0.30 to 0.35: strong, 0.35 to 0.40: very strong
Formula: sqr root (test statistic / N(k-1))
k is min(r,c)
What are the assumptions of a Chi squared test of independence?
Also name the tests used in case of violation
- Observations in each group are at least 5
violation: Fisher exact test (fisher.test()) - observations are independent
violation: McNemar test (mcnemar.test())
What is the problem with the z-test?
It assumes that the true population standard deviation is known, which is almost never the case.
When is Cohen’s D used?
To test the effect size of a t-test
What is the difference in assumptions between the Student t- test and Welch’s t-test?
Student t-test assumes homogeneity of variance (both variables have the same variance). This is almost never true. Welch does not have this assumption.
When is an independent samples t-test used?
To compare the means of two samples.
What is the formula for a Welch t-test?
mean1 -mean2 / Standard error, where SE = sqrt (sd1 / n1) / (sd2 / n2)
degrees of freedom: N-2
What are the assumptions of a one sample t-test?
- The observations are normally distributed
- The observations are independent
When is a paired samples t-test used?
To test the means between the same subject (such as same participants at a different time)
How can normality be tested?
- The Shapiro-Wilk test
- Histogram
- QQ plot
What are non-parametric alternatives for t-tests?
Wilcoxon tests
1. signed rank for one or paired samples
2. rank sum for independent samples
How can homogeneity of variance be tested?
Using Levene’s test.
What is ANOVA? And when is it used?
Analysis of variance, used to test more than two means.
How do you calculate ANOVA?
F-statistic = Mean square between groups (MSb) / Mean square within groups (MSw) (also called residuals)
Mean square = Sum of squares / degrees of freedom (DF)
DFb = groups - 1
DFw = N - G
When is eta squared used?
Also give the formula.
Calculate the effect size of an ANOVA.
Formula: SSb / SStot
(How much of the total Sum of squares can be attributed to the model?)
How do you find out which group is significant in an ANOVA?
And what is a big problem with this method? (also mention its solution)
Using a post hoc pairwise T-test
The problem: inflation of a type 1 error
Correction method: bonferroni method
Use TukeyHSD as post-hoc test
What are the assumptions of an ANOVA?
And how are these tested?
- Residuals are normally distributed checked by QQplot, histogram, Shapiro wilk
- Homogeneity of variance checked by Levene’s test
- Residuals are independent
What are non-parametric alternatives to ANOVA?
- Normality: Kruskal-Wallis rank sum test (kruskal.test())
- Homogeneity: Welch one-way test (oneway.test())
How is the between groups sum of squares calculated?
Squared difference between the group mean and true mean * the amount of observations in the group (N)
Do this for each group and add them together
How is the within sum of squares calculated?
For each observation, take the squared difference between the value and the group mean, and sum these all up
What is a crossed ANOVA design?
All combinations are represented.
What is a balanced ANOVA design?
Equal number of observations in each group.
What are marginal means?
The means of each row and column in an ANOVA table.
How is Sum of Squares for factor A calculated in a factorial ANOVA?
Sq diff between mean of factor A and overall mean, * number of observations in factor A.
With a balanced design, this is N * groups of factor A.
What are main effects?
The independent effects of independent variables
on the dependent variable.
What is an interaction in an ANOVA?
The effect of one independent variable depends on the effect of another independent variable.
In a lineplot, an interaction can be seen when the two lines have different slopes.
What are the degrees of freedom when testing for an interaction?
Both for the interaction itself, and the residuals.
Interaction: (r-1)(c-1)
Residuals: N - (R*C)
How is a two-tailed test statistic transformed into a one-tailed?
statistic / 2
What is the Yates continuity correction and when is it needed?
A correction method which is needed when a Chi square goodness of fit test only has one degree of freedom. To fix this, you subtract 0.5 from each observation
