Testing methods

Question 1

When is Cohen’s W used?

Also explain how the effect sizes are distributed

Answer 1

To test the effect size of a Chi squared goodness of fit test

0.10: small, 0.30: medium, 0.5: large

Question 2

What are the assumptions of a Chi-squared Goodness of fit test?

Answer 2

• The variable is nominal
• Observations are independent
• All groups have >= 5 observations

Question 3

What is a Chi squared goodness of fit test?

Explain: use, formula, degrees of freedom

Answer 3

1. Tests whether a sample is from a certain distribution
2. ∑((O - E)^2 / E)
3. Categories - 1

Question 4

What is a Chi squared test of association?

Explain: Use, formula, degrees of freedom

Answer 4

1. Tests whether two categorical variables are independent
   2.∑((O - E)^2 / E)
   Creating expected: (row amount * column amount) / N
2. (r-1) * (c-1)

Question 5

When is Cramer’s V used?

Answer 5

To calculate the effect size of a Chi squared test of independence

Effect size:
0 – 0.15: very weak, 0.15 – 0.20: weak, 0.20 – 0.25: moderate; 0.25 – 0.30:
moderately strong, 0.30 to 0.35: strong, 0.35 to 0.40: very strong

Formula: sqr root (test statistic / N(k-1))
k is min(r,c)

Question 6

What are the assumptions of a Chi squared test of independence?

Also name the tests used in case of violation

Answer 6

• Observations in each group are at least 5
  violation: Fisher exact test (fisher.test())
• observations are independent
  violation: McNemar test (mcnemar.test())

Question 7

What is the problem with the z-test?

Answer 7

It assumes that the true population standard deviation is known, which is almost never the case.

Question 8

When is Cohen’s D used?

Answer 8

To test the effect size of a t-test

Question 9

What is the difference in assumptions between the Student t- test and Welch’s t-test?

Answer 9

Student t-test assumes homogeneity of variance (both variables have the same variance). This is almost never true. Welch does not have this assumption.

Question 10

When is an independent samples t-test used?

Answer 10

To compare the means of two samples.

Question 11

What is the formula for a Welch t-test?

Answer 11

mean1 -mean2 / Standard error, where SE = sqrt (sd1 / n1) / (sd2 / n2)

degrees of freedom: N-2

Question 12

What are the assumptions of a one sample t-test?

Answer 12

• The observations are normally distributed
• The observations are independent

Question 13

When is a paired samples t-test used?

Answer 13

To test the means between the same subject (such as same participants at a different time)

Question 14

How can normality be tested?

Answer 14

• The Shapiro-Wilk test
• Histogram
• QQ plot

Question 15

What are non-parametric alternatives for t-tests?

Answer 15

Wilcoxon tests
1. signed rank for one or paired samples
2. rank sum for independent samples

Question 16

How can homogeneity of variance be tested?

Answer 16

Using Levene’s test.

Question 17

What is ANOVA? And when is it used?

Answer 17

Analysis of variance, used to test more than two means.

Question 18

How do you calculate ANOVA?

Answer 18

F-statistic = Mean square between groups (MSb) / Mean square within groups (MSw) (also called residuals)

Mean square = Sum of squares / degrees of freedom (DF)

DFb = groups - 1
DFw = N - G

Question 19

When is eta squared used?

Also give the formula.

Answer 19

Calculate the effect size of an ANOVA.
Formula: SSb / SStot

(How much of the total Sum of squares can be attributed to the model?)

Question 20

How do you find out which group is significant in an ANOVA?

And what is a big problem with this method? (also mention its solution)

Answer 20

Using a post hoc pairwise T-test
The problem: inflation of a type 1 error
Correction method: bonferroni method
Use TukeyHSD as post-hoc test

Question 21

What are the assumptions of an ANOVA?
And how are these tested?

Answer 21

• Residuals are normally distributed checked by QQplot, histogram, Shapiro wilk
• Homogeneity of variance checked by Levene’s test
• Residuals are independent

Question 22

What are non-parametric alternatives to ANOVA?

Answer 22

• Normality: Kruskal-Wallis rank sum test (kruskal.test())
• Homogeneity: Welch one-way test (oneway.test())

Question 23

How is the between groups sum of squares calculated?

Answer 23

Squared difference between the group mean and true mean * the amount of observations in the group (N)

Do this for each group and add them together

Question 24

How is the within sum of squares calculated?

Answer 24

For each observation, take the squared difference between the value and the group mean, and sum these all up

Question 25

What is a crossed ANOVA design?

Answer 25

All combinations are represented.

Question 26

What is a balanced ANOVA design?

Answer 26

Equal number of observations in each group.

Question 27

What are marginal means?

Answer 27

The means of each row and column in an ANOVA table.

Question 28

How is Sum of Squares for factor A calculated in a factorial ANOVA?

Answer 28

Sq diff between mean of factor A and overall mean, * number of observations in factor A.

With a balanced design, this is N * groups of factor A.

Question 29

What are main effects?

Answer 29

The independent effects of independent variables
on the dependent variable.

Question 30

What is an interaction in an ANOVA?

Answer 30

The effect of one independent variable depends on the effect of another independent variable.

In a lineplot, an interaction can be seen when the two lines have different slopes.

Question 31

What are the degrees of freedom when testing for an interaction?

Both for the interaction itself, and the residuals.

Answer 31

Interaction: (r-1)(c-1)
Residuals: N - (R*C)

Question 32

How is a two-tailed test statistic transformed into a one-tailed?

Answer 32

statistic / 2

Question 33

What is the Yates continuity correction and when is it needed?

Answer 33

A correction method which is needed when a Chi square goodness of fit test only has one degree of freedom. To fix this, you subtract 0.5 from each observation