Chi-square tests

Question 1

What are hypotheses?

Answer 1

Testable statements (not questions) which predict a relationship between variables
Variables need to be named precisely and consistently

Question 2

What are two assumptions made by parametric tests, and what happens when these are violated?

Answer 2

Assume normal distribution and at least interval level data

Violation leads to erroneous interpretation of data

Question 3

When would you use a non-parametric test?

Answer 3

When you don’t have a normal distribution, and when you have categorical (nominal/ordinal) data

Question 4

Why should non-parametric tests be used with care?

Answer 4

Not as powerful as parametric tests and can fail to detect some differences especially when sample size is low. Need to ensure large sample to detect smaller effects and demonstrate significance

Question 5

What type of test is chi-square? What assumptions are made when using it?

Answer 5

Non-parametric
Categorical data (coded as numbers), frequencies in each category
Assumes categories mutually exclusive and independent i.e. participants must only be in one
Expected frequencies must be >5 in each cell of contingency table

Question 6

How can the chi-square test of association/independence be used?

Answer 6

To investigate whether two variables are associated e.g. does gender influence smoking frequency?
Compare observed frequencies to expected frequencies predicted from the null hypothesis

Question 7

What is the calculation used to find expected frequencies?

Answer 7

Expected frequency = (column frequency total x row frequency total)/total sample size

Question 8

What is the formula for the chi-square calculation?

Answer 8

chi-square = SUM(((observed frequencies - expected frequencies) squared)/expected frequencies)
A bigger chi-square value represents greater divergence from the null i.e. stronger association

Question 9

In order to assess significance of our chi-square value we need two things: our degrees of freedom and our alpha value, and we can use these to find the CRITICAL VALUE which our obtained value needs to exceed to be significant. How do you calculate degrees of freedom for association tests?

Answer 9

df = (R-1) x (C-1) where R = rows and C = columns

Question 10

When would we use the Chi-Square Goodness of Fit test?

Answer 10

On UNRELATED data i.e. where every participant yields data for one single category and we are comparing different levels of ONE VARIABLE
Can be used to compare PROPORTIONS of a population distribution e.g. if there is a gender bias in the computing department

We essentially want to see whether data depart significantly from a theoretical distribution, often one that has more than just two theoretical values i.e. not just 50:50

Question 11

How do we calculate the goodness of fit?

Answer 11

We once again use observed and expected frequencies - observed is the number of participants measured in individual categories e.g. gender categories, while expected is the frequencies predicted by the null hypothesis (these can vary depending on specific null i.e. are we finding whether the genders are equal, in which case expected would be 50:50, or are we interested in whether the ratio in the one department is representative of the population ratio, in which case we would have to know population proportions and multiply the sample group by those proportions)

Question 12

What is the Goodness of Fit formula?

Answer 12

Same as for association

Question 13

Why do we divide by expected frequency?

Answer 13

So chi-square value is not influenced by variation in the size of the expected frequencies

Question 14

How do we calculate the degrees of freedom for goodness of fit tests?

Answer 14

df = C -1 where C is the number of categories

Question 15

In a goodness of fit test, what happens if we have more than 2 categories?

Answer 15

We cannot determine exactly which group difference meant that the chi-square value was significant and thus we simply say that obtained frequencies differed from those expected by the null hypothesis

Question 16

When would we use a Binomial test?

Answer 16

When a measurement classifies items into one of TWO categories and we are dealing with data in the form of frequencies (similar to chi-square but chi can have more than two categories)

Question 17

What are two assumptions made by the Binomial test?

Answer 17

Independence of observations and assumes each category has at least 10 values in it

Question 18

What kind of question would be answered by a Binomial test?

Answer 18

.”Does the proportion of counts in a coin toss falling into group A differ from the proportion in group B?”
Probabilities associated with each group can be considered p and q, and we say that, assuming the null hypothesis to be true, p=p(heads)=0.5 and q=p(tails)=0.5
If p=0.8 and q=0.2, under a p-value <0.05, we can say that the coin is biased

Question 19

When can the Sign Test be used?

Answer 19

To compare two conditions in a REPEATED MEASURES design
Doesn’t measure how much change has occurred but does demonstrate DIRECTION e.g. better/worse, increase/decrease
Calculation procedure is the same as for the binomial test

Question 20

How do you calculate expected frequencies in a one-row chi-square analysis?

Answer 20

N/k where N is total number of cases and k is the number of cells to average across

Question 21

How can effect size be estimated for 2x2 chi-square analyses?

Answer 21

Using the Phi coefficient/Cramer’s V
Cohen produced some effect size conventions for Cramers V that depend on degrees of freedom e.g. if df=1, a small effect would be 0.1, medium would be 0.3 and large would be 0.5

Question 22

When can a goodness of fit test sometimes be used?

Answer 22

To estimate whether a large sample approximates to a normal distribution (a requirement for some significance tests)
This involves calculation of z-scores
We can see whether observed frequencies depart from the ideal frequencies to a significant degree

Question 23

What can you do if you have unavoidably low frequencies in a chi-square?

Answer 23

Use Fisher’s Exact Test which calculates probability straight from frequencies

Question 24

How could you avoid low expected frequencies?

Answer 24

1) Avoid low samples for one category e.g. be wary of gathering data on left-handers
2) Avoid low samples overall - try to aim for above 20
3) Obtain significance at p less than or equal to 0.01 - decision to reject the null at this level is far more secure, far less chance of type 1 errors

Question 25

What must always be done when interpreting chi-square analyses?

Answer 25

ALWAYS use two-tailed values for chi-square even if the hypothesis was directional