Term 2 L7: Chi square and Contingency Tables Flashcards
Two types of chi square test:
for goodness of fit;
of independence
What level of data are chi square
The tests are often useful for
nominal or ordinal variables.
Both tests designed to test Hypotehses about:
frequencies (counts).
Goodness of Fit - what is it answering?
whether the deviation of observed results from expected values is large enough for us to reject the null hypothesis?
Is the difference between observed and expected values statistically significant?
The Chi-Square (Ο2) Statistic (goodness-of-fit test)
π
2 = β(πβπΈ)2 / πΈ
O - observed frequencies;
E - expected frequencies;
Ξ£ - sum (to be taken over all cells in the table
Degrees of freedom for the goodness-of-fit test
df = βNumber of cellsβ β 1.
The distribution of the Ο
2 statistic depends onβ¦.
the associated degrees of
freedom (df, denoted by k in the graph above)
Interpreting results of chi square goodness of fit
called a goodness-of-fit test,
because we are testing whether the distribution of a variable (here: accuracy of guesses) fits an expected distribution (under
the null hypothesis).
In this case, we saw that the data do not fit the expectation. Practitioners of Therapeutic Touch performed worse than their
theory would have led to suppose (and also worse than chance)
How do you report the result of chi square statstic?
Ο2 (1) = 4.128
Chi-square test of independence
Analysis of Contingency Tables
Contingency tables are frequently used in the analysis of
categorical variables
what does Chi-square test of independence test?
whether the null hypothesis
that the row variable and the column variable are independent
(i.e. that there is no association between the two variables)
Give an example of a hypothesis for a Chi square test of independence
Does the table
constitute evidence for an effect of βtype of treatmentβ on the likelihood of suicide attempt, (or could we have obtained the numbers in the table by chance, even if there was no effect in the
population)?
Marginals
allow us to work out expected outcomes
and if we have one of the observed values we can caluate all the observed values
Degrees of freedom for Ο2 test of independence
ππ = (π β 1) Γ (π β 1)
What to do with small expected frequencies? for test of independence
SPSS vs Howell what are their minimum values?
Small expected frequencies
The chi-square test can be inaccurate if one or more of the expected values are very small.
One convention is to set a minimum value to 5. This convention is overly cautious.
β’ For a 2Γ2 table, a sample size of n=10 will ensure reasonable accuracy (see Howell 2013, p. 152). This corresponds to an average expected value of 2.5!
β’ For larger tables, all cells should have expected values greater
than 1.
Measures of effect size
is chi sqare a measure of effect size?
why?
The chi-square statistic is not a measure of effect size.
Chi-square is sensitive to sample size. That means that you cannot
compare chi-square values computed on different tables, if the sample
sizes of the two tables are different.
β’ For example, if two samples have the same effect size, but one
sample is twice as large as the other, then the value for chi-square
for the larger sample will be twice as high as that of the smaller
sample.
Also: If the sample is large, the chi-square statistic may be significant
even if the actual effect is very small. This is an issue of power:
β’ If the sample is large, the chi-square test is very powerful, which
means that it may detect small effects that exist in the population β
including effects that are too small to be meaningful.
β’ If the sample is small, the test has little power, which means that
even large effects may not lead to statistically significant results.
There are several ways to measure the effect size in contingency
tables (4)
Absolute risk reduction:
Using MBT, by how many percentage
points do we reduce the risk of a suicide attempt compared to TaU?
β’ Numbers Needed To Treat:
How many patients do we need to treat
with MBT to avoid one suicide attempt (compared to TaU)?
Relative risk:
How much higher is the risk of a suicide attempt under
TaU compared to Mentalization-based Treatment MBT?
Odds Ratio:
How much higher are the odds of a suicide attempt
under TaU compared to MBT?
Interpreting odds ration
As before, letβs define: ππ
=
ππππ πππ/ ππππ ππ΅π
Then the observed odds ratio can be interpreted as follows:
OR = 1 β No effect (no difference between groups; odds
are the same).
OR > 1 β βPositiveβ effect. Higher odds for TaU than MBT.
OR < 1 β βNegativeβ effect. Lower odds for TaU than MBT.
Examples
OR = 2 β The odds of having a suicide attempt are twice as high under TaU as under MBT.
OR = 1.5 β The odds of having a suicide attempt are 50% higher under TaU compared to MBT.
OR = 0.5 β The odds of having a suicide attempt are halved under TaU compared to MBT.
CIs for Effect Sizes: Odds Ratios
What is effect size?
why calculate CIs? (SE)
Effect sizes computed from a sample are estimates of the βtrueβ effect size. Like all statistics, effect size estimates are subject to sampling error. It is therefore good practice to
report confidence intervals for effect size estimates
This means that we are 95% confident that the interval
between .94 and 12.97 contains the true odds ratio. This
confidence interval is rather wide, reflecting the uncertainty of
our estimate.
CIs is 1 or 0 an important figure?
1
We are 95% confident that the interval between
0.94 and 12.97 contains the true odds ratio.
- Note that the confidence interval contains the value 1.
- In the case of OR, 1 signifies βno effectβ.
β’ This agrees with the non-significant result of the chi-square test on these data (which led us to conclude that we had no evidence to reject the null hypothesis of βno effectβ).
