week 11 -- Comparing counts & Chi square Flashcards
What do we do when we carry out a statistical hypothesis test (e.g. one-sample t-test)?
Visualize our data (histogram). ● Check some assumptions: – Independent observations – Approximately symmetrical and unimodal distribution ● Notice something interesting. ● Quantify it with a statistic (mean). ● Compare this to its sampling distribution given the null hypothesis. ● Quantify uncertainty: – Confidence interval – p value
Calculate: Chi square
Calculate Obs – Exp
Square
Divide by Exp
Chi-square: Summary
for CATEGORICAL data (beard, no beard)
NOT QUANTITATIVE data (length of beard)
● Tests the fit of count data to hypothesised
proportions.
● High χ (and low p) => data do not fit.
● Residuals can tell us in what way the data do
not fit.
● Alternative for small samples: Fisher’s exact
test
Goodness-of-fit
Goodness-of-fit involves testing a hypothesis. We have specified a model for the distribution and want to know whether it fits. There is no single parameter to estimate, so a confidence interval wouldn’t make much sense. A one-proportion z-test won’t work because we have 12 hypothesized proportions, one for each sign. We need a test that considers all of them together and gives an overall idea of whether the observed distribution
differs from the hypothesized one.
Chi square – how’s and why’s
Are the discrepancies between what we observed and what we expected just natural sampling variability, or are they so large that they indicate something important?
It’s natural to look at the differences between these observed and expected counts, but just adding up these differences won’t work because some are positive;
others negative. SO we square them to get all positive positive values.
Because the differences between observed and
expected counts generally get larger the more data we have, we also need to get an idea of the relative sizes of the differences. To do that, we divide each squared difference by the expected count for that cell.
chi-square (or chi-squared) statistic,
is found by adding up the sum of the squares of the deviations between the observed and expected counts
divided by the expected counts
df for chi-square
number of degrees of freedom for a goodness-of-fit test is n - 1.
Here, however, n is not the sample size, but instead is the number of categories.
If the observed counts perfectly matched the expected, the x2 value would be 0.
The greater the differences, positive or negative…
the larger x2 becomes. If the calculated value
is large enough, we’ll reject the null hypothesis. What’s “large enough” depends on the
degrees of freedom. We use technology to find a P-value.
residuals.
Observed - Expected
The Trouble with Goodness-of-Fit Tests:
What’s the Alternative?
Prerequisite: Theory of what the proportions should be in each category
Unfortunately, the only null hypothesis available for a goodness-of-fit test is that the theory is true.
But the hypothesis-testing procedure allows us only to reject the null or fail to reject it.
We can never CONFIRM that a theory is true, which is often what people want to do.
The goodness-of-fit test compared
counts with a theoretical model.
The homogeneity test hypothesizes that the distribution
does not change from group to group
d
Chi square, indepenence
When we ask whether two variables measured on the same population are independent we’re performing a
chi-square test of independence.
they categorize subjects from a single group on two categorical variables rather
than on only one.
Contingency tables categorize counts on two (or
more) variables so that we can see whether the distribution of counts on one variable is
contingent on the other.
WHAT CAN GO WRONG?
■ Don’t use chi-square methods unless you have counts.
Data reported as proportions or percentages can be suitable for chi-square procedures, but only after they are converted to counts.
■ Beware large samples. chi-square tests are unusual. No hypothesized distribution fits perfectly, no two
groups are exactly homogeneous, and two variables are rarely perfectly independent.
But we have no measure of how far the data are from the null model. There are no confidence intervals to help us judge the effect size
■ Don’t say that one variable “depends” on the other just because they’re not independent. Dependence implies causation, but variables can fail to be independent in many different ways. When variables fail the test for
independence, you might just say they are “associated.”
A test comparing the distribution of counts for two or more groups on the same categorical variable (homogeneity)
A test of whether the distribution of counts in one categorical variable matches the goodness-of-fit distribution predicted by a model is called a test of goodness-of-fit.
A test of whether two categorical variables are independent examines the distribution of independence of counts for one group of individuals classified according to both variables.
d
df for chi square
(row-1) x (column - 1) = df
