Week 6 Flashcards
two sample versus one sample hypothesis testing
● To conduct one-sample hypothesis testing, we need information on the population
● Often we are interested in comparing 2 distinct groups, but only have sample information
● To do this, the equations and symbols differ, but the same logic
○ Suppose we want to know if two distinct groups are on average different from each other with regard to one aspect.
■ Men and women
● We could want to test if they differ one: the average income they get for a similar job
● If there is a difference in the means of the two samples, we want to test the following question:
○ Is the difference between two samples large enough to allow us to conclude taht the population that these samples represent are different?
two sample hypothesis testing (5 steps)
Step #1:
● Independent random samples?
○ yes: samples randomly selected, and no relationship between the selection of the respondents from Alberta and those from Ontario
● Level of measurement for the dependent variable is interval/ratio?
○ no (ordinal), but we choose to relax this assumption
■ Also the numbering of the categories can also indicate the average in a way
● Normal distribution?
○ the central limit theorem applies: N1+N2 > 120 - sum of the two sample sizes
● Equal variances in the two populations?
○ Different versions of the test - two sample t-test has two different versions
Step #2:
● H0: μ1 = μ2: NULL
○ No difference between the mean rank of that issue in Alberta and in Ontario
● H1: μ1 ≠ μ2: RESEARCH
○ There is a difference: on average, people from Alberta do not give the same importance to the economy and job creation as people from Ontario do
Step #3:
● Select the distribution:
○ t-distribution or z-distribution? Is N1+N2 ≥ 120?
○ yes, therefore use z-distribution
● Select level for critical region:
○ α = 0.05, two-tailed (because we don’t know what direction it will be we’re just asking if there’s a difference)
○ zcritical = ±1.96
Step #4:
Excel will produce the following results for the test:
Zobtained = -2.535
p (sig.) = 0.0115
Step #5:
● If zobtained falls within the critical region, reject the null
○ zcritical = -1.96 (i.e., for α=0.05), and zobtained = -2.53 (i.e., p = 0.0115)
○ reject the null hypothesis: p < α
Interpretation?
there is a statistically significant difference in the average opinion of people from Ontario and people from Alberta on the importance given to the state of economy and job creation during the federal election campaign
what can we make of that?
Is it good research:
● If you are reading about these results (and even the methodology) in a newspaper article, how can you be critical of the authors?
○ Strengths:
■ large enough sample
■ appropriate test for this hypothesis
○ Weaknesses:
■ different interpretations to the survey question? - the way we arrive at the data that we consider to be valid - some things were difficult to distinguish on the survey - issues of mutually exclusivity - issues of validity. Are you really measuring economic and jobs the same way across all respondents?
■ appropriate choice of possible answers? - collectively exhaustive
Cross-tabulations
Can we detect patterns by looking at frequencies in cross-tabs (contingency tables)?
Independent variable (IV) values as columns
Dependent variable (DV) as rows
● E.g., is there a relationship between:
● Level of human development, and
● Political liberties (as defined by Freedom House)
Choose a measure for each variable: both ordinal
● Human Development Index (HDI), recoded as (low/medium/high)
● Political Rights Index from Freedom House (low/high)
Compile combined frequencies for cases where we have data
So 22 cases are countries with a low HDI and few political rights, and so on
Columns and row “totals” represent the total cases for a given value of each variable (called the marginals). E.g., 40 total ‘Low HDI countries’.
We can look at these combined frequencies, but since the groups for the IV are of different sizes, we better transform it into percentages
Two possibilities:
1) Percentages of row totals
2) As percentages of column total
Chi-square test
2nd option to analyze bivariate relationship between nominal/ordinal var.
● Widely used method for determining the level of agreement between the frequencies in a distribution of observed data and the frequencies calculated on the assumption of a random distribution.
● Establishes statistical significance of the pattern we are observing
● Compares the (combined) frequencies we observe with the (combined) frequencies that we would observe if there were no relationship between the two variables
● After you choose your p-value (0.05), you calculate your degrees of freedom at that p-value and get the critical chi square on appendix c, to determine whether our observed values differ from the expected values, we compare the calculated chi square with the chi square on the appendix, if the calculated score exceeds the critical value we can conclude that the observed values differ significantly from expected values. For example a degrees of freedom of 1 is a chi square of 3.841 and if I receive a score of 0.74 then this is below the critical value suggesting that my results are not statistically significant
● The chi square depends on the strength of the relationship and the sample size. Smaller sample sizes means that the relationship may show up as not statistically significant.
In other words: tells us the likelihood of the discrepancies between a purely random distribution and the one we observe, if the null hypothesis were true.
If the probability of obtaining this discrepancy from a random sample is smaller than a critical level (α), then the relationship is statistically significant
Applying the chi-square test using the 5-step hypothesis testing
- Check assumptions and test requirements
a. No need for assumptions here: “nonparametric” test
b. Nominal and/or ordinal variables - State the hypotheses:
a. H0: there is no correlation between the two variables in the population
i. The pattern we observe in the sample is due to random chance
b. H1: there is a correlation between the two variables
i. The pattern we observe in the sample is indicative of a similar one in the population - Select the sampling distribution and determine the critical region
a. Here we are using the Chi-Square distribution (Appendix C)
b. Degrees of freedom: df = (nb_of_rows – 1)*(nb_of_columns – 1) - IN other words, you’re looking at the number of values of your independent variables and the number of values of your dependent variable and you subtract one then multiply
c. More importantly, we need to choose an alpha level (α): 0.05? 0.01?
d. Depends on how confident we want to be of not making a type I error (rejecting the null hypothesis by mistake) - no way to detect that because you’ll never the information on the complete population
e. In the HDI/political rights example, we have df = 2, and if α = 0.05, then our critical value of the Chi-Square statistic is χ2 = 5.991
4)
Calculate the test statistic for the sample that we have
The statistic considers:
● For each of the observed combined frequencies, what is the (squared) difference between this observed value and the one we would expect if there were no relationship at all
● The statistic is the sum of all these comparisons (for all the combinations of two value for the IV and the DV)
● For the HDI/PR example, we would obtain χ2 = 16.04 and p = 0.0003
5)
Apply the test: is the statistic obtained for our sample larger than the critical value we determined in Step #3?
We have 16.04, which is greater than the critical value of 5.991
We can reject the null hypothesis (0.0003 < 0.05, or 16.04 > 5.991)
Conclusion: the observed discrepancies between our sample and the expected values are too large to be attributed to chance (i.e., sampling error)
The relationship is statistically significant: there is a correlation between the level of PR and the level of HDI in the population (i.e., all countries in the world)
However: it does not tell us the direction or the strength of the relationship
The steps when investigating a bivariate relationship:
- Does a relationship exist between the variables? (stat. significance test)
- If so, how strong is the relationship? (measure of association)
- What is the pattern and/or direction of the relationship?
a. Since nominal variables cannot be ranked, the direction of a relationship between two variables is only possible with ordinal variables - Could the relationship be a causal one?
The Chi-Square or t test only tells us the answer to #1.
Measures of association
Measures of association tell us information on: the strength (magnitude) of the relationship the direction (sometimes)
Measures of association can be differentiated according to:
The level of measurement of the variables they apply to (nominal, ordinal, interval)
Whether or not their value has a direct interpretation (proportional reduction of error, or PRE)- this tells us the percentage of the variation in the dependent variable that is explained by variation in the independent variable
Measures of association for nominal and ordinal bivariate analysis
Nominal: 0-0.10 weak, 0.11-0.30 moderate and 0.31-1 strong. Range only from 0-1
- Phi (2x2 table requirement) and no PRE
- Cramer`s V (any size table) and no PRE
- Lambda (equal row marginals) and has a PRE
Ordinal: 0-0.30 weak, 0.30-0.60 moderate, 0.61-1 strong. Range from -1 to 1.
- Gamma (no more than 5 categories for each variable) and has a PRE
- Spearman`s Rho (any row of categories) and no PRE
- Rho squared (any number of categories) has a PRE, ranges from 0-1 but doesn`t tell you whether something is strong or not
Proportion reduction error explained:
● The PRE for lambda: tells us that the change in the number of correct predictions between knowing the value for the independent variable and not knowing that value. The accuracy of smoking prediction improves by 22.5 percent with a PRE of 0.225 which tells us the value of the independent variable.
● Gamma and spearman’s rho are -1 to 1 because they test directionally hypothesis if the direction is - two variables are negatively related if it is positive they are positively related
● Discordant pairs run in opposite directions while concordant pairs run in the same direction
● When squaring rho you get the PRE - for example a 0.80 rho squared is 0.64 - meaning that our errors of prediction dependent variable will be reduced by 64% once we know the independent variable (the rank of a cyclist DV and the number of endorsements) - prediction of cyclist endorsement rank will be reduced by 64% once we know their rank on the number of races won
