Week 23 Flashcards
A robust statistical procedure is one for which the Type I error rate remains close to the desired level even when the assumptions of the procedure are not met.
a) true
b) false
a) true
Explanation … Not much explanation is needed here. This is the definition of robustness. Note that robustness does not say anything about the Type II error rate. This means that even a robust procedure may lose power when its assumptions are violated.
ANOVA assumes that the observations in different cells of a factorial design are independent of each other.
a) true
b) false
b) false
Explanation … ANOVA assumes that all the observations within a cell are independent. The classic way in which this assumption may be violated is when responses of one participant are influenced by hearing or seeing the responses of another participant. In contrast to within-cell independence, observations in different cells don’t need to be independent of each other. Stable differences between individuals in a repeated measures can cause observations in different cells to be dependent on each other and this is exactly what a within-subjects analysis depends on.
The assumption of homogeneity of variance refers to the variance of the means.
c) true
d) false
b) false
Explanation … ANOVA is not robust to violations of the assumption of independence of observations. This means that you must take measures at the level of experimental design to avoid anything that would violate this assumption in the first place. There is no alternative analysis that will help you out and you can’t rely on a balanced design either..
The assumption of homogeneity of variance refers to the variance of the means.
c) true
d) false
d) false
Explanation … The numerator of any F-ratio in an ANOVA is always the variance of some set of means. But this is not the variance that the assumption of homogeneity of variance for ANOVA refers to. Instead, this assumption refers to the within-cell variances that are pooled together and form the denominator of any F-ratio. These sample variances are assumed to be “homogeneous” which means that they could all have been drawn from populations that share a common variance.
For a between-subjects ANOVA, if you do not pass Levene’s test you can still go ahead and use the results because ANOVA is robust to violations of homogeneity of variance.
a) true
b) false
b) false
Explanation … Under certain circumstances ANOVA is robust to violations of homogeneity of variance, but as a general statement this is not true. If you fail Levene’s test, you can still assume that ANOVA results will be robust as long as the design is balanced. The combination of an unbalanced design and failure to satisfy homogeneity of variance, however, means that your Type I error rate may not even be close to what you think it is.
For a within-subjects ANOVA, if you do not pass Levene’s test then you cannot meet the assumption of sphericity.
a) true
b) false
a) true
Explanation … Sphericity is a 2-part condition. One part has to do with the correlations between observations in different cells of a design. The other part is the assumption of homogeneity of variance. The 2 conditions jointly determine the assumption of sphericity. If you violate either one, then sphericity is violated.
When do you use the Greenhouse-Geisser test?
a) When the assumption of sphericity is violated
b) When the assumption of normality of the errors is violated
c) When the assumption of the independence of the errors is violated
d) When the dependent variable is ordinal rather than interval in nature
Explanation … The Greenhouse-Geisser test is one of a range of tests that are available for use in a within-subjects analysis when the assumption of sphericity has been violated. All of these tests provide the same ANOVA outputs as a standard ANOVA but seek to preserve the Type I error rate.
Twenty mock jury trials are conducted, each with different jurors. The main eyewitness in each trial is an actor who has been trained to either maintain eye contact with jurors when answering questions or to avoid eye contact. Each mock trial results in a guilty or not-guilty verdict. What test would you use to determine if addressing the jury directly affects the outcome of a trial?
a) 1-way repeated-measures ANOVA
b) Correlation
c) Chi-square test
d) Simple logistic regression
e) Friedman test
c) Chi-square test
Explanation … As in every question here, the primary task is to decide what sort of variables are involved in the research design being described. Once you know this, answering the questions will be a simple matter of looking up the correct test in the table at the end of this document.
Let’s think about the independent variable first. From the description of the research design it appears that what is being manipulated is the witness’s eye contact with jurors. Eye contact is either maintained or not so this is categorical data (with 2 levels). Moreover, there are different juries at each level so this is an independent groups design. We should thus be looking in the table for entries corresponding to the information …”1 IV with 2 levels (independent groups)”.
Now let’s think about the dependent variable. In each case what is being observed is the jury verdict. There are only 2 possibilities, guilty and innocent, and so this is also categorical data. Consulting the table reveals that a Chi-square test is appropriate for this design. This is the same Chi-square test of independence you learned in 2nd year statistics. It tests to see if the proportion of guilty verdicts is the same at each level of jury engagement.
Physicians who are on call for long periods may make poor diagnoses due to fatigue. You arrange for 40 physicians to complete complex mental puzzle tasks at the end of their shifts. The length of the shift and the number of errors made in solving the puzzle are recorded. What procedure should be used to determine whether puzzle-solving ability changes with time spent on call?
a) 1-way repeated-measures ANOVA
b) Correlation
c) Chi-square test
d) Simple logistic regression
e) Friedman test
b) Correlation
Explanation … Once again, the issue is the nature of the variables. Each physician in the study will have 2 measurements made on them … the length of the shift they have just worked and the number of errors they make. Both of these variables are at least interval in nature (in fact they are both ratio in nature because they are both measured on scales having true zeros). We are therefore looking for tests where both the independent and dependent variables are interval. The table offers 2 possibilities … correlation or simple linear regression. Both are legitimate possibilities but only correlation is offered in the answer list. (Note: One of the possibilities in the answer list is “simple logistic regression”. This is not the same as the “simple linear regression”.
In a separate study following the one in the previous question, you arrange for 65 physicians who have already been on call for 12 hours to complete puzzle tasks after 1) spending 10 minutes sitting quietly, 2) exercising vigorously, or 3) taking a shower. Each physician undertakes all 3 activities (on different days). What procedure should be used to determine whether time needed to solve a complex puzzle depends on type of activity (assume that the distribution of puzzle completion times is normal)?
a) 1-way repeated-measures ANOVA
b) Correlation
c) Chi-square test
d) Simple logistic regression
e) Friedman test
a) 1-way repeated-measures ANOVA
Explanation … This experimental design should appear familiar to you. The idea is to find out whether an interval variable (time needed to solve a complex mental puzzle) which is normally distributed depends on 3 types of activities undertaken after 12 hours on call. Since there are 3 types of activity this is a categorical independent variable with 3 levels. Moreover since, on different days, physicians undertake all 3 types of activity, this is a “within-subjects” design which in the table is denoted “1 IV with 2 or more levels (dependent/matched groups)” . The table therefore lists a 1-way repeated-measures ANOVA are the appropriate test.
The same study as in question 3 is carried out but the researchers are concerned that the distribution of puzzle completion times is highly skewed and kurtotic at all levels of the treatment. What procedure should be used to determine whether time needed to solve a complex puzzle depends on type of activity?
a) 1-way repeated-measures ANOVA
b) Correlation
c) Chi-square test
d) Simple logistic regression
e) Friedman test
Explanation … Sometimes nonparametric tests are used not because of the type of data involved in a design but because the distribution of values does not satisfy the assumptions of ANOVA or t-tests. In this design a 1-way repeated-measures ANOVA would normally be used (see the explanation for the previous question) but we are told that the dependent variable values are not normally distributed (and in fact are “highly” non-normal). This means that we should try a nonparametric analogue of the 1-way repeated-measures ANOVA which according to the table is called the “Friedman test”. This test turns the dependent variable of times needed to solve puzzles into and ordinal variable by ignoring actual completion times and only recording who finished with the quickest, the second quickest, third quickest, etc., in each condition. Since you can’t even define a normal distribution for ordinal data, no assumption of normality needs to be in place for the test to work.
