Summa Week 9 Flashcards
When we want to explore whether the effects of different treatments influence the dependent measure, we can use tests of
t-test - two means, and one predictor and one independent variable
ANOVA - an extension of t-test
- compares several means
- can manipulate lots of IVs
If we want to compare several means why don’t we compare pairs of means with t-tests?
can’t look at several independent variables and inflates the Type I error rate
What is PC?
error rate per comparison
PC is the prob of making a ______ error on a ____ comparison, assuming the null hypothesis is ____
Type I
single
true
If alpha = 0.05, there is a 5% chance that you are rejecting the null hypothesis _______
incorrectly
If we ran a bunch of t-tests of a = .05 then the per comparison error rate would be
.05
FW?
error rate familywise
FW is the prob of _____ rejecting at least one null hypothesis in a family of c comparisons, assuming that each of the c null hypothesis is ____ in a set (or family) of comparisons
incorrectly
true
Familywise alpha is
1-(1 - a’)^c
FW where a’ is the
per comparison error rate
FW where c is the
number of comparisons
When we have k = 6 (k is the number of experimental conditions), we will have c =
c = 6*(6-1)/2 = 15 comparisons
If we have a error rate per comparison of a’ = .05, then familywise alpha is
FW = 1 - (1-.05)^15 = .537
The aim of ANOVA is to determine if treatment effect is present by comparing ______ and _______ _____
errors
treatment effect
What is error also known as?
random variance
What are random errors?
individual differences
What are measurement errors?
problems of accurately collecting data
What is systematic variance?
treatment effect, the action of the IV
When the population means are equal, the differences among the group means will reflect the operation of _________ _____ alone (no ______ _______)
experimental error
treatment effect
What is the theory of ANOVA?
SS total = SS treatment + SS error
SS total =
total sum of squares = variability between scores
SS treatment =
model sum of squares - variability due to the experimental manipulation
SSerror =
residual sum of squares - variability due to individual differences in performance
What is the F-ratio?
MStreatment/MSerror
If the model explains a lot more variability than it can’t explain then the experimental manipulation has had a _______ _______ on the outcome (DV)
significant effect on the outcome
What is within-groups variability?
within-groups is intragroup
what is between-groups variability?
between-groups is intergroup
How many degrees of freedom impact the shape of the F distribution?
2
Is ANOVA two-tailed?
No, only positive
Like a t-test, ANOVA tests the null hypothesis that the ______ are the same
means
What is the experimental hypothesis?
means differ
What type of test is ANOVA?
an Omnibus test
Omnibus tests are described as
test for overall difference between groups
group means are different
significant difference is not determined
Assumptions for 1-way ANOVA:
RANDOM SAMPLING
each sample is a random sample from its population
Assumptions for 1-way ANOVA: random sampling robustness?
considered inappropriate to conduct if violated, but some argue it is robust if violated
Assumptions for 1-way ANOVA: independence of cases
each case is NOT influences by other cases, and NOT robust to violations
Assumptions for 1-way ANOVA: normality
the DV is normally distributed in each pop, provided the sample size (N) is large and the n of the groups is EQUAL
What is assumed for 1-way ANOVA to be robust?
sample size is large, and group sizes are equal
Assumptions for 1-way ANOVA: HoV
the degree of variability in the pop are equivalent
Assumptions for 1-way ANOVA: HoV robustness
robust to violations if the sample size is large and groups are about equal
What are the 2 types of means of interest in ANOVA?
marginal means (overall mean of each group) and the Grand Mean, or M..
What is the sum of squares for SStotal
the difference of the individual scores and the marginal means (within/sserror) and the marginal means and the grand mean (between/sstreatment)
What is df for sstreatment?
df = k-1
What is df for error?
df = N - k
What is df for sstotal?
df = N -1
What is the formula for ANOVA?
F(dfbetween, dfwithin) = MStreatment/MSerror
What is MStreatment?
SStreatment / dftreatment = MStreatment
What is MSerror?
SSerror/dferror = MSerror
What is SS total?
SSbetween (sum of squares of the model) + SSwithin (sum of squares of individual differences)
If F value is _____ than F Critical value, we can reject our ___ hypothesis and conclude that not all group/sample ________ are equal
greater
null
means
What is a condition of ANOVA?
we don’t know where exactly the differences lie in calculations
How do we determine where differences in means occur?
using post-hoc tests
What is on ZW’s favourite slide?
SStotal (total variance in the data) = SStreatment (variance due to the treatment) + SSerror (errors in model)
what is the calculation for SStotal?
sum of (marginal mean for group 1 - grand mean)^2, or.... sgrand^2 x (N-1)
e.g. M… = 3.467
marginal means are 2.20, 3.20, and 5.00
sgrand^2 = 3.124
n = 5, k = 3, therefore N = n x k = 15
so SStotal = 3.124(15-1) = 43.74
What is the calculation for SS total?
sgrand^2 x (dftotal)
What is the calculation for SStreatment?
sum of (total partic # group 1 x (marginal mean group 1 - grand mean)^2 + (total particip group 2 x (marginal mean group 2 - grand mean)^2 + (total particip group 3 x (marginal mean group 3 = grand mean)^2
= 5(2.2 - 3.467)^2 + 5(-0.267)^2 + 5(1.533)^2
= 8.025 + 0.355 + 11.755
= 20.135
What is the calculation for SSerror?
= sum of (stat group 1 - marginal mean group 1)^2
= sum of (standard variance group 1 x (n - 1)
e.g. M… = 3.467
marginal means are 2.20, 3.20, and 5.00
sgrand^2 = 3.124
n = 5, k = 3, therefore N = n x k = 15
so…
=sgroup1^2 (n1-1) +sgroup2^2…
= 1.70(5-1) + 1.70(5-1) + 2.50(5-1)
23.60
1-way ANOVA dftotal =
N - 1
dftreatment 1-way anova is
k - 1
dferror 1-way anova is
N-k
SStotal =
sstreatment + ss error
dftotal =
dftreatment + dferror
MStreatment =
SStreatment/dftreatment
MSerror =
SSerror/dferror
F(dfbetween,dfwithin) =
MStreatment/MSerror
If F value is ______ than F critical value, we can _____ our null hypothesis and clucde that…
great
reject
not all group/sample means are equal
What is the downside to 1-way anova?
we don’t know where the differences lie, therefore we use post-hoc tests
Significant t and F ratios show that there is ____ ____ of the treatment, a real _____ between the groups that _____ be explained by chance
a real effect
difference
cannot
effect size measures how
big the effect of the treatment is
a significant effect depends on
size of the mean differences
size of the error variance
degrees of freedom
How to determine raw effect size (looking)
just looking at the raw difference between groups
how to determine raw effect size (depending)
can be illustrated as the largest group difference or smallest
how to determine raw effect size (comparisons)
CANNOT be compared across samples of experiments
Standardized effect size …
expresses raw mean differences in standard deviation units
Another name for standardized effect size is
Cohen’s d
What is a small, medium and large effect for Cohen’s d, respectively?
.2, .5, .8
What is eta-squared?
an OVERESTIMATION of the degree of overlap in the population
n^2 = SStotal -SSerror/total, or…
= SStreatment/SStotal
What is omega–squared?
a better estimate of the percent of overlap in the population than eta squared, it corrects for the size of error and the number of groups
What is the formula for omega squared?
oo^2 = (SStreatment - (k-1)MSerror) / (SStotal + MSerror)
n^2 is a…
sample estimate of the proportion of the variance in the DV that is accounted for by the IV
What do you use for POPULATIONestimates of effect size?
oo^2, or omega squared
What is partial eta-squared?
the proportion of the total variability attributable to a given factor
npartial^2 formula
SStreatment / SStreatment + SSerror
partial omega squared?
oo^2 = SStreatment - (k-1)MSerror / SStreatment + (N - (k-1))MSerror
What is effect size for correlation?
r, or Pearson’s correlation
What is the small, medium and large effect size for correlation?
r, which is .1, .3 and .5
What is effect size for ANOVA (first of 2)?
eta squared
What is the small, medium, and large effect size for ANOVA (first of 2)?
n^2, 0.01, 0.06 and 0.14
What is effect size for ANOVA (second of 2)?
omega squared
What is the small, medium and large effect size for ANOVA (second of 2)?
omega squared, 0.01, 0.06, and 0.14
What is effect size for t-tests?
Cohen’s d
What is the small, medium, and large effect sizes for t-tests?
Cohen’s d 0.2, 0.5, 0.8
What is effect size for 2 x 2 tables?
odds ratios
What is the small, medium, and large effect sizes for 2 x 2 tables?
odds ratios, 1.5, 3.5, 9.0
What are Welch statistics?
when the Levene’s F test reveals HOV assumption is NOT met (i.e. p <= .05), then the Welch’s F test should be used
How do you get Welch’s F in SPSS?
Analysis - Compare Means - 1-way anova - option, and use the F under statistic a, as well as the new df and significance score. ALWAYS CREATE WHOLE NUMBERS FOR DFS
There are many effect siz measures that indicate the amount of total variance that is accounted for by the effect. What does no relationship look like?
DV and A circles are completely disjointed
There are many effect siz measures that indicate the amount of total variance that is accounted for by the effect. What does a small reltaionship look like?
DV and A circles are just barely touching
There are many effect siz measures that indicate the amount of total variance that is accounted for by the effect. What does a moderate relationship look like?
DV and A circles are touching about 1/4 of their surface area each
There are many effect siz measures that indicate the amount of total variance that is accounted for by the effect. What does a strong relationship look like?
DV and A circles are touching more than 1/2 of their surface areas
wHAT IS THE proportion of variance accounted for by the regression model?
R^2
Multiple R^2 is equal to
eta-squared
Adjusted R^2 is equal to
omega-squared
What is the formula for R^2?
SStreatment/SStotal (omega-squared)
Why would you select levels of the IV that are very different?
to increase the effect size and make the study more powerful
What can be more liberal to create a more pwoerful study?
alpha level
What can you reduce for designing a powerful study?
reducing error variability
What would you compute for the necessary amount for adequate power when designing powerful studies?
the sample size
How do you access effect size for 1-way ANOVA studies?
Analyze - Univariate, define DV and define IV and check for fixed or random models then click options
What do you need to ensure before preceding with the F value data?
that Levene’s test of equality of error variance is insignificant, meaning that the variance between groups are homogeneous and that we can assume that the test for the DV is equal across groups
Why is ANOVA an omnibus test?
it tests for overall difference between groups, tells us that group means are different, yet sadly does not say where exactly the significant difference lies
What’s the deal with post-hoc tests?
they are done after ANOVA doing pairwise comparisons to control FW
When are post-hoc tests appropriate?
only when you are doing exploratory research (a.k.a fishing for significance)
How many post-hoc tests are there for our interest?
5
What is the Bonferroni method?
a type of posthoc test that minimizes the familywise alpha
What are the 5 posthoc tests to discern where the difference in the means lie in an ANOVA?
1) Bonferroni
2) Tukey’s HSD
3) Dunnett’s C
4) Scheffe’s test
5) Fisher’s LSD procedure
What are priori comparisons?
planned comparisons before the data are collected usually with an idea of what to expect
What do planned comparisons almost never involve?
very many of the possible comparisons. It is a really bad idea to do pairwise t-tests among all pairs of means
If the comparisons are planned, then you test them without
any correction.
Each F-test for the coparison is treated like any other F-test. You look up an F-critical value in a table with dfCOMP AND DF ERROR
How do you do a priori comparisons with correction?
a t-test by using MSerror and tcritical at dferror
When do you use Bonferroni t-test (Dunn’s test)?
with correction and a t-test by using MSerror and tcritical at dferror
Which a priori comparisons require equal sample sizes?
Bonferroni and Dunn’s tests
What is trend analysis?
if you have ordered groups, you often will want to know whether there is a consistent trend across the ordered group (e.g., linear trend)
How do you tell whether there is a conssitent trend across the ordered group?
a linear trend
When does trend analysis come in handy?
there are orthogonal weights already worked out depending on the number of groups
What does trend analysis depend on for the number of groups?
orthogonal weights are already worked out
When is trend analysis best done/
as a PLANNED comparisons, although can be done posthoc
Reporting ANOVA in APA format:
In this study, a random sample of states from each of the census reigions were taken and the average salary in each clinical counseling, and school psychologists was record mean salary in the Northeast region was $77,730 (SD = $1,030), $63,550 (SD = $930) in the Midwest, $61,370 (SD = $1039) in the South and $68,830 (SD = $870) in the West. The overall effect of analysis of variance showed a statistically significant effect in the region, F(3, 120) = 3.52, p = .049, oo^2 = 0.32.
Howeverpost-hoc testing showed that the onlly statistically significant difference that existed was between the Northeast and the South – the salaries for clinical, counseling, and school psychologists are statistically higher in the Northeast than in the South (Tukey’s HSD, p < .05). The results of this study suggest that it might be worthwhile for a psychologist living in the South to move to the North, but there would be noadvantage to moving to the West or Midwest. If this study is replicated, it would be advisable to take into account the cost of living as higher salaries in a region may be mitigated by a higher cost of living.
