Stats Test Flashcards
Sharon's Flashcards
What is an ANOVA
Analysis of Variance
When to use and ANOVA
When we are testing experiements that have 3 or more levels of independent variables (e.g., comparing a control vs caffeine in the morning vs caffeine at night)
Why don’t we use multiple t-tests
Type 1 error will increase
What does ANOVA produce
F-ratio
What is an F-ratio
Compares systematic variance to unsystematic variance
What can / can’t an ANOVA tell us?
It can tell use there was an effect but it cannot tell us what the effect was
How do we find out what the effect was when doing ANOVA
Planned comparisons or post-hoc tests
What is the bonferroni correction
A wat to control type 1 error by dividing the alpha (0.05) by the number of tests
This then sets the new p-value for a test to be significant
What is planned comparisons
A set of comparisons between group means that are constructure before any data is collected
this is theory led
and there is more power to these than post hoc tests
What assumptions need to be met when doing ANOVA
- Normal distribution
- Homogentiy of variances
- Sphericity
Tests of homogeneity of variances for independent ANOVA’s
Levene’s test
significant Levene’s = assumption of homogeneity of variance has been violated
Test of Sphericity for dependent ANOVAs
Mauchly’s test
Significant Mauchly’s = assumption of sphericity has been violated
Define homogeneity of variance
Assumption that the variance of one variable is similar at all levels of another variable
Define Sphericity
The difference taken from the same participatn / entity are similar
What is a one-way ANOVA
One independent variable will be manipulated
What is one-way independent ANOVA
Experiments with 3+ levels of the independent variable and different participants in each group
How to run a one-way ANOVA on SPSS
- Check Levene’s test - if significant then assumption of homogeneity of variances has been violated
- Between-group effects = SSm (variation due to the model aka experimental effect). To find the total experiment effect look at between-group sum of squares
- Within-group effects = SSr (unsystematic variation)
- To be able to compare between groups and within groups we look at the mean squares.
- Look at the F-ratio, if significant do post-hoc tests
What post-hoc tests do you run after a significant ANOVA (you want to control for type 1 error)
Bonferroni correction
What post-hoc tests to run after a significant ANOVA (you have very different sample sizes)?
Hochberg’s GT2
What post-hoc tests to run after a significant ANOVA (you have slightly different sample sizes)
Gabriel’s procedure
What post-hoc tests to run after a significant ANOVA (you have doubts about variance)
Games-Howell procedure (this one is a safe bet)
What is effect size
the magnitude of an effect
r
How to calculate effect size
R squared = SSm / SSt
Square root this to get effect size (r)
What is SSt
Total sum of squares
total amount of variation within our data
What is SSm
Model sum of squares
variation explained by our model
What is SSr
Residual sum of squares
variation not accounted for in our model
What is a two-way ANOVA
Two independent variables will be manipulated
How to run a two-way ANOVA on SPSS
- Check Levene’s tests - if significant then assumption of homogeneity of variances has been violated. If violated then transform your data or use a non-parametric test or report inaccurate F value
- Summary table will include an effect for each independent variable (aka main effects) and the combined effect of the independent variables (aka interaction effects)
- Bold items are the SSm, Error = SSr
- Look at the F-ration, if significant then complete post hoc tests
What is a repeated measures ANOVA
Three or more experimental groups with the same participants
How to run a repeated measures ANOVA on SPSS
- Check sphericity (equal variances between treatment
levels). If Mauchly’s test is significant then the assumption of
sphericity has been violated.
- If sphericity has been violated, we can look at either the
Greenhouse-Geisser Estimate, the Huynh-Feldt estimate or
the lowest possible estimate of sphericity (aka lower bound).
- Use Greenhouse when Mauchly’s is LESS than 0.75, use
Huynh when Mauchly’s is MORE than 0.75 - If the effect is significant, we need to look at ‘pairwise
comparisons’ to see where the effect lies.
- Look for significant values ie. less than 0.05 - Calculate effect size - use benchmarks of .10 / .30 / .50
When to use greenhouse-geyser and when to use Mauchlys
- Use Greenhouse when Mauchly’s is LESS than 0.75, use
Huynh when Mauchly’s is MORE than 0.75
What happens when we violate sphericity
violating sphericity = less power = increases type 2 error
What is a mixed ANOVA
Independent variables are measured using both independent and repeated measures groups
How to run mixed ANOVA on SPSS
- As mixed ANOVA uses both independent and repeated design
we need to check if assumption of homogeneity of variances AND
sphericity have been violated. - Look at both output tables and find the main effects (one for
each INDEPENDENT VARIABLE) and one interaction term. (words
in CAPITALS are your INDEPENDENT VARIABLEs you need to look
at these) - Look at the F-ratios in both tables.
- If the effect is significant then we can run t-tests to see where
the effect lies, make sure to use Bonferroni method (independent
variable alpha 0.05 by the number of tests you will run)
- Look at both ‘paired samples test’ tables. → this is known as a
SIMPLE EFFECTS ANALYSIS. - Calculate effect size - use benchmarks of .10 / .30 / .50
What is ANCOVA
sometimes we conduct research we know some factors have influence on our DVs (from previous research e.g., age and memory)
These factors are called covariates and we can include them in our ANOVA
Why do we use ANCOVA
to reduce the error variance (increase how much variance we can explain)
eliminate confounds (by including the covariates we remove the bias of these variables)
How to run ANCOVA on SPSS
- Check Levene’s test of homogeneity of variances.
- If significant, transform the data of complete a non-
parametric test. - The output will look the same, it will just include the
covariates. - Look at the F-ratio for all the main effects and for
the covariates.
- If the covariate is significant, this means that it has
a relationship with our main independent variable. - Calculate effect size - use benchmarks of .10 / .30
/ .50
What is MANOVA
Multivariate analysis of variance
ANOVA but when there are several dependent variables
How to run MANOVA on SPSS
- Check for independence, random sampling, multindependent
variableariate normality and homogeneity of covariances matrices.
- If Box’s test is significant then the assumption of homogeneity of
covariances matrices has been violated. - Look at the multindependent variableariate test ‘group’ table. This is
showing the effect of the Independent variable on the DV. - When looking at the output Pillai-Bartlett test (Pillai’s trace) statistic
is the most robust. - If there is a significant F ratio then we need to look at the
unindependent variableariate tests or run a discriminant analysis.
How to interpret unindependent variableariate test statistics? -
Levene’s should be non-significant
- then look at ‘tests of between-subjects effects’ → corrected model
and group row stats should be significant if there is an effect between
IVs and DVs.
How to interpret discriminant analysis
Look at the ‘covariance matrices’ to see the direction and
strength of the relationships
Eigenvaues percentage of variance = variance accounted
for, square the canonical correlation to use as an effect size.
Wilks’ Lambda table shows significance for all variables,
look for the significant ones.
Use the Standardised Canonical Discriminant Function
Coefficients table to see how the DVs have contributed.
Scores can range between -1 - 1, high scores = variable is
important for the variate. Look down the ‘function 1’ column,
if one value is positindependent variable and the other is
negatindependent variable then the variate (aka function)
has discriminated the two groups
What is power analysis
The ability for a test to find an effect is known as statistical power
What is power of a test
Power of a test = the probability that a test will find an effect if there is one
We aim to achieve a power of 0.8
Power of a statistical test depends on
- how big the effect is
- how strict we are with our alpha level (i.e., 0.05 or 0.01)
- How big the sample size is - the bigger the sample size, the stronger the power
What are confidence intervals
A range of values that are believed to contain the true population value
eg. a 95% confidence interval means that if
we were to take 100 different samples and
compute a 95% confidence interval for each
sample, then approximately 95 of the 100
confidence intervals will contain the true
mean value
How to interpret confidence intervals
- If 95% CI do not overlap = means come from
different populations. - CIs that have a gap between the upper and
lower end of another - p <0.01 - CIs that touch end to end - p = 0.01
- CIs that overlap moderately - p = 0.05
What are common effect sizes
Cohen’s D
Pearson’s correlation coefficient r
odds ratio
What is Cohen’s d
The difference between two means divided by the SD of the mean of the control group, or a pooled estimate based on the SD of both groups
What are the benchmarks for Cohen’s D
small d = 0.2
medium d = 0.5
large d = 0.8
What are the benchmarks for Pearson’s correlation coefficient
small r = 0.1
medium r = 0.3
large r = 0.5
0 = no effect, 1 = perfect effect
What does an odds ratio of 1 mean
the odds of an outcome are equal in both groups
How to calculate the odds ratio
calculate by dividing the probability of the event happening by the probability of it not occurring
What is categorical data
Data which can be divided into groups (e.g., gender, age group)
How to analyse categorical data
Pearson’s chi squared test
The likelihood ratio
Yates continuity correction
Log linear analysis
when to use Pearson’s chi squared test
when we want to see if there is a relaitonship between two categorical variables
if the expected frequency is less than 5 then we need to use Fisher’s exact test
When to use th likelihood ratio
to be used instead of chi squared test when samples are small
When to use Yate’s continuity correction
When we have a 2x2 contingency table then type 1 error increases
Yate’s continuity correction fixes this by lowering the chi squared statistic
What is a 2x2 contingency table
2 variables with two level e.g., males vs female / phone vs no phone
When to use log linear analysis
When there are 3+ categorical vairbales
What are the assumptions when analysing categorical data
independence of residuals (as such you cannot use chi squared on repeated measures)
expected values: should not be less than 5
When to use chi-squared test
use a chi-squared test if you have nominal (categorical) data
the chi squared test can be used to see if these observed frequencies differ from those that would be expected by chance
Types of chi squared test?
Chi squared goodness of fit test (one IV)
Chi squared as a test of association (Two IVs)
When to use chi-squared good ness of fit
Used to compare an observed frequency
distribution to an expected frequency
distribution.
- Eg. when picking fruit are people more
likely to pick an apple vs a banana.
- If significant then, some fruit get picked
more than we would expect by chance.
When to use Chi squared as
a ‘test of association’ (two
independent variables)?
Used to see if there is an association
between two independent variables.
- Eg. is there an association between
gender and choice of fruit.
- If significant then, there is an association
between the two variables.
What is additivity and linearity
the outcome variable is linearly related to predictors
What are the parametric test assumptions
At least interval data
Additivity and linearity
Normally distributed
Homoscedasticity/homogeneity of variance
Independence
What is homoscedasticity / homogeneity of variance
Variance of the outcome variable
should be stable at all levels of the
predictor variable.
What is independence
errors in the model should be dependent
How to spot issues with assumption of normality
- look at the histogram (it should look like a
bell curve)
-look at the p-p plot (dots should fall
on/near the line) - Look at descriptive statistics (skewness
and kurtosis should be near to 0)
How to spot issues with
assumption of
linearity/homoscedasticity/
homogeneity of variances?
Look at scatter plots
Look at Levene’s test - significant =
variances unequal = assumption of
homogeneity of variances has been
broken.
What does a scatterplot look like when data is normal
dots scattered evenly everywhere
What does a scatter
plot look like when data
= heteroscedasticity?
funnel shape
what does a scatter plot look like when data is non-linear
curve
What does a scatter plot look like when data is non-linear and heteroscedasticity
curve and funnel (e.g., a boomerang)
Non-parametric alternatives to ANOVAs
kruskal-wallis
Friedman’s ANOVA
Non parametric alternative to one-way independent ANOVA
Kruskal-Wallis
Non parametric alternative to repeated measures ANOVA
Friedman’s ANOVA
How to interpret the
Kruskal-Wallis test?
- Look at the ‘ranks’ table, the mean ranks tell us which
condition had the highest ranks - If the chi squared test is significant then there is a difference
between groups (but we do not know what kind of difference) - To see where the difference lies, look at the box-plot and
compare the experimental group to the control group. - OR we can do a Mann-whitney test and use Bonferroni
correction (divide alpha by the number of tests), look to see
which conditions are significant. - Calculate the effect size by dividing the z score by the
number of obvs square rooted.
- use benchmarks of .10 / .30 / .50
What is an alternative to
the one way repeated
measures ANOVA?
Friedman’s ANOVA
How to interpret the
Friedman’s ANOVA?
- Look at the ‘ranks’ table, the mean ranks tell us which condition
had the highest ranks. - If the chi squared test is significant then there is a difference
between groups (but we do not know what kind of difference) - To see where the difference lies, look at the box-plot and
compare the experimental group to the control group. - OR we can do a Wilcoxen test and use Bonferroni correction
(divide alpha by the number of tests), look to see which conditions
are significant in the ‘test statistics’ box. - Calculate the effect size by dividing the z score by the number of
obvs square rooted.
- use benchmarks of .10 / .30 / .50
What are
correlations?
relationships between variables