Group difference Flashcards
What are the two types of group when measuring group difference?
- Independent (mutually exclusive)
2. Dependent (mutually paired)
What are examples of mutually paired (dependent) groups?
- Same person being measured twice
2. two people bound in some way (husband-wife)
Dependent groups can be different sizes, T/F
FALSE!
Think about it - same person getting measured twice etc
Independent groups can be difference sizes, T/F
TRUE!
But if they are different sizes it means the design is imbalanced.
In independent group design, can a participant belong to more than one group?
No!
What are the relevant assumptions when investigating mean differences between two INDEPENDENT groups (3)
- Observations are independent
- Observed scores are normally distributed
- Variances in the two groups are the same (homogeneity of variance assumption)
With INDEPENDENT groups, there is a circumstance in which it doesn’t matter so much if assumption #3 (homogeneity of variance) is violated… what is that circumstance?
When the design is BALANCED
How do you test homogeneity of variance assumption?
There are two ways, you know em
- Levene’s test
2. Fligner-Killeen’s test
You’re doing independent or dependent group difference…
your observed scores are not normally distributed…
do you use standardised or unstandardised CIs…?
Unstandardised!
Ironically
These are robust against mild-to-moderate non-normality
So you want to standardise your group differences…
there are two ways to do this, what are they?
Bonett’s squiggle
Hedge’s g
Of the two ways to standardise group difference - Hedge’s g and Bonett’s squiggle - and both of them require the observations to be normal.
But one of them also needs the variance to be homogenous. Which one needs the homogeneity… of the variance…? WHICH ONE?!
Hedge’s g!
When testing for homogeneity of variance using Fligner-Killeen and Levene’s, what are you actually looking for, actually…?
p values
And you want em to be big
You’re looking at the Fligner-Killeen test result and it says p = .25… what does it mean?!
It means your variance is homogenous and you can finally relax
You’re looking at the following results:
Levene’s: p = .04
Fligner Killeen: p = .18
What should you do?
Its inconclusive so you should be conservative and assume HETEROgeneity of variance
When doing DEPENDENT groups, what is the name of the score you care about
The Difference Score
What are the relevant assumptions the investigating means differences between two DEPENDENT groups (2)
- Observations are independent
2. Observed scores are normally distributed
Why don’t you need to worry about homogeneity of variance when doing DEPENDENT group comparisons?
Nobody knows
When applying contrast weights, is it important to consider which weight to make positive and which to make negative, or is this decision entirely arbitrary?
It’s arbitrary…
When using contrast weights and comparing group difference, what does can you claim when your design is balanced and your contrast weights are orthogonal?
You can claim that…
- the mean differences do not overlap and
- do not contain redundancies
When looking at OBSERVED mean difference scores for two independent groups, what’s the ACTUAL rule for when to look at the EQUAL vs UNEQUAL variance output from R?
Counterintuitively, you the thing you actually have to look at is the normality of the distribution.
When the distribution is normal (or at least, moderately normal), then you should read the ‘EQUAL’ output…
EXCEPT when the design is UNBALANCED and the variance is UNEQUAL
When looking at STANDARDISED mean difference scores for two independent groups, what are the ACTUAL rules for when to look at Hedge’s G and Bonnett’s d, and when to just give up entirely?
- If the distribution is non-normal then just give up (your CIs won’t be robust)
Assuming your distribution is normal… then
- If variances are equal, go for Hedge’s d
- If variances are unequal, go for Bonnett’s d
In R, what does the eff.ci() function give you?
Standardised mean differences for a two group (independent?) one way design… using contrast weights
In R, when looking at the output of a eff.ci() function (for a two group one way design), what is the ‘observed mean contrast’
This is the OBSERVED difference between the two groups you made up using contrast weights
What are two other terms for describing a ‘dependent groups’ design?
- a ‘within subjects’ design
2. a ‘repeated measures’ design
What are two other terms fo describing a ‘within subjects’ design?
- a ‘dependent groups’ design
2. a ‘repeated measures’ design
What are two other terms of describing a ‘repeated measures’ design?
- a ‘within subjects’ design
2. a ‘dependent groups’ design
Name two common applications of dependent groups / within subjects / repeated measure designs?
- multiple measures across time
2. a single group being measured after multiple stimulus (ie reactions to three different images)
Name a circumstance in which a qqplot may not be helpful in ascertaining normality?
When the sample size is teeny weeny
Spehericity means…
The variances of all possible difference scores between pairs of three or more within-subject conditions (or levels) being homogeneous at a population level.
WTF is ‘compound symmetry’
A covariance matrix that has the same variance in each diagonal element and the same covariance in every off-diagonal element of the matrix.
PICTURE THE MATRIX
How does ‘compound symmetry’ relate to any of this?
If you have compound symmetry, by definition you have sphericity
Sphericity can be calculated from a covariance matrix, T/F
TRUE
Sphericity as a population parameter is notated as…
Epsilon
Epsilon’s lower bound value is what?
1/the number of levels in the within subjects factor
In the context of dependent groups…
Perfect Sphericity equals…
1
that is, the smaller the number the lower the sphericity
In the context of dependent groups…
What are the names of the two sphericity estimators?
- Greenhouse-Geissner (epsilon hat)
2. Huynh-Feldt (epsilon with tilde)
In the context of dependent groups…
Greenhouse-Geissner and Huynh-Feldt are examples of what?
Sphericity estimators
In the context of dependent groups…
Of the two sphericity estimators - Greenhouse-Geissner and Huynh-Feldt - which is the more conservative?
Greenhouse-Geissner (epsilon hat)
In the context of dependent groups…
What are orthogonal polynomial contrasts, and what are they used for?
They are a default set of inbuilt contrasts in R
They are used when looking at changes over time to separate out…
the
- linear
- quadratic
- cubic and
- the quartic
In the context of dependent groups…
Orthogonal polynomials provide a complete explanation of all the possible ways in which change is occurring
Okay
In the context of dependent groups…
When we define the key variable as being ‘ordered’,, R will automatically general polynomial coefficients when undertaking an ANOVA
How interesting
Precise meaning of ‘p value’
p = PR(Tobs | H0 = True)
Precise meaning of Type 1 error
Type 1 error = Pr(H0 = TRUE | H0 is rejected)
ie Falsely rejecting a true null hypothesis
Using an Alpha criterion in NHST is done to control for which type of error?
Type 1 error (over the long f run)
If the null hypothesis is false, is it possible to make a type 1 error?
No!
A p value can be thought of as a measure of the consistency or compatibility of our sample data with the null-hypothesised population parameter.
Okay
Does the p value tell us…
the observed effect is due to chance alone?
No!
The p value is calculated on the assumption that chance alone is operating, but does not indicate the probability of chance being the explanation.
Tell me about how CIs relate to p values
A confidence interval calculated in a single sample defines the complete set of null hypothesised values that would not be rejected if used in a NHST test on the sample statistic.
Tell me more about confidence intervals
It is in this sense that we can regard a confidence interval as containing a set of plausible values for the unknown population parameter value.
And more and more about CIs
A confidence interval contains the same fundamental statistical information as a single NHST…but it just contains a lot more of the same type of information.
But what else about CIs
A confidence interval provides us with a range of values for the unknown population parameter to which our data are compatible or consistent in the NHST sense.
Does a confidence interval mean…
a 95% chance of capturing the true effect size
- Any single confidence interval either captures the unknown population parameter value (i.e., the population size of effect), or it does not.
- Over the long run, 95% of all confidence intervals calculated from independently- replicated samples will contain the true effect size.
Does a confidence interval mean…
Any value outside the interval are population effect that are ruled
out
No!
- Values not captured by the interval correspond to null-hypothesised values that would be rejected by a NHST.
- But this does not mean these values can be definitely ruled out as population effect sizes.
- Nor can it be said that values outside the interval have only a 5% chance of being true.
Paul showed that when doing multiple NHST on multiple contrasts (ie NHST that are dependently related - don’t ask what that means), that the chance of a false rejection rate OVERALL was higher than the alpha criterion was not as bad as when the NHST were independent, but it was still pretty high.
What was the characteristic of the thingo that impacted how high that false rejection rate would be?
The mean squared error (MSE)
What is the term for the possible inflation of the false rejection error rate in the case of multiple NHST?
‘Curse of multiplicity’
What curse is the Bonferroni correction a response to?
‘Curse of multiplicity’
What are the names of the two categories of alpha value in the Bonferroni correction?
- ‘per comparions’ alpha value
- ‘family wise’ alpha value
What is the ‘per comparison’ alpha value
The alpa value assign to EACH NHST
What is the ‘family wise’ alpha value
The general, overall alpha for a clutch of NHSTs
How would you calculate the ‘per comparison’ alpha value from the ‘family wise’ alpha value?
Divide the latter by the number of tests you’re going to make
When doing group difference analysis, which form requires sphericity, multi-variate or univariate?
Univariate
In the context of INDEPENDENT group difference, if the design is unbalanced should you check extra hard for homogeneity of variance?
And how would you do that?
Yup totes
Using your two main guys Fligner-Killeen and Levene
When does sphericity even matter tho?
When
- doing within subjects designs, and
- there are 3 or more groups
- and the approach is univariate
In a within subjects design, if you are taking a multivariate approach do you need to worry about sphericity?
Nope!
If the assumption of sphericity is met, which is better, univariate or multivariate?
UNIvariate, comrades
When you are looking at lots of confusing things, and ‘Pillai’ its among them, what should you look at?
Pillai!
But why?`
