L05 Comparing means Flashcards
How do we compare means?
t-tests
What do t-tests check?
Whether the difference between means of two groups or conditions is statistically significant
How likely it is that difference between comparisons could be attributed to sampling error if H0 is true.
Effect size
Measuring the size of the observed effect usually relative to background error
Degree to which differences in dependent variable are attributed to independent variable
Types of t-tests
- Independent or Unrelated samples t-test
- Paired/Dependent/Related samples t-test
- One-sample t-test
Who devised t-test?
William Gossett (pseudonym Student)
Test statistic for t-test
Student’s t or t value
Independent t-test - what?
Also called unrelated t-test or between-participants
Used in between-subject design
to compare means that come from conditions consisting of different entities
Dependent t-test - what?
Paired samples or Related t-test
Within-subject or repeated measures design
to compare means that come from the same or from related entities
One sample t-test - what?
Compare the mean of a group to a pre-defined value
Rationale behind t-test
Signal-to-Noise ratio - ratio of systematic variance to unsystematic variance
or a ratio of the measure of between-group variance (due to difference in groups or experimental manipulation) to within-group variance (error)
t-test general formula
Observed difference between means (- estimated difference between means under H0 = 0)
divided by Estimated standard error in the Difference of means
Assumptions of t-test
(parametric test)
- Data: ratio or interval scale
- Normally distributed data* (because parametric test; but t-tests are quite robust to normality - slight skews are okay)
- Data in one group should be independent of the data in other
- Homogeneity of variance
* For repeated measures, normality assumption refers to the normal distribution of the differences between scores, not the scores themselves
Testing Normality for a t-test
Kolmogorov-Smirnov test
(or Shapiro Wilk test for N<50 - more power)
should be NON-significant
(t-tests are quite robust to normality; can be ignored for n>100)
Homogeneity of variance
Aka homoscedasticity
assumption that the spread of outcome scores is roughly equal at different points on the predictor variable
Tested with Levene’s test
Evaluated by testing standardized predicted values to standardized residuals from the data (zpred to zres)
Testing Homogeneity of variance for a t-test
Levene’s test
Beware of the issues that come with using a measure of NHST
Systematic variance
Variation as a consequence of manipulation of IV
-
Unsystematic variance
Variation due to uncontrolled factors
What to report in reporting t-test
For both groups (M=_, SD = _)
t (df) = test statistic (p = associated p value)
df for independent t-test
(n-1) + (n-1) where each term corresponds to each sample - or N-2
df for a dependent/related t-test
n-1
df for within participants or one sample t-test
n-1
Degrees of Freedom
“freedom to vary”
The number of individual scores that can vary without changing the sample means
or without breaking any constraints
Why we need df for a t-test
We need df to calculate p value from test statistic t
Changes probability distribution of the test statistic
More df, smaller t gives significant results
5 values you report for every test
- Descriptive stats of the variable (center and spread)
- Name of test
- Value of test statistic
- Degrees of Freedom
- Actual value of p
