L05 Comparing means

Question 1

How do we compare means?

Answer 1

t-tests

Question 2

What do t-tests check?

Answer 2

Whether the difference between means of two groups or conditions is statistically significant
How likely it is that difference between comparisons could be attributed to sampling error if H0 is true.

Question 3

Effect size

Answer 3

Measuring the size of the observed effect usually relative to background error
Degree to which differences in dependent variable are attributed to independent variable

Question 4

Types of t-tests

Answer 4

1. Independent or Unrelated samples t-test
2. Paired/Dependent/Related samples t-test
3. One-sample t-test

Question 5

Who devised t-test?

Answer 5

William Gossett (pseudonym Student)

Question 6

Test statistic for t-test

Answer 6

Student’s t or t value

Question 7

Independent t-test - what?

Answer 7

Also called unrelated t-test or between-participants
Used in between-subject design
to compare means that come from conditions consisting of different entities

Question 8

Dependent t-test - what?

Answer 8

Paired samples or Related t-test
Within-subject or repeated measures design
to compare means that come from the same or from related entities

Question 9

One sample t-test - what?

Answer 9

Compare the mean of a group to a pre-defined value

Question 10

Rationale behind t-test

Answer 10

Signal-to-Noise ratio - ratio of systematic variance to unsystematic variance
or a ratio of the measure of between-group variance (due to difference in groups or experimental manipulation) to within-group variance (error)

Question 11

t-test general formula

Answer 11

Observed difference between means (- estimated difference between means under H0 = 0)
divided by Estimated standard error in the Difference of means

Question 12

Assumptions of t-test

Answer 12

(parametric test)

1. Data: ratio or interval scale
2. Normally distributed data* (because parametric test; but t-tests are quite robust to normality - slight skews are okay)
3. Data in one group should be independent of the data in other
4. Homogeneity of variance
   * For repeated measures, normality assumption refers to the normal distribution of the differences between scores, not the scores themselves

Question 13

Testing Normality for a t-test

Answer 13

Kolmogorov-Smirnov test
(or Shapiro Wilk test for N<50 - more power)
should be NON-significant
(t-tests are quite robust to normality; can be ignored for n>100)

Question 14

Homogeneity of variance

Answer 14

Aka homoscedasticity
assumption that the spread of outcome scores is roughly equal at different points on the predictor variable
Tested with Levene’s test
Evaluated by testing standardized predicted values to standardized residuals from the data (zpred to zres)

Question 15

Testing Homogeneity of variance for a t-test

Answer 15

Levene’s test

Beware of the issues that come with using a measure of NHST

Question 16

Systematic variance

Answer 16

Variation as a consequence of manipulation of IV

-

Question 17

Unsystematic variance

Answer 17

Variation due to uncontrolled factors

Question 18

What to report in reporting t-test

Answer 18

For both groups (M=_, SD = _)

t (df) = test statistic (p = associated p value)

Question 19

df for independent t-test

Answer 19

(n-1) + (n-1) where each term corresponds to each sample - or N-2

Question 20

df for a dependent/related t-test

Answer 20

n-1

Question 21

df for within participants or one sample t-test

Answer 21

n-1

Question 22

Degrees of Freedom

Answer 22

“freedom to vary”
The number of individual scores that can vary without changing the sample means
or without breaking any constraints

Question 23

Why we need df for a t-test

Answer 23

We need df to calculate p value from test statistic t
Changes probability distribution of the test statistic
More df, smaller t gives significant results

Question 24

5 values you report for every test

Answer 24

1. Descriptive stats of the variable (center and spread)
2. Name of test
3. Value of test statistic
4. Degrees of Freedom
5. Actual value of p

Question 25

Paired samples t-test formula

Answer 25

Average difference of means /

Standard error of differences

Question 26

Independent t-test formula

Answer 26

Mean1 - Mean2 /
Estimate of the standard error

where estimate of SE is calculated by adding variances together (variance sum law) and taking the square root - if n is the same; otherwise, pooled variance estimate