Differences between means and effect size

Question 1

When there are 2 variables/samples in a study, what do we look for?

Answer 1

Differences between means or relationships between variables

Question 2

Finding a difference between data in our conditions depends upon…?

Answer 2

1) How far apart our pop. means are
2) How much variability there is in the pop.
3) How do means and variances/s.d. interact with our samples

Question 3

Less spread in our sample data most likely means that there’s…?

a) More variability in the population
b) Less variability in the population
c) Less variability in the s.d.
d) More variability in the s.d.

Answer 3

b) Less variability in the population

Question 4

How do we assess differences between 2 conditions?

Answer 4

1) Calculate descriptive statistics and compare

e.g. Means, medians, s.d., confidence intervals

2) Calculate “effect size” using Cohen’s d*

• A measure of distance between means of two conditions which takes variability into account

3) Use some kind of inferential test based on known probability distributions

Question 5

What does Cohen’s d measure?

Answer 5

Measures difference in means and also taking s.d. into account (takes into account how variable data are)

Question 6

What is the formula for Cohen’s d population mean?

Answer 6

d pop. = (pop. mean 1 - pop. mean 2) / pop. mean SD

Question 7

What is the formula for Cohen’s d sample mean?

Answer 7

d sample = (sample mean 1 - sample mean 2) / sample mean SD

Question 8

Cohen’s d measures how many s.d.’s apart the means for the 2 variables/conditions are. What does an increase in overlap tell us about the effect size?

Answer 8

Increase overlap = Increase effect size

Question 9

Cohen’s d measures how many s.d.’s apart the means for the 2 variables/conditions are. What does a decrease in overlap tell us about the effect size?

Answer 9

Decrease overlap = Decrease effect size

Question 10

What constitutes a big effect size?

Answer 10

0.8

Question 11

What constitutes a small effect size?

Answer 11

0.2

Question 12

What constitutes a medium effect size?

Answer 12

0.5

Question 13

What does an overlap of d = 0.2 mean in terms of percentages?

Answer 13

85% chance (higher chance) that your samples are close together

Question 14

What does an overlap of d = 0.8 mean in terms of percentages?

Answer 14

53% chance (lower chance) that your samples are close together

Question 15

What does an overlap of d = 0.5 mean in terms of percentages?

Answer 15

67% chance (in between high and low chances) that your samples are close together

Question 16

Calculate Cohen’s d for your samples

You measure weight gain in a month in 2 experimental conditions in which participants eat only pies or only salads

Sample statistics for these data are shown below:

mean salads = -0.2kg
sd salads = 1.2kg

mean pies = 0.7kg
sd pies = 1.0kg

Answer 16

Formula=
1) d sample = (mean pies - mean salads) / mean SD
2) mean SD = (SD 1 + SD 2) / 2

Calculation =
1) d sample = (0.7 - (-0.2)) / 1.1 = 0.82
2) mean SD = (1.0 + 1.2) / 2 = 1.1

Answer = 0.82 (2 d.p.)

Question 17

The primary sign of health and vitality on planet Ziltoidia10 is a ‘skullet’ - long hair at the sides and back with a bald patch in the middle, with Ziltoidians believing in particular that a large bald patch is a sign of excellent genetics. Visiting aliens from Ziltoidia10 believe that being on earth is leading to more rapid central balding in male Ziltoidians. To test this hypothesis a newly arrived (NEW) cohort of ziltoidians are compared to an (OLD) cohort of ziltoidians who have been on earth for 1 Ziltoidian year (3.14 Earth years). The sample means and sample s.d.s of bald patch areas for the NEW and OLD groups are:

NEW
mean = 0.4 cm squared, s.d. = 0.1 cm squared

OLD
mean = 0.475 cm squared, s.d. = 0.15 cm squared.

What is Cohen’s d (to 1 d.p.)?

Answer 17

Formula=
1) d sample = (s. mean new - s. mean old) / mean SD
2) mean SD = (s. SD 1 + s. SD 2) / 2

Calculation =
1) d sample = (0.4 - 0.475) / 0.125 = 0.6
2) mean SD = (0.1 + 0.15) / 2 = 1.125

Answer = 0.6 (1 d.p.)

Question 18

A clinical psychologist wishes to investigate a new intervention designed to reduce anxiety levels. She collects anxiety questionnaire data from 25 people pre-and post-intervention. The data are summarised below. Work out Cohen’s d (to 2.d.p.) for the effect of the intervention.

Pre-intervention: mean = 15.3, s.d. = 1.1

Post-intervention: mean = 14.5, s.d. 1.2

Answer 18

Formula=
1) d sample = (s. mean pre - s. mean post) / mean SD
2) mean SD = (s. SD 1 + s. SD 2) / 2

Calculation =
1) d sample = (15.3 - 14.5) / 0.15 = 0.70
2) mean SD = (1.1 + 1.2) / 2 = 1.15

Answer = 0.70 (2 d.p.)

Question 19

A researcher has symptom severity data from two groups with a clinical diagnosis of Schizophrenia who have undergone different treatments for their condition. Sample means and s.d.s are presented below. Work out Cohen’s d to 2 d.p. for the effect of the treatment.

Treatment group 1: mean 12.4, s.d. 3.7

Treatment group 2: mean 13.4, s.d. 3.9

Answer 19

Formula=
1) d sample = (s. mean 1 - s. mean 2) / mean SD
2) mean SD = (s. SD 1 + s. SD 2) / 2

Calculation =
1) d sample = (12.4 - 13.4) / 3.8 = 0.26
2) mean SD = (3.7 + 3.9) / 2 = 3.8

Answer = 0.26 (2 d.p.)