Lecture 1

Question 1

What is the methodology during NHST?

Answer 1

1. Researcher has a research question
2. Formulates a null hypothesis
3. Collects data
4. Either reject or accept null hypothesis

Question 2

Why would it not be possible to demonstrate the null hypothesis?

Answer 2

A non-significant result could be due to the null-hypothesis being true OR a failure to gather sufficient evidence

Question 3

How do researchers set up their research in terms of the null hypothesis?

Answer 3

So that the ‘desired’ outcome rejects the null hypothesis

Question 4

List three factors which lead to statistical significance

Answer 4

Sample size, critical alpha, effect size

Question 5

Why is statistical significance not practical significance?

Answer 5

With a sufficiently large sample, very small effects can become statistically significant, although they may be irrelevant for any practical purpose

Question 6

What does significant mean in statistics?

Answer 6

In statistics, significance implies that something unlikely to have happened occurred by chance (and may therefore have a systematic cause)

What is considered to be ‘unlikely’ depends on an arbitrarily defined significance threshold

A critical perspective: significance at a 5% threshold indicates limited evidence that the data is not entirely random

Question 7

Why is the null hypothesis always wrong?

Answer 7

You are assuming that if you had 2 groups, the difference between the 2 groups is exactly the same - unrealistic

Question 8

What are some proposed alternatives to NHST?

Answer 8

• Effect size
• Confidence intervals
• Bayesian statistics
• Minimum-effect null-hypothesis: doesn’t assume the difference is 0, but that it is very small
• Confirmation of a null hypothesis: showing an effect is significantly smaller than some small effect or that it is smaller than the effect found in previous studies

Question 9

One problem with NHST is that you have to make some comparison to another figure - why is this a problem?

Answer 9

This is significant in comparison to what was found in another study

Question 10

What is effect size?

Answer 10

It provides an estimate of the size of group differences or the effect of treatment

Question 11

Name some uses of effect size

Answer 11

• Measure of how large an effect is
• Used in estimating the sample size needed for sufficient statistical power
• Used when combining data across studies (meta-analysis)

Question 12

Name the types of effect sizes

Answer 12

• Group difference indices (e.g. Cohen’s d)
• Strength of association (‘variance explained’, e.g. eta squared, R squared)
• Risk estimates (e.g. relative risk)

Question 13

Give some examples of group differences

Answer 13

• Males vs. females
• Treatment vs. control group
• Young vs. old participants

Question 14

How do you calculate effect size?

Answer 14

The difference in means

Question 15

What is a disadvantage of the way we calculate effect size?

Answer 15

Measure is dependent on measurement scale

Question 16

Name a disadvantage of Cohen’s d and Hedge’s g

Answer 16

• Cohen’s d slightly overestimates effect size, leading to it being biased
• Hedge’s g corrects for this, but multiplying it by a value which is slightly bigger than one used by Cohen’s d, meaning it is unbiased

Question 17

How do you calculate Cohen’s d?

Answer 17

The difference between the sample means/SD pooled

Question 18

How do we classify effect size?

Answer 18

Between 0.2-0.49 = small effect size
Between 0.5-0.79 = medium effect size
0.8 or higher = large