2. Confidence Intervals and Logic Of Hypothesis Testing

Question 1

What makes a statement scientific?

Answer 1

It has to be testable and falsifiable

Question 2

If we’d collected 100 samples, calculated the mean and then calculated a confidence interval for that mean, then for 95 of these samples the confidence intervals we constructed would…

Answer 2

contain the true value of the mean in the population

Question 3

What does a significant test statistic tell us?

Answer 3

That the test statistic is larger than we would expect if there were no effect in the population.

Question 4

Of what is p the probability?

Answer 4

p is the probability of observing a test statistic at least as big as the one we have if there were no effect in the population (i.e., the null hypothesis were true).

Question 5

A Type I error occurs when

Answer 5

We conclude that there is an effect in the population when in fact there is not

(False positive)

Question 6

A Type II error occurs when

Answer 6

We conclude that there is not an effect in the population when in fact there is.

(False negative)

Question 7

Power is the ability of a test to…

Answer 7

detect an effect given that an effect of a certain size exists in a population.

Question 8

We can use ____ to determine how large a sample is required to detect an effect of a certain size.

Answer 8

power

Question 9

True or False:

Power is linked to the probability of making a Type II error.

Answer 9

True

Question 10

What is the relationship between sample size and the standard error of the mean?

Answer 10

The standard error decreases as the sample size increases.

• The standard error is the standard deviation of the distribution of sample means
• the sample mean is closer to the population mean

Question 11

In general, as the sample size (N) increases, the confidence interval…

Answer 11

gets narrower

Question 12

In large samples, can small effects be deemed ‘significant’?

Answer 12

Yes

Question 13

The assumption of homogeneity of variance is met when:

Answer 13

The variances in different groups are approximately equal

• To make sure our estimates of the parameters that define our model and significance tests are accurate we have to assume homoscedasticity (also known as homogeneity of variance)

Question 14

If the p value is very small then we conclude that the model….

Answer 14

fits the data well (explains a lot of the variance) and we gain confidence in the alternative hypothesis H1

Question 15

What does effect size tell us?

Answer 15

An effect size is a standardized measure of the size of an effect:

• Standardized = comparable across studies
• Not (as) reliant on the sample size
• Allows people to objectively evaluate the size of the observed effect

Question 16

Effect size (Cohen’s d):

Small =
Medium =
Big =

Answer 16

Small = 0.20
Medium = 0.50
Big = 0.80

Question 17

Effect size (Pearson’s r):

Small =
Medium =
Big =

Answer 17

Small = 0.10
Medium = 0.30
Big = 0.50

Question 18

What does statistical power tell us?

Answer 18

statistical power is the likelihood that a study will detect an effect when there is an effect there to be detected. If statistical power is high, the probability of making a Type II error, or concluding there is no effect when, in fact, there is one, goes down

• It is generally accepted that power should be .8 or greater

Bigger sample size with smaller SD = bigger power

Bigger difference or standard error between H and H1 = bigger power

Question 19

What is Levene’s test?

Answer 19

Tests if variances in different groups are the same

Significant = Variances not equal

Non-Significant = Variances are equal

Question 20

Ways of reducing bias in the data:

Answer 20

Trim the data:

• Delete a certain amount of scores from the extremes

Windsorizing:

• Substitute outliers with the highest value that isn’t an outlier

Analyze with Robust Methods:

• Bootstrapping

Transform the data:

• By applying a mathematical function to scores.
  (log transform, square root transform & reciprocal transform)