Confidence Intervals/Hypothesis Testing

Question 1

95% Confidence Interval

Answer 1

Range of scores constructed such that the population mean will fall within this range in 95% OF SAMPLES

NOT an interval within which we are 95% confident that the population mean will fall.

Question 2

Null Hypothesis

Answer 2

An effect doesn’t exist

Question 3

Alternative hypothesis

Answer 3

An effect exists

Question 4

Test statistic

Answer 4

Calculate the probability that we would get a value as big as the one we have if the null hypothesis were true.

Question 5

Type 1 error

Answer 5

We believe that there is an effect when there isn’t

Question 6

Type 2 error

Answer 6

We believe there is no effect when there is

Question 7

Power

Answer 7

Probability that a test will find an effect when one exists

Question 8

Effect of sample size

Answer 8

Same effect will have different p-values in different-sized samples

Small differences can be deemed ‘significant’ in large samples

Large effects might be deemed ‘non-significant’ in small samples.

Question 9

Disadvantages of NHST

Answer 9

All-or-nothing thinking (0.05 cut-off)

Biased by researchers deviating from their initial sampling frame (e.g., by stopping data collection earlier than planned).

Question 10

Researcher degrees of freedom

Answer 10

Scientists can influence p-value:

selective exclusion of data

fitting different statistical models but reporting only the one with the most favourable results

stopping data collection at a point other than that decided at the study’s conception

including only control variables that influence the p-value in a favourable way.

Question 11

P-hacking

Answer 11

Practices that lead to the selective reporting of significant p-values

i.e trying multiple analyses and reporting only the one that yields significant results

Question 12

HARKing

Answer 12

Hypothesising after results are known