Hypothesis Testing & Power Analyses

Question 1

Goal of a Hypothesis Test

Answer 1

To rule out chance as a plausible explanation for the results from a research study

Question 2

Null Hypothesis (H0)

Answer 2

Assumes there is no treatment effect/difference between conditions/groups

First step in hypothesis testing

Question 3

Alternative Hypothesis (H1)

Answer 3

What you are hoping to find

Hoping to be able to reject H0, in favour of H1

Question 4

Statistical Inference procedure

Answer 4

• Make an assumption (H1)
  1. State H0
  2. Select statistical test and significance level
  3. Select sample and collect the data
  4. Find the region of rejection
  5. Calculate the test statistic
  6. Make the statistical decision

Question 5

Central Limits Theorem

Answer 5

Distribution of means derived from sets of random samples taken from any population will trend toward normality

Greater the sample, greater the conformity to normal

Question 6

p-value

Answer 6

The probability of observing a sample mean that is (at least) as far from the population mean as the value we have observed

Question 7

Z-scores

Answer 7

0 represents mean of normal distribution
Each integer represents one SD away from mean

i.e. z = -1.5 corresponds to mean value minus 1.5*SD

Question 8

Alpha level

Answer 8

Probability threshold, determined by what amount of risk (of Type II error) we deem acceptable

p-value below the threshold -> reject H0

Chosen prior to the experiment

Typically 0.05

Question 9

Type I and Type II errors

Answer 9

Type I: incorrectly rejecting H0 (false positive)

Type II: incorrectly failing to reject H0 (false negative)

Question 10

One-tailed vs Two-tailed tests

Answer 10

Depends on whether or not it matters which direction the effect will be in

Two-tailed test will produce a bigger p-value

e.g. if alpha = 0.05, then 0.025 is assigned to each side

Question 11

Critical Value

Answer 11

Z-value defining the rejection region

Function of the chosen alpha level

Question 12

Cohen’s d

Answer 12

Standardised measure of effect size:

Distance between two means, measured in SDs

Question 13

Power

Answer 13

Probability of being able to reject H0 when it would be correct to do so

Depends on:

1) alpha level
2) (predicted) effect size
3) sample size

0.8 usually desired

Question 14

Power Analysis

Answer 14

Calculating how big the sample needs to be for desired amount of power

e.g. small predicted effect + small sample –> low power
Solution? Increase sample size