Statistical Tests

Question 1

What is a hypothesis?

Answer 1

• allows you to state your idea in a specific testable form
• used to describe a working theory about the data sets you’re considering

Question 2

Define ‘null hypothesis’

Answer 2

• a working assumption that there’s no difference between the data sets you wish to compare
• i.e. there is no difference/relationship/association

Question 3

Define ‘alternative hypothesis’

Answer 3

• assumption that there is a difference between data sets
• i.e. there is a difference/relationship/association

Question 4

Define ‘significance’

Answer 4

• a measure of likelihood that the NULL hypothesis is correct

Question 5

Define ‘two-sided hypothesis’

Answer 5

• states the difference could be in either direction
• null = NO difference between methods A and B
• alternative = A difference between methods and B

Question 6

Define ‘one-sided hypothesis’

Answer 6

• states a difference in a specific direction
• alternative = test results using method A are HIGHER/LOWER than those using method B
• null = test results using method A are NOT HIGHER/LOWER than those using method B

Question 7

What is a ‘type I error’?

Answer 7

• false positive (reject the NH when it is true)

Question 8

What is a ‘type II error’?

Answer 8

• false negative (accept the NH when it is false)

Question 9

What is the type II error rate (b)?

Answer 9

• the probability of incorrectly retaining a false null hypothesis

Question 10

What is meant by ‘power’?

Answer 10

• the probability of correctly rejecting a false null hypothesis
• power = 1 - b (type II error rate)

Question 11

How can we reduce the chance of type I error?

Answer 11

• choose a lower probability/higher significance
• e.g. P = 0.01

Question 12

What is the issue of choosing a higher significance?

Answer 12

• critical value of test statistic increases
• thus probability of type II error increases

Question 13

What are the significance levels (P-values) and what they mean?

Answer 13

• P > 0.05 = not significant
• P < or = 0.05 = significant
• P < or = 0.01 = highly significant
• P < or = 0.001 = very highly significant

Question 14

What is a parametric test?

Answer 14

• makes particular assumptions about mathematical nature of population distribution from which the samples were taken
• better able to distinguish between true and marginal differences between sample (have greater ‘power’)

Question 15

What is a non-parametric test?

Answer 15

• doesn’t assume that data fit a particular pattern, but may assume some characteristics of their distributions

Question 16

What is ‘effect size’ (ES)?

Answer 16

• measures the strength of the result is solely magnitude-based
• it does not depend on sample size

Question 17

What is meant by ‘A priori power analysis’?

Answer 17

• done in planning phase to determine N (study size)
• necessary to justify project resources in funding processes
• minimise use of animals or risk to patients in clinical research
• involves estimating the sample size required for a study based on predetermined maximum tolerable Type I and II error rates and the minimum effect size that would be clinically, practically, or theoretically meaningful

Question 18

What is meant by ‘Post-hoc power analysis’?

Answer 18

• done on completion of study to determine Observed Power
• Necessary to check that your expected and measured ES align well
• i.e. Did you have sufficient subjects to detect differences reliably?

Question 19

What is meant by ‘Sensitivity power analysis’?

Answer 19

• done in planning phase when the sample size is predetermined by study constraints e.g. if there are only 20 subjects available in a pilot study
• instead we determine what level of effect we might be able to find, referred to as the minimal detectable effect (MDE) or minimum clinically important difference (MCID)

Question 20

How does sample size affect the effect size?

Answer 20

• as sample size increases, the effect size decreases

Question 21

What is the relationship sample size and power?

Answer 21

• increase in sample size increases power
• but NOT a linear relationship

Question 22

When would you use a t-test?

Answer 22

• comparing means from TWO independent samples

Question 23

What other situations would you use a t-test?

Answer 23

• comparing means of paired data
• comparing a sample mean with a chosen value

Question 24

When would you use ANOVA?

Answer 24

• comparing means from TWO OR MORE samples

Question 25

Similarity between t-test and ANOVA

Answer 25

• both assume data has a Gaussian distribution and variances of the samples are homogeneous

Question 26

What are the 2 chief non-parametric tests for comparing locations of 2 samples?

Answer 26

• Mann-Whitney U-test
• Kolmogorov-Smirnov test

Question 27

What does the Mann-Whitney U-test assume and what sample’s size should it be?

Answer 27

• assumes that the frequency distributions of samples are similar
• sample’s size must more or equal to 4

Question 28

What sample’s size must Kolmogorov-Smirnov test have?

Answer 28

• more or equal to 4
• samples must have equal sizes

Question 29

Significant differences found with the Kolmogorov-Smirnov test could be due to what?

Answer 29

• differences in location
• or shape of distribution
• or both

Question 30

What are the 2 suitable non-parametric comparisons of location of paired data?

Answer 30

• Wilcoxon’s signed rank test
• Dixon and Mood’s sign test

Question 31

What is the Wilcoxon’s signed rank test?

Answer 31

• used for quantitative data
• assumes that distributions have similar shape
• sample size equal to or more than 6

Question 32

What is the Dixon and Mood’s sign test used for?

Answer 32

• paired data scores where one variable is recorded as ‘greater than/better than’ the other
• sample size equal to or more than 6

Question 33

What are the 2 suitable non-parametric comparisons of location for 3 or more samples?

Answer 33

• Kruskal-Wallis H-test
• Friedman S-test

Question 34

What is the Kruskal-Wallis H-test?

Answer 34

• number of samples is without limit and can be unequal in size
• underlying distributions are assumed to be similar