Significance Tests

Question 1

What is the null hypothesis?

Answer 1

States the two treatments are equally effective.

Question 2

Purpose of a significance test?

Answer 2

Uses the sample data to assess how likely to null hypothesis is to be correct.

E.g. ‘there is no difference in the prevalence of colorectal cancer in patients taking low-dose aspirin compared to those who are not’.

Question 3

What is the alternative hypothesis?

Answer 3

The opposite of the null hypothesis –> i.e. there is a difference between the treatments.

Question 4

What is the p value?

Answer 4

The probability of obtaining a result by chance at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.

It is therefore equal to the chance of making a type I error (see below).

Question 5

What are the two types of errors that may occur when testing the null hypothesis?

Answer 5

1) Type I
2) Type II

Question 6

What is a type I error?

Answer 6

The null hypothesis is rejected when it is true.

I.e. Showing a difference between two groups when it doesn’t exist, a false positive.

This is determined against a preset significance level (termed alpha).

Question 7

Is the chance of making a type I error affected by sample size?

Answer 7

No - as the significance level is determined in advance

Question 8

What increases the chance of making a type I error?

Answer 8

If the number of end-points are increased.

For example if a study has 20 end-points it is likely one of these will be reached, just by chance.

Question 9

What is a type II error?

Answer 9

The null hypothesis is accepted when it is false.

i.e. Failing to spot a difference when one really exists, a false negative.

Question 10

What is the probability of making a type II error termed?

Answer 10

Beta

Question 11

What is the probability of making a type II error determined by?

Answer 11

Sample size & alpha

Question 12

What error is made when:

The study accepts the null hypothesis (H0) but the reality is the alternative hypothesis (H1)?

Answer 12

Type II error (beta)

As the study as failed to spot a difference when one really exists (false negative).

Question 13

What error is made when:

The study rejects the null hypothesis (H0) and the reality is the null hypothesis (H0)?

Answer 13

Type I error (alpha)

As the study has shown a difference between two groups when one doesn’t exist (false positive).

Question 14

What is the power of a study?

Answer 14

The probability of (correctly) rejecting the null hypothesis when it is false

i.e. the probability of detecting a statistically significant difference

Question 15

Formula for power?

Answer 15

Power = 1 - the probability of making a type II error

Question 16

What can power be increased by?

Answer 16

Increasing the sample size

Question 17

What does the type of significance test used depend on?

Answer 17

Where the data is parametric (something which can be measured, usually normally distributed) or non-parametric.

Question 18

What are the 2 types of parametric significance tests?

Answer 18

1) Student’s t-test: paired or unpaired

2) Pearson’s product-moment coefficient: correlation

Question 19

What does paired data refer to?

Answer 19

Data obtained from a single group of patients e.g. measurement before and after an intervention.

Question 20

What does unpaired data refer to?

Answer 20

Comes from two different groups of patients e.g. comparing response to different interventions in two groups

Question 21

What are the 4 types of non-parametric significance tests?

Answer 21

1) Mann-Whitney U test

2) Wilcoxon signed-rank test

3) Chi-squared test

4) Spearman, Kendall rank

Question 22

What is the chi-squared test used for?

Answer 22

Used to compare proportions or percentages e.g. compares the % of patients who improved following two DIFFERENT interventions.

Question 23

Purpose of significance tests?

Answer 23

To determine if the results of an experiment support or reject a null hypothesis (H0), and if an alternative hypothesis (H1) can be accepted

Question 24

What is a Wilcoxon signed rank test used for?

Answer 24

Compares two sets of observations on a single sample, e.g. a ‘before’ and ‘after’ test on the same population following an intervention.

Question 25

What is a Spearman’s rank correlation coefficient and Kendall’s rank correlation coefficient used for?

Answer 25

measure the strength and direction of the association between two variables