Statistical Inference

Question 1

What is hypothesis testing?

Answer 1

procedures to decide if hypotheses about a population statistic can be accepted

Question 2

What is a hypothesis?

Answer 2

Proposition of fact that will be tested

Question 3

What is the alternative hypothesis?

Answer 3

H1 = experimental hypothesis, the thing we would like to be true

Question 4

What is the null hypothesis?

Answer 4

H0 = the opposite of the experiemental hypothesis
e.g. there is no benefit of a new treatment compared to the current standard (this does not mean that the new treatment is worse but that they are the same)

Question 5

Why use null hypotheses?

Answer 5

This is ‘safe’ - we won’t change practice unless the data suggests the current practice (null hypothesis) is incorrect and we reject it

Question 6

What are the two types of hypotheses?

Answer 6

Can be 1 or 2 sided

1 sided - one direction e.g. better or worse
2 sided - different in either direction

Question 7

What is a parallel group study?

Answer 7

A study where 1 group has 1 intervention the other has the other

Question 8

What is a crossover trial?

Answer 8

In this study one group starts with one intervention and then switches to the other intervention (and the other gorup does the same in a different order)

Question 9

What is a p value?

Answer 9

The probability, assuming the null hypothesis is true, that the data (test statistic) you see is at least as big as, or larger than, observed

aka

The probability of coming to the wrong conclusion that we are happy to accept
- there is a small random chance that the sample we took doesn’t represent the population

Question 10

When is a p value statistically significant?

Answer 10

With a 95% ‘signifience level’, any p
< 0.05 is statistically significant

Question 11

What is the difference between clinical and statistical significance?

Answer 11

e.g. progression free survival of 0.42 weeks might be stastisitically significant but isn’t clinically signficiant

whereas a pain score of 85 vs 65 could not be statistically significant but is clincially signficiant

sometimes a clincally significant result with a statistically insignificant result can be restudied with a larger trial

Question 12

What is a type 1 error?

Answer 12

aka
* ‘false positive’ rate
* significance level
* alpha

the study finds a differene due to the random sample of the data and wrongly rejects the null hypothesis when the null hypothesis is true and there actually isn’t a difference
therefore clinical practice might get chnaged nincorrectly

Question 13

What is a type II error?

Answer 13

aka
* false negative
* beta

the study finds no difference and wrongly accepts the null hyothesis when in reality there is a difference
therefore clinical practice doesn’t change when it should

This is due to statistical power - the study needs to be big enough

We never actually know if we have made this error we just try to power the study enough that we don’t make it

Question 14

What is statistical power?

Answer 14

Power = 1-B
set at study design stage

Question 15

What is the issue caused by type I errors?

Answer 15

We are wrong 5% of the time but we don’t know when!!

Question 16

How do multiple tests affect Type I errors?

Answer 16

Multiple tests (comparisons) increases the risk of a type I error (false positive) - each time there’s a 5% chance of error

Question 17

How can Type I errors be avoided or dealt with?

Answer 17

1. Ignore them
2. Control the familywise error rate - aim for a 5% error rate across all the tests together
3. Control the false discovery rate - make sure each individual test has a 5% of error

Question 18

Ignoring type I errors

Answer 18

leave other people to apply their own correction criteria on your data
but leaves the risk of Type I error unmodified

Question 19

What is familywise error rate?

Answer 19

family means tests in the same sample

this controls the overall type I error rate of all the tests combined

This is a very strong control and means results will be conservaive

Question 20

What are the two corrections that can control the familywise error rate?

Answer 20

Bonferri correction - divide alpha by number of tests and use that as new threshold. E.g. if 6 tests 0.05/6
Sidak correction - less conservative

Question 21

What is ‘false discovery’ rate?

Answer 21

Make each individual test error rate 5% to control the overall rate of type I errors

so instead of 5% chance of type I error overall a 5% chance of all tests being a tyoe I error is maintained, so overall could be wrong 5% of the time

Benjamin-Hochberg procedure