Hypothesis Testing

Question 1

What is the null hypothesis?

Answer 1

There is no statistical significant difference between specified population, any observed difference is due to sampling or experimental error (chance).

Question 2

What do we ask when we conduct a hypothesis test?

Answer 2

1) is the value of the test statistic extreme enough for us to reject the null hypothesis?
2) what is our test statistic
3) what should the distribution of the test statistic be if the null hypothesis is true.

Question 3

What is the p-value?

Answer 3

The probability of obtaining a particular value of the test statistic or a more extreme value if the null hypothesis is true.

Question 4

What is alpha?

Answer 4

This is a pre determined level of significance which enables us to decide whether to reject the null hypothesis.

If the p-value is <= alpha we regard it as significant band reject the null hypothesis.

If the p-value is >= alpha we fail to reject the null hypothesis

Question 5

What is a Type I error?

Answer 5

This is when the null hypothesis is actually true but we reject it.

Question 6

What is a Type II error?

Answer 6

This is when the null hypothesis is false but we fail to reject it.

Question 7

What is the probability of making a Type I error?

Answer 7

This is equal to the value of alpha (significance threshold)

Question 8

Write a null hypothesis and a hypothesis for a one sample T-test?

Answer 8

Null hyp: the mean of this sample is equal to the hypothesised value

Hyp: The mean of this sample is different from the hypothesised value.

Question 9

In a T-test in which the null hypothesis is true where should the value of t come from?

Answer 9

The value of t should come from a t- distribution with n-1 degrees of freedom

Question 10

When would you use a Paired T-test?

Answer 10

When there is a pair of measurements per subject and you are measuring a before and after change.

(We still only have n independent data points even if we have 2n numbers)

Question 11

What distribution is tested in a paired T-test

Answer 11

Difference between pairs of values are calculated and then distribution of differences is tested to see if it is equal to 0.

(It’s a one sample T-test where the hypothesised value is always zero)

Question 12

What is a general null hypothesis and hypothesis for a 2 sample T-test.

Answer 12

Null hyp: the means of these two samples are the same

Hyp: the means of these two samples are different

(We are looking at the difference between two mean)

(Difference between two means also has a sampling distribution)

Question 13

What does a 2 sample T-test assume?

Answer 13

1) The data is normally distributed

2) The variances of the two groups are equal

Question 14

If the null hypothesis is true where should the t value come from?

1) For a single sample T-test
2) For a 2 sample T-test

Answer 14

The t-value should come from a t- distribution with n-2 degrees of freedom (where n is the total number of data points from both groups)

Question 15

1) When is a 2 sample T-test robust?

2) What are the exceptions?

Answer 15

1) When sample sizes of both groups are >= 30 (provided groups show equal variances)
2) if each group being compared have similar numbers in each group then it’s ok to use the test even when the standard deviations in the two groups differ by up to 3 fold

Question 16

What is a non parametric substitute for the 2 sample T-test?

Answer 16

Mann Whitney Wilcoxon test

Question 17

State general null hypothesis and hypothesis for Analysis of Variance (ANOVA)

Answer 17

Null hyp: All of the means of these groups are the same

Hyp: at least one of these groups has a different mean from the others

Question 18

When is an Analysis of Variance (ANOVA) used?

Answer 18

Used to test the null hypothesis on a data set with 2 or more groups.

Question 19

What is the test statistic of an ANOVA?

Answer 19

F ratio

Question 20

What is the test statistic for T-tests?

Answer 20

T value

Question 21

What is the F ratio?

Answer 21

F= the treatment Mean square/ Error Mean square

Therefore a ratio between 2 variances

Question 22

What is the Treatment Mean square?

Answer 22

This is the explained variation (signal)

Question 23

What is the Error mean square?

Answer 23

This is the unexplained variation (noise)

Question 24

What is the total sum of squares (in ANOVA)?

Answer 24

This is the treatment sum of squares + the error sum of squares

Question 25

What is the total degrees of freedom (in ANOVA).

Answer 25

Treatment degrees of freedom + error degrees of freedom

Question 26

What determines which f distribution to use for the null distribution?

Answer 26

Total degrees of freedom

Question 27

How are mean squares constructed?

Answer 27

These are constructed by dividing the sum of squares by the degrees of freedom.

Question 28

How do you calculate the treatment mean of square?

Answer 28

Treatment sum of squares/Treatment degrees of freedom

Question 29

How do you calculate the Error mean square?

Answer 29

Error sum of squares/Error degrees of freedom

Question 30

What are F ratios defined by?

Answer 30

1) Whichever number of degrees of freedom are associated with top of F ratio fraction.
2) whichever number are associated with the bottom of the F ratio

Question 31

1) When aren’t F ratios significant?

Answer 31

F ratios aren’t significant when they are <= 1

An f ratio of 1 means the amount of noise is equal to the amount of variation

Question 32

What are Chi squared tests used for?

Answer 32

Chi squared is used to check interactions between count data.

You test whether the distribution of counts for different levels of one factor are consistent with/ have an effect across levels of another factor

Question 33

1) What are the assumptions of a Chi squared test?

2) what is generally accepted?

Answer 33

1) The expected counts must be >= 5

2) If counts aren’t >= 5 it is generally accepted as long as all the expected counts are >= 1 and 80% of them are >= 5.

Question 34

Why is >= 5 important in Chi squared?

Answer 34

Chi squared is constructed to make the assumption that the data follows a normal distribution and that can only be done for a Poisson distribution (count data) if the mean is at least 5.

Question 35

What is a common correction that is used when testing multiple hypotheses on the same data?

Answer 35

Bonferroni correction

Question 36

What is the Bonferroni correction?

Answer 36

If you have an overall significance threshold of alpha and you carry out m hypothesis tests. To reject the null hypothesis for any of the tests the p value must be <= a/m

Question 37

What are the assumptions of ANOVA?

Answer 37

1) Assumes residuals are normally distributed.

2) Assumes variance big unexplained variation is constant throughout the data set ( homogeneity of residuals).

Question 38

How would you correct for right skew?

Answer 38

1) Root data

Or

2) log data

Question 39

How can you correct for left skew?

Answer 39

1) squaring data

Question 40

What are parametric tests?

Answer 40

Tests that rely on pre defined distributions e.g normal distribution

Question 41

What are non parametric tests?

Answer 41

Tests that do not rely on specific pre defined distributions

However they still make some assumptions

Question 42

What is a general null hypothesis for the Mann Whitney Wilcoxon Test?

Answer 42

Two groups being compared come from the same distribution, with the same median but it does not matter what the shape of the distribution is?

Therefore the 2 groups being compared must have similar skews.