HYPOTHESIS TESTING - LEARNING OUTCOMES

Question 1

What are the general measures we would look out when studying continuous data with a normal distribution?

Answer 1

We would look at the mean and standard deviation followed by the mean difference between the two groups - preferably with the associated 95% confidence interval.

Question 2

What does hypothesis testing essentially test for?

Answer 2

Hypothesis testing calculates the likelihood that the difference we are seeing between two groups actually happened by chance and that the null hypothesis (no difference) is actually true. How likely it is that we would get these results if there is actually no difference between our two groups?

Question 3

Summarise hypothesis testing.

Answer 3

We start by specifying the study hypothesis and the null hypothesis. We assume the null hypothesis is true and proceed to calculate the probability of getting the observed difference by chance - this is what is termed the p-value

Question 4

What does a small p-value indicate?

Answer 4

A small p-value implies that there is a statistically significant difference between our two groups. In this case we can reject the null hypothesis.

Question 5

What does a large p-value indicate?

Answer 5

A large p-value tells us that there is no evidence of a difference between the groups. In this case we would fail to reject the null hypothesis (not the same as saying we accept the null hypothesis, there may still be an effect there but our study is not powerful enough to detect it).

Question 6

What is accepted as a small p-value?

Answer 6

Convention is to use p=0.05 as a cut off. Less that 0.05 we would term a significant difference. More than 0.05 we would say there is no evidence of difference. However, 0.05 is not a magic figure. It is better to give the actual figure and let readers make up their own minds.

Question 7

What is a type I error?

Answer 7

A type I error is to reject the null hypothesis when it is actually true. This is essentially a false positive result. The frequency of this type of error is the same as the alpha level (significance cut off).

Question 8

What is a type II error?

Answer 8

A type II error is the failure to reject the null hypothesis when it is actually false. This is essentially a false negative result. Type II error is very much dependant on the sample size and power of the study.

Question 9

How do we decide which hypothesis or statistical test to carry out?

Answer 9

Depends on the type of outcome variable and the number of groups you are analysing. Depends on a number of criteria about the data.

Question 10

Give an example of a paired sample.

Answer 10

Measurements before and after an event - e.g. heart rate before and after a period of exercise.

Question 11

What 3 things does the p-value depend on?

Answer 11

1. How big the observed difference is
2. Sample size
3. Variability of measurement

Sample size and variability in measurement are related to each other in terms of the standard error where standard error is the standard deviation divided by the square root of your sample size.

Question 12

How is the T-statistic calculated?

Answer 12

T-statistic = Observed mean difference / Standard error of the difference between means

Look this value up in tables of normal distribution or use a stats programme to do this for you.

Question 13

What assumptions are required for a t-test to be applicable?

Answer 13

1. The outcome should be continuous and must be following a normal distribution
2. The variance of the two groups is equal. SPSS will test this for you automatically using Levene’s test.

Question 14

What is the first thing we need to look at in the output table for the difference between two means in SPSS?

Answer 14

We first need to look at and interpret the Levene’s test results before we even think about interpreting the output from a t-test.

For the Levene’s t-test the null hypothesis is that there is no difference in variance between the two groups.

Essentially a significant result for Levene’s test means that there is a difference in variance between the two groups.

If the result from the Levene’s test is not significant then we fail to reject our null hypothesis of no difference between variance and we can use the top line of SPSS output.
(Levene’s test more than 0.05 then use top line).

If the result from the Levene’s test is significant then we reject our null hypothesis of no difference between variance and use the bottom line of SPSS output representing the adjusted t-test that hasn’t assumed equal variance.
(Levene’s test less than 0.05 then use bottom line).

Often the two lines of output will give you the same conclusion, but there may be a difference in confidence intervals.

Question 15

We would want to back up the significance results from a t-test with a measure of effect - what measure of effect would be most likely use for this?

Answer 15

Mean difference with the associated confidence intervals

Question 16

When our data aren’t drawn from a normal distribution and we therefore can’t use a t-test we can attempt to use data transformation. Describe the general principles of data transformation.

Answer 16

Firstly you can try transforming the data with an algorithm and then carry out analysis on that data. You can’t assume the transformation is going to work and give you a variable that is following a normal distribution - still need to check those assumptions e.g. plot histogram for normality. If the data is ok can then use a parametric test (e.g. t-test_ on the transformed variable.

Question 17

What sort of data transformations can we try and when are they appropriate?

Answer 17

The transformations we may try are dependant upon what the data looks like. e.g:

moderate positive skew - logarithm
strong positive skew - reciprocal
weak positive skew - square root

moderate negative skew - square
strong negative skew - cube

unequal variation - log, reciprocal, square root

Question 18

What can we do with continuous data not drawn from a normal population if transforming the data proves to be unsuccessful?

Answer 18

We can use non-parametric tests. These are tests that are designed for data where you don’t have to worry about the underlying distributions.

Question 19

What are the advantages and disadvantages if using non–parametric tests?

Answer 19

Advantages:
- Make no assumptions about underlying distribution of data

Disadvantages:

• Less powerful than parametric tests
• Difficult to get confidence intervals

Question 20

How do we describe skewed variables?

Answer 20

If data is not normally distributed then:

• We need to present the medians not the means
• We need to present the range or interquartile range, not the standard deviation
• If we are comparing two groups of non-parametric data then we need to present the difference between the two medians (but can’t easily get 95% confidence intervals)

(Median and IQR)

Question 21

What is the non-parametric equivalent of the t-test?

Answer 21

The Wilcoxon rank sum test or the Mann-Whitney U test.

These tests are appropriate if you have two independent groups with a continuous variable that isn’t following a normal distribution.

Question 22

How does the Wilcoxon rank sum test work?

Answer 22

It ranks the data and then works on the rank of the data rater than the raw variables themselves. Most non-parametric tests work on rank.

For example:

We have two independent groups group 1 and group 2 where group 1 is the smallest sized group.

We rank all observations into ascending order.

The sum of ranks for group 1 = test statistic T.

Look up T on Wilcoxon rank sum table of critical values to get P value.

In practice you would do this using SPSS.

Question 23

What is the Kruskal-Wallis test?

Answer 23

Another non-parametric test. It is an extension of the Mann-Whitney test for use when you have more than 2 groups to compare. It is the non-parametric equivalent of the ANOVA.

Use it when you have a continuous, skewed outcome variable with more than 2 independent exposure groups.

Question 24

What is the Spearmans correlation coefficient?

Answer 24

For comparing two continuous variables where at least one of them is not following a normal distribution.

Question 25

What is the general method of assessing association between two categorical variables?

Answer 25

Chi-squared test

Question 26

What are we really looking at in a hypothesis test to look at association between two categorical variables?

Answer 26

We are essentially looking at how likely it is that we would get the difference that we have observed (in the odds ratio for example) if the truth was that there was no association between our two variables.

Each percentage in our results table is subject to sampling error. We need to assess whether the differences between them could be due to chance. We conduct a chi-squared test to get a p-value and this p-value tells us how likely the value is to have occurred by chance if there is truly no association.

Question 27

Describe the stages of the chi-squared test.

Answer 27

1. State the null hypothesis - no association between the two variables.
2. Calculate the test statistic - how close are the observed values in the table to the values expected were there no true association?

Expected number = row total x column total / overall total

The chi-squared test will compare the expected numbers under the null hypothesis to the numbers we actually got to see if there is a significant difference.

For each cell we then subtract the expected (E) from the observed (O), then square and divide by E:

(O-E)^2 / E

Then sum all cells to give the chi-squared statistic. The larger the value of the chi-squared statistic, the less consistent the data are with the null hypothesis.

3. Obtain a p-value - refer value of chi-squared to table of chi-squared distribution.

Degrees of freedom = (number of rows - 1) x (number of columns - 1)

4. Interpret the p-value

Question 28

What is the general formula for calculating the expected numbers under the null hypothesis for a categorical variable results table?

How do we calculate the chi-square test statistic?

Answer 28

Expected number = row total x column total / overall total

The chi-squared test will compare the expected numbers under the null hypothesis to the numbers we actually got to see if there is a significant difference.

For each cell we then subtract the expected (E) from the observed (O), then square and divide by E:

(O-E)^2 / E

Then sum all cells to give the chi-squared statistic. The larger the value of the chi-squared statistic, the less consistent the data are with the null hypothesis.

Question 29

What does a large value of the chi-squared statistic imply?

Answer 29

The larger the value of the chi-squared statistic, the less consistent the data are with the null hypothesis of no association.

Larger value of chi-squared = Smaller p-value

Question 30

What is mean by the p-value?

Answer 30

The p-value is the probability of getting a difference between groups as large or larger than that observed, were there really no true association. For example, P

Question 31

What assumptions associated with the chi-squared test?

Answer 31

1. Each subject contributes data to only one cell

2. The expected count in each cell should be at least 5

Question 32

The chi-squared test relies on the assumption that the expected count in each cell should be at least 5. What can we do if we have numbers smaller than this in our 2x2 table?

Answer 32

If you have small numbers in the 2x2 table we can use Yates continuity correction for any expected values less than 10. Alternatively, if you have very low numbers you can use a variant of the chi-squared test called Fishers exact test (any expected values

Question 33

Can we extend the chi-squared test to tables larger than 2x2?

Answer 33

Yes. We can use the chi-squared test for tables with more than 2 rows and columns - i.e. outcome and/or exposure has more than 2 categories.

SPSS won’t give you the odds ratio for more than a 2x2 table, but you can still use the chi-squared.

We can compute descriptive statistics such as odds ratios and 95% confidence intervals (assuming outcome binary) for each exposure group relative to baseline BUT doesn’t tell us about the overall association. You would then use the chi-squared to tell us about the overall association between those two values.

The p-value is used to assess the overall significance of association.

Question 34

For bigger tables when is chi-squared valid?

Answer 34

For large tables the exact test is computationally intensive.

It is ok to use the chi-squared test if no more than 20% of expected values are less than 5 and no expected values are less than 1. SPSS tell you about these conditions.

If the test is not valid you may have to combine rows or columns with small numbers (make sure it is sensible and biologically relevant).

Question 35

What is the chi-squared test for trent?

Answer 35

The chi-squared test for trend is a special test for when the exposure variable is ordered and the outcome is binary.

e.g. our exposure variable is age group and our out come is disease status.

The chi-squared test for trend will look at whether there is any evidence for our outcome changing with a change in our exposure groups.

Question 36

What does the chi-squared test assess?

Answer 36

The chi-squared test assesses whether the proportion with the outcome differs between exposure groups.

Question 37

What does the chi-squared test for trend assess?

Answer 37

The chi-squared test for trend is more sensitive than the standard chi-squared test and looks for an increasing or decreasing trend in the proportions across the exposure categories. As before, the null hypothesis is that there is no association. This test is displayed automatically as a ‘linear-by-linear’ test whenever you perform a chi-squared test.

Question 38

What measures tell us about the magnitude of an association?

Answer 38

The odds ratio or risk ratio. Hypothesis testing tells us about the significance of an association. We really need to be presenting both in order to describe both the magnitude of effect and the likelihood that you would see that effect.

Question 39

What kinds of chi-squared test would be appropriate if you have large numbers with no ordering?

Answer 39

Standard Pearson chi-squared test

Question 40

If you have and ordered variable with large number then what kind of chi-squared test would you use?

Answer 40

The chi-squared test for trend

Question 41

If you have small numbers in your table what options do you have for hypothesis testing?

Answer 41

If you have a 2x2 table and low numbers you can use chi-squared with continuity correction.

If you have very small numbers you can use Fishers exact test.

You may need to merge categories as long as it is sensible to do so.

Question 42

What is the first step for looking at the relationship between two continuous variables?

Answer 42

Plot them on a scatter diagram with the outcome on the y axis.

Question 43

What does correlation analysis do?

Answer 43

Essentially this just measure the closeness or degree of association between two continuous variables.

The correlation coefficients (r) lie between -1 and +1. +1 indicates perfect positive association and -1 indicates perfect negative association. 0 indicates no association.

Question 44

What is the most common correlation coefficient used for normally distributed data?

Answer 44

Pearson’s correlation coefficient. This requires both height and weight to be normally distributed.

Pearson’s correlation coefficient measures the degree of linear association.

Remember that the correlation coefficient is not assessing the steepness of the slope.

Question 45

When can we use Pearson’s correlation coefficient?

Answer 45

In order to use Pearson’s correlation coefficient:

• Both x and y variables have to be approximately normally distributed - plot histograms to check.
• Can also look if cloud of dots falls in oval shape.

Question 46

What is the null hypothesis for the correlation test?

Answer 46

The null hypothesis is that there is no association between the x and y continuous variables - we are essentially testing whether the correlation coefficient (r) is different to 0.

i.e. The null hypothesis is that the correlation coefficient is 0.

Question 47

What is the meaning of r^2 with regards to correlation?

Answer 47

r^2 (Pearsons r) = the proportion of the variance of outcome variable which is explained by exposure variable.

Essentially - how much of our outcome is due to our exposure?

Question 48

How do we examine correlation if we don’t have data that follow normal distributions?

Answer 48

If one or both variables are not normal:

• Try transforming and using pearson’s r
• If can’t be transformed, used Spearman’s rank correlation (or Kendall’s). This ranks values and examines how closely the ranks are correlated. The closer to +1 or -1 the greater the degree of association.

Question 49

What correlation statistics are designed for non-parametric data?

Answer 49

The non-parametric version of Pearson’s is called Spearman’s.

Spearman’s rank correlation (or Kendall’s). This ranks values and examines how closely the ranks are correlated. The closer to +1 or -1 the greater the degree of association.

Question 50

What is the non-parametric version of Pearson’s correlation coefficient?

Answer 50

Spearman’s

Question 51

When should we not use correlation?

Answer 51

1. When observations are not independent - e.g. multiple measurements on subjects
2. Data dredging
3. To assess trends over time
4. Subgroup analysis - unless a priori aim because expect different relation
5. To assess agreement or reliability
6. If one is a function of the other e.g. baseline BP vs change in BP

Question 52

Assuming no bias in study design, does correlation equal causation?

Answer 52

Correlation does not equate to causation. There are a number of possibilities:

1. X influences or causes Y
2. Y influences X
3. Both X and Y are influenced by one or more other variables (confounders)

Question 53

What might be the cause of getting a relationship between two variables that is highly statistically significant, but has a weak correlation coefficient?

Answer 53

This might be the case when we have a very large sample size. This is why you need to interpret the size, direction and statistical significance all together.

Question 54

What is the purpose of linear regression?

Answer 54

Linear regression is an extension of correlation analysis. Correlation assesses how closely two continuous variables are associated with each other. If we see that there is a relationship we can then use linear regression to describe that relationship a bit further in terms of how much our outcome variable (y) increases or decreases in terms of how much our exposure variable (x) increases. Linear regression is based on fitting a best fit line through the data.

Question 55

What is the equation of a straight line?

Answer 55

y = a + bx

a = the intercept where it crosses the y axis
b = slope of the line
y and x =the y and x variables

Question 56

How is the line of best fit fitted to the data?

Answer 56

The line of best fit is fitter to minimise the square of vertical distances between each observation and the line (method of least squares).

Question 57

What is another term for the regression coefficients?

Answer 57

The beta coefficients.

Question 58

What does a represent in the equation for a straight line?

Answer 58

a is the intercept. It is the value of y when x=0. It is not always biologically plausible but can be used for predictions.

Question 59

What does b represent in the equation for a straight line?

Answer 59

b is the slope. It tells us how much, on average, y increases/decreases for each unit increase in x. It is an estimate of the magnitude of effect.

Question 60

Describe the regression coefficient, b.

Answer 60

• b can be any value and depends on scale, i.e. units used (e.g. cm or m).
• If b is positive it means that outcome increases as exposure increases.
• If b is negative it means that outcome decreases as exposure increases.
• If b=0 the outcome and exposure are not related.
• Value of b is only an estimate of the true slope.

Question 61

How do we assess statistical significance in regression analysis?

Answer 61

The value of b is only an estimate of the true slope. We can construct a 95% confidence interval around b (computed by SPSS). We can also obtain a p-value for the null hypothesis of b=0 (given by SPSS).

Question 62

What assumptions must be fulfilled for linear regression analysis to be valid?

Answer 62

1. The relationship between y and x should be linear (look at scatter plot).
2. Values of y should be normally distributed around each value of x - this is different to both x and y having to be normally distributed themselves. You can still do linear regression even if one of your outcome or exposure variables is not drawn from a normal distribution. You check in SPSS that for every value of x the scatter of the values of y is drawn from a normal distribution.
3. Variability of y should be similar for each value of x - is scatter similar along the line?

Question 63

Having obtained estimates of a and b we can use linear regression to make predictions. For what range of x will these predictions be valid for?

Answer 63

Having got estimates of a and b we can use the equation to make predictions. These predictions are only valid for the range of x for which we have data available.

Question 64

Describe how we go about interpreting the SPSS output for linear regression.

Answer 64

1. Look at the Model Summary box - this tells us how good the prediction is likely to be. The important thing to look at is the value of R Square. The adjusted R Square becomes important when you start taking into account multiple variables. This value of R Square tells us how much of our variation in output variable is explained by exposure variable.
2. We then get our coefficients from the Coefficients table.

a can be found in the first row of the Unstandardised coefficients column under (Constant).

b can be found in the row under a.

3. Our p-value for the slope is testing whether the slope of the line is any different to 0.

CI will be shown on the right hand side of the Coefficients table.

Question 65

What is the purpose of multiple linear regression?

Answer 65

Multiple linear regression can be used to extend the equation y=a+bx to include 2 or more exposure variables in cases where we want to take into account more exposure factors that are having an effect on our outcome variable.

Question 66

What is the equation for multiple linear regression?

Answer 66

y = a + b1x1 + b2x2 ……..

For example if we want to look at blood pressure taking into account both weight and age:

Bp = a + (b1 x weight) + (b2 x age)

This would give us the age-adjusted regression coefficient for the effect of weight on blood pressure.

The output when taking account multiple variables may give us a higher R Square value - meaning that we have explained more of our variation in output variable with our exposures variables and improved the predictability of our model.

Question 67

Summarise correlation and regression.

Answer 67

• Methods are mathematically related but serve different purposes
• Need to choose a method appropriate to the study aim
• Correlation is used if we want to know if 2 variables are related and how closely
• Linear regression can be used if we want to describe or model the relationship between 2 variables, or make predictions

Question 68

What should we always do before carrying out any formal correlation analysis to look for outliers etc?

Answer 68

A scatter plot.

Question 69

What is one way to compare observed categorical data to expected categorical data?

Answer 69

By using a statistical test such as chi-squared

Question 70

What does a chi-squared test do?

Answer 70

A chi-squared test compares be served categorical data to expected categorical data.

Question 71

When can we use the standard Pearson chi-squared statistic?

Answer 71

When none of the expected counts are less than 10