Correlation and Simple linear regression

Question 1

What does a correlation coefficient measure?

Answer 1

The linear association between 2 continuous variables

Question 2

What does a correlation coefficient have values between?

Answer 2

-1 and +1

Question 3

What does a correlation coefficient of -1 indicate?

Answer 3

Perfect negative association

Question 4

What does a correlation coefficient of +1 indicate?

Answer 4

Perfective positive association

Question 5

What type of statistic is a correlation?

Answer 5

Parametric

Question 6

What are parametric tests?

Answer 6

Those that make assumptions about the parameters of the population distribution from which the sample is drawn. This is often the assumption that the population data are normally distributed.

Question 7

Why do the two continuous variables need to be approximately
normally distributed?

Answer 7

Correlation is a parametric statistic

Question 8

What is a non-parametric measure of association between ranks and thereby does NOT require this assumption?

Answer 8

Spearman rank correlation coefficient is a non-parametric measure of association between ranks. So does NOT require this assumption

Question 9

Correlation assumes causation.

True or false

Answer 9

FALSE

Correlation does. not assume causation

Question 10

What does exact linearity between variable y and variable x mean?

Answer 10

That one variable is a linear function of the second

Question 11

What does the following equation mean?

Y= B0 + B1X

Answer 11

The intercept B0 is the value that y taken when x is zero.
- If the intercept is zero then y increases in proportion to x (i.e. double x then y doubles)

The slope B1 determines the change y when x changes by one unit

• It measures the gradient of the line.

Question 12

Why might the linear relationship only apply to the expected value of y?

Answer 12

Other things influence y other than x
- (The expected value of y is the average over several instances with the same value of x).

-Any single y measurement might differ from the line.

Question 13

What is the mathematical model of a linear relationship?

Answer 13

E[Y]=B0+B1X

Or
y= B0 + B1x+E

• where E = Y-E[Y] is the error (discrepancy between the expected and observed y)

Question 14

What does a linear model stipulate?

Answer 14

A linear relationship between variable x and the expectation of y.

Question 15

What are scatterplots useful for?

Answer 15

Understanding the bivariate distribution of two continuous variable.

Question 16

What do linear models allow us to make an assessment of?

Answer 16

Whether a linear relationship is a realistic assumption.

Question 17

So assuming that an approximate linear relationship is an appropriate model, how can we find the best line?

Answer 17

Ordinary Least Squares (OLS) – minimises the squared deviations between the data points and the fitted line.

Maximum Likelihood (ML) – chooses the line under which the observed data was most likely to have occurred
Needs to know the distribution under which the errors arise

OLS = ML if the errors have a normal distribution

Question 18

What does the ordinary least squares approach pick?

Answer 18

The line that minimizes the
deviations

Question 19

What does the maximum likelihood approach involve?

Answer 19

If the deviations (also know as residuals) {ei} are assumed to be normally distributed then this line is also the maximum likelihood estimate – the line that makes the observed data seem as likely an
occurrence as possible.

The hypothesized normal distribution for the residuals is assumed to have constant variance

Question 20

What are the advantages of a linear model?

Answer 20

Linear model can fit well even where we “know” it’s the wrong model. It is often a good approximation over a restricted section of a non-linear relationship.

It is also a parsimonious description(A parsimonious model is a model that accomplishes the desired level of explanation or prediction with as few predictor variables as possible)

It might represent a causal/explanatory relationship between height and weight but this should NOT be assumed (stretching children does not make them heavier).

Accurate prediction does NOT require a relationship to be causal/explanatory.
(We call y the response and x the explanatory variable – but this does mean that the relationship is necessarily causal).

Question 21

What is the Regression intercept?

Answer 21

Predicted value of y when x=0

Question 22

What is the confidence interval formula?

Answer 22

95% CI = [B - 1.96xSE(B), B + 1.96x SE(B)]

where SE(B) = standard error of B

Question 23

In addition to the “point” estimates of the regression coefficients what does a linear model also show us?

Answer 23

Estimates of how precisely we have estimated them – the standard error (Std. Error)

Question 24

In a regression model the multiple correlation measures what?

Answer 24

The (Pearson) correlation between the observed and predicted values of y.

Question 25

In a simple linear regression model (one x) the correlation between the response and explanatory variable is what?

Answer 25

The multiple correlation.

It is also the standardised regression coefficient (beta).

Question 26

The square of the multiple correlation, R2, measures what?

Answer 26

The amount of variance in y that can be explained by differences in x.

Thus R2 provides a measure of fit of the regression model.

Question 27

How can we test the exposure-outcome association?

Answer 27

We can test the null hypothesis that he slope B1=0 .

The alternative hypothesis is B1 ≠0 .

This null hypothesis amounts to testing whether the dependent variable (outcome) is associated with the explanatory variable (exposure).

The test statistic is t=B/s.e.(B) .

The value of the test statistic can be compared with a t-distribution with n-2 degrees of freedom to obtain a p-value. (I.e. for large samples values |t|>2 are significant at the 5% level).

Question 28

What assumptions are required for inference?

Answer 28

The following assumptions are made for inference:

The observations are independent.

The observations may be selected for values of x but are otherwise a random sample.

The residuals have
a normal distribution (stronger than required by OLS)
an expected value of zero for all values of x (linearity)
constant variance (homoscedasticity).

This is the equivalent of assuming that the within group variances are equal when doing a two-samples t-test.