Linear regression and correlation

Question 1

Fitting a line to scatterplot?

Answer 1

Method of least squares; makes use of differences between line and points being positive and negative so must square them. Similar to standard deviation! Choose line that has smallest value for sum of squares.

Question 2

In regression, does σ of FEV1 (Y) depend on height (X axis)?

Answer 2

No: assumed that SD is constant for all heights (i.e. spread of FEV1 about the mean FEV1).

Question 3

In regression, is it true that the FEV1 of a given student depends on his height?

Answer 3

No - only that the mean of the population of FEV1 depends linearly on the other variable. Y axis given as mean; X is actual value. Consider the FEV1s for the population of students with a given height and assume that this mean varies linearly with height.

Question 4

Three parameters to estimate in regression?

Answer 4

σ (SD about the line), α (intercept) and β (slope). Just as for a single variable can estimate the population mean μ and σ, here μ depends on α + βx, which are population parameters and cannot be known. Instead, use sample, fit line where y = a+bx and use these sample estimates. Also need to estimate σ about the line using s. Intercept (a) is on the same scale as y so uses same units. B is y per x.

Question 5

b and β?

Answer 5

Just as use SE to see if m is a good estimate of μ, can see if b is a good estimate of β. Again, use SE, but more complicated.

Question 6

Hypothesis testing in regression?

Answer 6

Has a limited role; more about how good estimate of β is. The one hypothesis of interest is whether β = 0. This is because if β=0 then the mean of Y does not change with the x variable (i.e. no association).

Question 7

Regression in minitab?

Answer 7

Skip all but “the regression equation is”. Gives “FEV1” when should be mean FEV1. Also look at Predictor, Constant, Ht (x). Constant = intercept. Get SE of Ht listed under SE Coef, and get P value for Ht too. Next bit of output is S, the same as our s, i.e. estimate of SD (spread of FEV1 about the fitted line).

In summary, get estimated slope and intercept (under Coef), the SE of the slope, under SE Coef, the P value for β=0, and the SD about the line (s).

Question 8

How can intercept be negative?

Answer 8

This is because intercept is mean FEV1 when height = 0 which would obviously not make sense. For this reason intercept is only important for plotting the slope.

Question 9

What does a P value of 0.005 mean for the hypothesis β=o?

Answer 9

Means that there is very strong evidence that FEV1 mean does depend on the height. Does not mean that FEV1 can be determined once height is known, just that mean FEV1 changes with height.

Question 10

“Naming” regression?

Answer 10

Regression of y on x (seeing how mean y varies with x).

Question 11

What does s account for in regression?

Answer 11

The variation left in FEV1 values AFTER height has been accounted for

Question 12

Clinical applications of regression?

Answer 12

Some important variables might be very difficult or invasive to measure so instead predicting it from other, related variables would be helpful. In practice, the natural variability often gives wide limits.

Question 13

Single value estimate of FEV1 for height of 180cm?

Answer 13

As FEV1 = α+hβ, but we must use a and b, mean FEV1 for his height is -9.19 + 180 * 0.07 = 4.20L.

Question 14

Intervals in regression?

Answer 14

Before, acknowledge uncertainty in m as an estimate of μ by computing interval where expect μ to lie. These intervals widen as sample from which the interval is calculated increases (as estimate of μ becomes more precise). The same in regression: estimate the mean using a+bh means that uncertainty in a and b will give estimate of h (4.20L) uncertainty. This is more complicated, but intervals still get smaller as sample size increases.

Question 15

Why are confidence intervals not appropriate for regression?

Answer 15

Because if estimating mean FEV1 for one person of given height, it does not follow that making the sample as large as possible will make this interval indefinitely smaller as variation will still remain. Therefore cannot have a formula that allows sample size to reduce interval to 0.

Question 16

What interval is used instead of confidence intervals?

Answer 16

Instead, use prediction intervals. Can also calculate confidence intervals (indicate that the mean FEV1 for students with height 180 cm is 3.8-4.6 with 95% confidence), but prediction interval indicates that 95% of students with height 180cm have FEV1 2.9-5.5. Confidence intervals will be curved because our estimate of mean FEV1 is better near the “centre”, and prediction intervals will be near-straight because they rely mainly on intrinsic variability, which is constant. The small amount of curve comes from incorporating the CIs.

Question 17

How do confidence intervals and prediction intervals change with sample size?

Answer 17

Confidence intervals will get much smaller; prediction intervals will change slightly.

Question 18

Pitfalls in using regression?

Answer 18

Often is merely a relationship between two variables using the sample itself; therefore unwise to use the line generated for other kinds of superficially similar data e.g. children, older males or female students. Also be wary out outliers as can alter estimates.

Question 19

Why cannot reverse regression?

Answer 19

The equation is MEAN FEV1 = a+bh and so cannot simply swap h and FEV1. The regressions of each on the other are different and it is important to choose which is appropriate.

Question 20

Assumptions in regression?

Answer 20

That the mean of the y variable at a given value of the x variable changes linearly with x, that the spread of the data about this line is constant and thus does not change as x changes, and that the DEVIATIONS from this line follow a normal distribution (important if calculating confidence or prediction intervals or hypothesis test).

Question 21

Assessing the linearity assumption in regression?

Answer 21

Draw scatterplot; see by eye if is plausible.

Question 22

Assessing the spread about the line in regression being constant?

Answer 22

Use residuals (vertical distance of a point from the fitted line). Positive if above the line, negative if below. If the fitted data truly reflects the structure of the data then the residuals are a sample from a distribution with population mean = 0 and they all have the same SD. Can see if have the same SD by plotting residuals against the height of the individual; should change little with height.

Question 23

Assessing the normality of deviations from the line?

Answer 23

As residuals = deviations from the line, this means checking that the residuals are from a common normal distribution. The best way of doing this is using a normal probability plot for the residuals.

Question 24

Are any assumptions made regarding the x variable in regression?

Answer 24

No! Can even be discrete.

Question 25

The correlation co-efficient?

Answer 25

We use the product-moment correlation or the Pearson correlation, given the symbol r (rho for population).

Question 26

Properties of r?

Answer 26

Always between -1 and 1, if the points were exactly on a straight line then r would either be -1 or 1; a value of 0 = no LINEAR relation but could be circular etc.; can be computed for data which comprises pairs of continuous variables.

Question 27

What does a negative/positive value of r mean?

Answer 27

Means that the y variable tends to decrease where the x variable increases. If positive then tend to increases or decrease together.

Question 28

Does the sign of r affect strength?

Answer 28

No - just the direction.

Question 29

Hypothesis testing for r?

Answer 29

Often done (r=0.35 (P=0.07)) but actually if just testing that the population correlation ρ=0 (i.e. that there is no linear relation between the variables) then this is exactly the same test as β=0 in a regression of y on x (P value will be the same). Also need one of the variables to be continuous for this; if both are continuous then can do confidence intervals for ρ but not very useful.

Question 30

Real problem with hypothesis testing in correlation?

Answer 30

Even if have very weak relationship (r=0.3, for example), then a large enough sample size may lead to a significant P value for ρ=0 testing. This will be evidence against the two variables being unrelated, but does not really tell you that they are closely related. In general, values of r betwen 0 and 0.6 or so are pretty hard to interpret. Regression usually more comprehensive.