L12: Linear regression

Question 1

How do we estimate the effect of independent variable on dependent variable?

Answer 1

Using regression analysis.

If dependent variable (y) is a continuous variable –> linear regression
y is ordinal –> ordinal regression
y is nominal (dichotomas) –> logistic regression

Question 2

What is simple vs multiple regression?

Answer 2

Simple –> only 1 independent x variable

Multiple/multivariable –> >1 independent x variable

Question 3

What is the difference between correlation and simple linear regression?

Answer 3

Correlation:

• quantifies the degree to which two variables are related, provided that the relationship is linear
• makes no distinction between the two variables (treated symmetrically)

Simple linear regression:

• determines the best fitting straight line to investigate the change in dependent variable y (continuous) that corresponds to a given change in independent variable x (continuous, ordinal or nominal), provided that there is significant correlation.
• two variables are assymmetrical

Question 4

Can we extrapolate beyond the observed range of values?

Answer 4

No, do not extrapolate the regression line beyond the observed range.

Question 5

What is the general equation for simple linear regression model?

Answer 5

y = alpha + beta (x)

alpha = y intercept = mean value of y when x = 0 
beta = slope = change in the MEAN value of y when there is a one-unit change in x

Question 6

What does the simple linear regression model use?

Answer 6

Method of least squares (smallest residual sum of squares)

Question 7

How do we evaluate the goodness of fit of the simple linear regression model?

Answer 7

• using the coefficient of determination (R2)
• in simple linear regression, R2 = r2 (r is the pearson product moment correlation coefficient)
• R2: proportion of variability among the observed values of y that is explained by the linear regression model
• range of R2 from 0 to 1 (bc r is from -1 to 1)
• if R2 = 1 –> all data pts lie exactly on the best fitting line
• if R2 = 0, there is no linear relationship between x and y

Question 8

How to check for statistical significance of linear regression model?

Answer 8

Look at p-value for the beta

Question 9

What is multiple linear regression?

Answer 9

• when there is >1 independent variable –> have multiple betas

Question 10

What does multiple linear regression use?

Answer 10

Method of least squares

Question 11

When are dummy variables used?

Answer 11

Used when we have nominal independent variables

Question 12

How many dummy variables do we need for a nominal variable with k categories?

Answer 12

k-1

Question 13

How do we evaluate the goodness of fit of the multiple linear regression model?

Answer 13

• inspect the coefficient of determination (R2)
• R2 is the proportion of variability among the observed values of y that is explained by the linear regression model containing the set of independent variables
• range of values is 0 to 1

Question 14

How do we compare between models that contain different numbers of independent variables?

Answer 14

Compare the adjusted R2
(adjusted R2 increases when inclusion of independent variable improves the ability to predict y, and decreases when it does not)

-HOWEVER, adjusted R2 cannot be directly interpreted as proportion of variability among observed values of y explained by the linear regression model.

Question 15

What does beta represent in a multiple linear regression model?

Answer 15

e.g.
For every 1 unit increase in x, the MEAN y will increase/decrease by ___, after controlling for other x variables (keeping them constant).

Question 16

What can we conclude from the p values from multiple linear regression model?

Answer 16

If statistically significant:

x variables are independently associated with y.