Multiple linear regression

Question 1

disadvantage of doing regressions separately?

Answer 1

ignores potential synergy effect–>lead to misleading results

Question 2

RSE?

Answer 2

sqrt(1/(n-p-1) * RSS). n-(p+1) denominator is the degrees of freedom

Question 3

why does R squared increase when non-zero inputs are added?

Answer 3

RSS will decrease when choosing another (non-zero) parameter to estimate logic for finding a new set of coefficient based on minimising RSS

Question 4

what does adjusted R squared do?

Answer 4

adds a penalisation factor to account for the number of predictors included in the model

Question 5

formula of adjusted R square?

Answer 5

1-(n-1)/(n-1-p)*RSS/TSS. ALWAYS SMALLER THAN R SQAURE(can be neg)

Question 6

what is the null hypothesis H0?

Answer 6

all regression coefficients are 0 simultaneously

Question 7

formula of F stats?

Answer 7

((TSS-RSS)/P)/(RSS/(n-1-p))

Question 8

when no relation what is F stat?

Answer 8

1

Question 9

when to reject null hypothesis?

Answer 9

p-value<0.05

Question 10

how does forward selection work?

Answer 10

1. start with null model with intercept but no predictor
2. successively include most informative variable (lowest RSS, highest R square)
3. stop when stopping rule is reached (all variables have p-value<0.05)

Question 11

how does backward elimination work?

Answer 11

1. start with full model with intercept and all predictors
2. successively remove least informative variable (highest RSS, lowest R squared)
3. stop when stopping rule is reached (all variables have p-value<0.05)

Question 12

how does cross validation work?

Answer 12

1. split dataset into training and testing set
2. train model using training set
3. validate fitted model using testing set

Question 13

how is validation error rate assessed?

Answer 13

mean squared error (1/n*RSS)

Question 14

process of leave one out CV?

Answer 14

1,fit training data (obs=n-1) into a model

2. validate the model using testing set (obs=1)
3. compute to test MSE for first round
4. repeat 1-3 for n times to obtain n MSEs
5. construct LOOCV estimate as avg for MSEs

Question 15

K-fold CV?

Answer 15

1. randomly split observations into k groups
2. fit training data (obs=n-n1) into a model
3. validate the model using testing set (obs=n1)
4. compute to test MSE for first round
5. repeat 2-4 for k times to obtain k MSEs
6. construct K-FOLD CV estimate as avg of k MSEs

Question 16

lm for multiple linear regression?

Answer 16

lm_fit=lm(y~var1+var2,data=Boston)

summary(lm_fit)

Question 17

get model with all variables?

Answer 17

lm_fit1=lm(y~. , data=Boston)

Question 18

remove one or two variables?

Answer 18

lm_fit2=lm(y~. -var1 , data=Boston)

lm_fit3=lm(y~.-var1 -var2, data=Boston)

Question 19

get null set?

Answer 19

lm_fit4=lm(y~1,data=Boston)

Question 20

get correlation for all inputs?

Answer 20

round(cor(Boston),2)

Question 21

how to visualise the pair-wise correlation matrix?

Answer 21

install.packages(‘corrplot’)
library(corrplot)
cor_matrix=round(cor(Boston), 2)
corrplot(cor_matrix, type = “upper”, order = ‘alphabet’,
tl.col = “black”, tl.srt = 45, tl.cex = 0.9,
method = “circle”)

Question 22

for variables with high correlation, how to remove additive assumption?

Answer 22

lm_non_add=lm(y~var1*var2, data=Boston)

Question 23

scatterplot with linear assumption?

Answer 23

attach(Boston)

plot(var1,y,pch=16,col=’gray50’)

Question 24

for polynomial?

Answer 24

lm_nonlinear=lm(y~poly(var1,2),data=Boston)

Question 25

add line to scatterplot of non linear?

Answer 25

lines(sort(y),fitted(lm_nonlinear)[order(y)],lwd=2,col=’deeppink3’)

Question 26

find just the coefficient of variable n intercept?

Answer 26

glm_fit=glm(y~x,data=Boston)

coef(glm_fit) #same as lm_fit

Question 27

find LOOCV estimates?

Answer 27

cv_err=cv.glm(Boston,glm_fit)

cv_err$delta[1]

Question 28

finding CV of different models and lowest MSE?

Answer 28

glm_fit2=glm(y~poly(x,2), data=Boston)
cv_err=cv.glm(Boston,glm_fit2)
OR
USE FOR LOOP
cv_error=rep(0,10)
for (i in 1:10){glm_fit=glm(y~poly(x,i), data=Boston)
cv_error[I]=cv.glm(Boston,glm_fit)$delta[1]}

Question 29

plotting CV errors?

Answer 29

plot(cv_error,xlab=’polynomial’,main=’test MSE’, ylab=’‘,type=’b’, pch=16)

Question 30

CV for K fold?

Answer 30

cv_error=rep(0,10)
for (i in 1:10){glm_fit=glm(y~poly(x,i), data=Boston)
cv_error[I]=cv.glm(Boston,glm_fit,K=10)$delta[1]