week 5 DSE

Question 1

What is the disadvantage of running simple linear regressions separately instead of multiple?

Answer 1

ignores potential confounding factors or
synergy effect, leading to misleading results

Question 2

If there are 2 or more independent vairbales, how to find error?

Answer 2

use least squares to find the regression plane that best fits the data.

Question 3

If there are 2 predictors, how many parameters are we supposed to estimate

Answer 3

3

p+1

Question 4

What is the eror term of the ith point?

Answer 4

ϵi = yi − yˆi

Question 5

What does b2 =188 mean?

Answer 5

Holding the expenditure on x1 constant, every increase in x2 by 1 unit increases sales by 188 units

Question 6

How many degrees of freedom does RSE have for multiple linear model

Answer 6

n-(p+1)

Question 7

What is R^2?

Answer 7

fraction of variance explained

Question 8

What is a flaw of R^2 in multilinear regression?

Answer 8

value never decreases, even if we add redundant variables to the regression model.

Question 9

What causes the flaw in R^2 in multilinear regression?

Answer 9

equation solves for the coefficients such that RSS are minimized

If the variable does not improves model fit, the estimated coefficient will be zero. BUT R2 remains unchanged.

If the variable improves model fit, the estimated coefficient will be nonzero ⇒ R2 increases.

r^2 cannot decrease, can only remain same or increase

Use adjusted R^2

Question 10

Can adjusted R^2 be negative?

Answer 10

Yes

Question 11

What is the formula for penalization factor?

It is always ______

Answer 11

(n-1)/ ( n-(p+1) )

It is always larger than 1

Question 12

How does R^2 adjusted compare to R^2?

Answer 12

• always smaller, can be negative
• when new independent variable added, r^2 decreases

INCLUDES PENALIZATION FACTOR

Question 13

What is H0 and H1 for multiple lienar regression?

Answer 13

H0 : β1 = β2 = … = βp = 0
H1 : at least one βj is not zero.

Question 14

What test for multi regerssion hypothesis testing?

Answer 14

F-statistic

refer to P-VALUE of f stats from R output table

Question 15

What is interpolation?

Answer 15

Predicting Y for a value of X that is within the range of the original data

Question 16

What is extrapolation

Answer 16

Predicting Y for a value of X that is outside the range

Question 17

How does overfitting arise?

Answer 17

Using too many variables or too complex of a model can often lead to overfitting

Question 18

How do you know when you are overfitting?

Answer 18

Wont be able to predict any other new observatio
Only able ot fit the current data perfectly

Question 19

What is the problem of collinearity in multi linear regres?

Answer 19

We can only hold factors constant figuratively.

However, when two or more Xs are highly correlated, we can’t even hold factors constant figuratively

Question 20

What are 2 solution to collinearity

Answer 20

Remove the redundant variable. You will need subject knowledge to understand which one is redundant.

Combine the collinear variables into a single variable

Question 21

How to write operations within linear regression model equations in R?

Answer 21

use I

lm_mpg2 = lm(mpg ~ horsepower + I(horsepowerˆ2), data = Auto)

Question 22

If x1= a and x2= a^2, is it still a linear model? Why?

Answer 22

Yes. Linear in coefficients

Question 23

How to exclude variables in R?

Answer 23

use .-

eg lm(mpg ~ . - name, ..)
.- means everything EXCEPT FOR NAME

Question 24

If you have a categorical variable, how many of coefficients for them will you see in R?

Answer 24

n-1

If you figure out 2, you can figure out hte last 1. ( If first 2 are 0, last one must be 1)

Question 25

How to add labels to nominal categorical data?

Answer 25

Auto$origin = factor(Auto$origin,
labels = c(“American”, “European”, “Japanese”))

Question 26

How to test whether there is a relationship between Y and any of the Xs?

Answer 26

look at the F-statistics and its p-value.

Reject h0 if p value small

Question 27

What are some modelling guidelines?

Answer 27

Start from simple to complex model

Make sure you understand what the parameters mean, especially after using transformations like log(Y ).

After fixing the initial model, add other variables one at a time, or by logical groups like demographics

Watch out for signs of collinearity (high or perfect correlation, large changes in estimated coefficients AFTER ADDING VARIABLE!!!, reversed signs, etc

s long as you know that conceptually a variable should be in the model, then the variable should be in the model

Run model diagnostics(make sure assumptions met, may need to try polynomial)

Check for interactions

Sometimes simplicity of presentation is preferred to a better fitting model

Question 28

Should you include a variable in a model if it is not statistically significant?

Answer 28

Yes, as long as you know that conceptually a variable should be in the model, then the variable should be in the model

Question 29

If differnece is major, is simplicity preferred to better fitting

Answer 29

NO!!