Linear regression (week 3-5)

Question 1

Linear regression formula

Answer 1

Y = alpha + BetaiXi +E

Question 2

What is Y in Linear Regression?
What is alpha in Linear Regression?
What is Beta in Linear Regression?
What is X in Linear Regression?
What is E in Linear Regression?

Answer 2

Y is dependent var
Alpha is intercept parameter
Beta is regression coefficient
X is explanatory variable
E is i.i.d error term (use N(o, var))

Question 3

Estimated Linear Regression

Answer 3

same, but put hat on all the coeff and E is Ɛ

Question 4

E vs Ɛ?

Answer 4

E is i.i.d error (capture uncertainty) and Ɛ is residual term (diff data from model, is better it act more like E, contains uncertainty and what not captured)

Question 5

What if its not linear?

Answer 5

Use log

Question 6

How to minimize Ɛ

Answer 6

use minimize SSR (Sum Squared of Error)
1. Sum symbol (Y - Yhat)
2. derive! alpha and beta
3. Set the derivation to 0

Question 7

what is δ^2?

Answer 7

it represents (Σ(y-yhat)^2)/n-2

Question 8

why have n-2 in the δ^2?

Answer 8

it represent unbiased estimator

Question 9

what if is too large?

Answer 9

use the formula alpha with squingy line on top and beta with squingy line and test the hypothesis for both alpha with squingy line on top and beta with squingy line

Question 10

Goodness-of-fit measured using

Answer 10

R^2 = regression SS / Total SS
between 0% to 100%
calc test stats
use F1,n-2

Question 11

R^2 means?

Answer 11

proportion of total data variability explained by model

Question 12

Total Sum Of Square means?

Answer 12

Deviations between data and sample mean (total variability)

Question 13

Regression SS mean?

Answer 13

Deviations between model estimate and sample mean (data variability explained by model)

Question 14

Residual SS mean?

Answer 14

Deviations between data and model estimate (data variability unexplained by model)

Question 15

what does Y* means

Answer 15

its using the new Y, or Y in the future trs dibagi 100

Question 16

prediction interval of Y*

Answer 16

(alpha + (beta times x) ± tn-2 times sqrt(δ^2) times sqrt (1 + 1/n+ (x- mean of old x)^2 / ∑x^2 - n times sum of old x^2)

Question 17

residual is .. - …
and good if

Answer 17

dependent var - fitted value
good if randomly scattered and normally distributed s

Question 18

model fitting

Answer 18

estimate alpha and beta and the δ^2

Question 19

Predict with 95% interval!

Answer 19

1. model fitting
2. goodness of fit
3. plot residual (lebih scattered lebih bagus)
4. prediction interval

Question 20

Multicollinearity is

Answer 20

When 2 or more explanatory variable highly correlated, so it become vague, imprecise, and unreliable parameter estimates.

Question 21

Adjusted R vs R^2

Answer 21

As explanatory var increase, R^2 also increase. However, overly complex model is also not good so we introduce adjusted R

Question 22

Adjusted R formula

Answer 22

(1-(n-1)(1-R^2))/(n-k-1)

Question 23

Forward selection

Answer 23

-start with single explanatory variable and see the adjusted R
-If the R is higher, its good then add the variable into the model

Question 24

Backward selection

Answer 24

-Start with all explanatory variable
-Remove one by one, see the adjusted R

Question 25

Interaction term

Answer 25

add x1:x2 if relationship between Y and x1 depends on x2

Question 26

Categorical variable is

Answer 26

-discrete categories/levels/classes
-qualitative by nature
-tackled by set dummy variable
-can include continuous and categorical var in linear regression
-can include interaction term

Question 27

Nominal vs Ordinal

Answer 27

Unordered, ordered

Question 28

How many dummy var needed?

Answer 28

n-1, because we put one of the explanatory var as the base

Question 29

What can we do to improve the adjusted R?

Answer 29

-Check the stars (significant or not)
-We can combine variable
-Do not forget to check any multicollinearity

Question 30

From where we can now to combine this var with others?

Answer 30

Try use the similar estimate (neighbours)

Question 31

If we test the cor and found it multicollinearity detected, we have to test ..?

Answer 31

Test if it is significant:
assume corr(x1,x2) = rho
t-test = rho * sqrt((n-2)/(1-rho^2))
use qt(0.975, n-2)