Basics of Multiple Regression

Question 1

When should you use Logistic regression models?

Answer 1

If the dependent Y variable is discrete

If out independent X variables is qualitative

Question 2

When should you use Multiple regression models?

Answer 2

When the dependent variable is continuous (not discrete) and there is more than one explanatory variable (more than one dependent variable).

When multiple independent variables determine the outcome of a single dependent variable.

Dependent Y Variable is continuous
We have more than 1 Dependent Y variable

Question 3

Assumption of Regression models

Answer 3

L.I.I.N.H.

Linearity: Relationship between dependent Y variable and Independent X variable is linear.

Independent of Errors: Regression residuals are uncorrelated across observation.

Independent: Independent X variable is not random, there is no exact linear relationship between 2 or more independent variables.

Normality: Regression residuals are normally distributed.

Homoscedasticity: Constant variance of regression residuals

Question 4

How to determine if a variable is significant?

Answer 4

|T-Stat| > 1

Question 5

Degrees of freedom for SSR

Answer 5

N-k

Question 6

Degrees of freedom for SST

Answer 6

N-1

Question 7

Degrees of freedom for SSE

Answer 7

N-K+1

Question 8

What will happen to adjusted R-Square if we have insignificant varibles

Answer 8

Adjusted R-Square decreases

Question 9

R-Square formula

Answer 9

SSR/SST = Explained Variation / Unexplained variation

1-(unexplained variation/total variation)

Question 10

What kind of test is this?

H0: bi = Bi
Ha: bi /= Bi

Answer 10

Two tail test

Question 11

What kind of test is this?

H0: bi <= Bi
Ha: bi > Bi

Answer 11

Right tail test

<= - is heading right

Question 12

What kind of test is this?

H0: bi => Bi
Ha: bi < Bi

Answer 12

Left tail test

=> is heading left

Question 13

Formula and purpose of AIC

Answer 13

AIC = n * ln(SSE/n)+ 2(K+1)

AIC is better for forecasting purposes

Question 14

Formula and purpose of BIC

Answer 14

BIC = n * ln(SSE/n) + Ln(n)(k+1)

Better for evaluating goodness-of-fit

Question 15

How do we test joint coefficients?

Answer 15

F-Stat
[(SSE restricted - SSE unrestricted) / q] / (SSE unrestricted / N-k-1)

alternative formula…
(SSE restricted - SSE unrestricted) x ( N-K-1) / (SSE unrestricted x Q)

-SSE restricted: Model 1 does not include the two variables we want to test, so it is the restricted model.
SSE unrestricted: Model 2 that includes the two variables we want to test, so it is the unrestricted model.

Question 16

Adjusted R square formula

Answer 16

1 - [(n-1)/(n-k-1)] x (1-R Squared)

Question 17

Holding all other variables constant, the adjusted R-Square will decrease when all of the following variables increase expect…

Answer 17

The number of observation

Question 18

Formula For F-Test

Answer 18

MSR / MSE

[ RSS / K ] / [SSE / n-(k+1) ]

[ Regression / K ] / [ Residual / n-(k+1) ]

Question 19

Hypothesis test for F test

Answer 19

F test > F stat : Reject null. b1 = b2 = bn = 0

F test < F stat : Fail to reject null. b1 =/ b2 =/ bn =/ 0

Question 20

Are the coefficents correlated?

y=2+3x1

y=1.5+2x1+ 3x2

Answer 20

Yes, because when we added an aditional coefficient, their values changed

Question 21

The null hypothesis for F-test

Answer 21

All regression coefficients are equal to zero. In other words, none of the independent variables have a significant effect on the dependent variable.

H0 = β1=β2 =β3 =⋯=βk=0

This implies that the model has no explanatory power, and the variation in the dependent variable 𝑦.
y is not explained by the independent variables.

Question 22

The Alternative hypothesis for F-test

Answer 22

At least one regression coefficient is different from zero. In other words, at least one independent variable has a significant effect on the dependent variable.

H1:Atleastoneβi /=0 (forsomei=1,2,…,k)

Question 23

How Calculate the joint F-statistic

Answer 23

[(SSE of restricted model−SSE of unrestricted model)/𝑞] / SSE of unrestricted model/(𝑛−𝑘−1)

restricted model: The model that does not include the coefficient. Value is obtained from SS in Residual row.