Topic 3: Regression Diagnostics

Question 1

linear regression assumptions

Answer 1

• linearity
• normality
• homoscedasticity
• independence
• outliers
• multicollinearity

Question 2

linearity

Answer 2

the relationship between x and y is linear

Question 3

normality

Answer 3

the error term follows a normal distribution

Question 4

homoscedasticity

Answer 4

the error term has a mean 0 & a constant variance

Question 5

independence

Answer 5

the error terms are not related to each otehr

Question 6

outliers

Answer 6

there are no outliers

Question 7

multicollinearity

Answer 7

there are no high correlations among IVs

Question 8

testing normaltiy

Answer 8

skewness & kurtosis, shapiro-wilk test, normal quantile plot

Question 9

skewness

Answer 9

the spread of the data

Question 10

kurtosis

Answer 10

how peaked the data are

Question 11

interpreting skewness & kurtosis

Answer 11

if t skewness or t kurtosis > 3.2, violation of the respective assumption

Question 12

shapiro-wilk test

Answer 12

tests for normality

Question 13

null hypothesis of shapiro-wilk test

Answer 13

the sample comes from a normal distribution

Question 14

interpreting shapiro-wilk results

Answer 14

significant result = may not come from a normal distirbution

Question 15

normal quantile plot

Answer 15

sorts observations from smallest to largest, calculates z-scores of the sorted observations, and plots the observations against corresponding z-scores

Question 16

intepreting normal quantile plot

Answer 16

if close to normal, the points will lie close to some straight line

Question 17

dealing with non-normality

Answer 17

data transformation or resampling methods (ex., bootstrap, jackknife)

Question 18

bootstrap

Answer 18

uses resampling with replacements to emulate the process of obtaining new samples so that we can estimate the variability of a parameter estimate without generating additional samples

Question 19

what happens if homoscedasticity is violated?

Answer 19

• the variances of regression coefficient estimates tend to be under-estimated
• thus, t-ratios tend to be inflated

Question 20

testing homoscedasticity

Answer 20

residual plots

Question 21

residuals

Answer 21

differences between Yi & Ŷi

Question 22

interpreting residual plots for homoscedasticity

Answer 22

funnel shape = violation of homoscedasticity

Question 23

dealing with heteroscedasticity

Answer 23

data transformation, other estimation methods, other regression methods

Question 24

testing linearity

Answer 24

residual plots

Question 25

interpreting residual plots for linearity

Answer 25

curve shape = violation of linearity

Question 26

dealing with non-linearity

Answer 26

data transformation, add another IV to the equation (non-linear function of one of the other IVs), use non-linear methods

Question 27

testing independence

Answer 27

Durbin-Watson (d) of autocorrelation

Question 28

Durbin-Watson test

Answer 28

tests the correlation between error terms ordered in time or space

Question 29

interpreting Durbin-Watson test results

Answer 29

1.5-2.5 = normal
below 1 or above 3 = abnormal

Question 30

dealing with dependence

Answer 30

data transformation, use other estimate methods, use other regression methods

Question 31

outlier

Answer 31

a data point disconnected from the rest of the datas

Question 32

checking outliers

Answer 32

cook’s distance

Question 33

interpreting Cook’s distnace

Answer 33

Cook’s D > 4 suggests potentially serious outliers

Question 34

dealing with outliers

Answer 34

if an unusual case is not likely to reoccur, delete the case or use robust regression

Question 35

consequences of multicollinearity

Answer 35

1. unstable regression coefficient estimates (lower t-ratios)
2. a high r2 (or significant F) but few significant t-ratios
3. unexpected signs of regression coefficients
4. the matrix inversion problem

Question 36

checking multicollinearity

Answer 36

tolerance, VIF, condition index

Question 37

tolerance

Answer 37

R2 for the regression of each IV on the other IVs, ignoring the DV

Question 38

interpreting tolerance

Answer 38

values < 0.1 = multicollinearity problem

Question 39

variance inflation factor (VIF)

Answer 39

1/tolerance

Question 40

interpreting VIF

Answer 40

values > 10 = multicollinearity problem

Question 41

condition index

Answer 41

measure of dependency of one variable on the others

Question 42

interpreting condition index

Answer 42

values > 30 = multicollinearity

Question 43

dealing with multicollinearity

Answer 43

drop a variable, incorporate more info (composite variable), or use other regression methods

Question 44

r2 vs. R2 in linear regression

Answer 44

• Simple linear regression: r2 = R2
• Multiple linear regression: r2 ≠ R2