Regression Diagnostics & Assumptions

Question 1

When checking bias in regression models what are the 3 general things to check?

Answer 1

The question
Aspects to investigate in results
The general procedure

Question 2

What questions regarding the model is important to investigate in regression?

Answer 2

Is the model accurate for the sample and can the model be generalized?

Question 3

What are the important aspects to be investigated regarding results?

Answer 3

Outliers and distribution of residuals

Question 4

What general procedure is important to check regarding bias in regression?

Answer 4

Regression diagnostics and assumption assessment

Question 5

What is an outlier?

Answer 5

A case that differs substantially from the main trend of the data

Question 6

Why can an outlier constitute a problem?

Answer 6

An outlier can affect the precision of the estimation of the regression coefficient

Question 7

How can we detect outliers?

Answer 7

By searching for large residuals and also by searching for influential cases

Question 8

How can you tell if an outlier has a small or large residual?

Answer 8

By looking at how close the outlier is to the line of best fit. Close is a small residual and far away is a large residual.

Question 9

Large residual outliers can be found by looking at the standard normal distribution. What are the general rules?

Answer 9

Standardized residuals below -3 and +3 is a cause for concern because in a typical sample they are unlikely to occur.
Of more than 5% of values have a residual either below or above -2 or +2 we should be concerned because that exceeds what we would normally expect.

Question 10

What table can you use to look at outliers with large residuals?

Answer 10

Casewise diagnostics

Question 11

How can you detect influential cases?

Answer 11

By looking at Cooks’s distance in the residual statistics table.

Question 12

What is the maximum number for cook’s distance before we should be concerned?

Answer 12

1. Greater than 1 is a cause for concern.

Question 13

What happens if we find outliers?

Answer 13

You check if 
- the outliers is not due to entry error
if data entry is correct you can 
-transform data
- consider deleting the case

Question 14

What are the assumptions about residuals in regression?

Answer 14

Normality
Linearity
Homoscedasticity

Question 15

What is the assumption about normality in assumptions about residuals in regression?

Answer 15

The residuals should be normally distributed

Question 16

What are the assumption about linearity?

Answer 16

The residuals should have a straight line relationship with the predicted outcome scores

Question 17

What is homoscedasticity?

Answer 17

The variance of the residuals about the predicted outcome scores should be approximately equal for all predicted scores.

Question 18

How do you check for normality, linearity and homoscedasticity?

Answer 18

Look at the scatterplot. The scatterplot should be roughly rectangularly distributed, with most points centered around the center (point 0).

Question 19

What is an alternative way of checking for normality of residuals?

Answer 19

A histogram.

Question 20

What is independence of error?

Answer 20

For any two observations, the error of prediction (residuals)should be uncorrelated (it should be independent from one another).
That is, the error produced by two observations should not be due to the same reason.

Question 21

What do you check for independence of error?

Answer 21

Look at the model summary table and see the Durban Watson.

Question 22

What numbers should the Durbin Watson numbers lie within?

Answer 22

1 and 3 is good. If it’s closer to 0 or to 4 then it’s a cause for concern.

Question 23

What does the Durbin Watson test check?

Answer 23

If the adjacent residuals are correlated.
Numbers between 0-2 indicates positive correlation and between 2-4 indicates a negative correlation.
A number closer to 2 means lack of correlation.

Question 24

What is multicollinesrity?

Answer 24

In multiple regression, it occurs when the predictor variable in the models are highly correlated (>.80).
It means that the predictors have a lot of shared variance.

Question 25

What in multicollinearity a problem?

Answer 25

When you are interested in the separated effects of different predictors on an outcome but not when looking at the outcome from a set of predictors.

Question 26

How do you check for multicollinearity?

Answer 26

Inspecting correlations in the correlation matrix that the correlation is not great than .80.

Question 27

What is the solution to multicollinearity?

Answer 27

Delete one variable or more from the model (if correlation is greater than .90)
Combine the collinear variables into a composite variable (if the correlation between the 2 variables are >.80).