Multiple Regression

Question 1

Properties of fitted regression line

Answer 1

1. exogeneity assumption
2. no-correlation between residuals and fitted values
3. mean of residuals = 0
4. variance decomposition = explained variance + unexplained variance
   …

Question 1

Assumptions

Answer 1

1. Normality of residuals - histogram, plots of residuals, formal tests
2. Homoscedasticity - homogeneity of variance of residuals - scatterplots of residual variance, Levene’s test
3. Linearity - scatterplot
4. Non-correlated errors

Question 2

Cohen’s f values

Answer 2

0.02 - small
0.15 - medium
0.35 - large effect

Question 3

Measures of model fit

Answer 3

1. R2 - might overestimate the explained variance while we add other variables
2. adjusted R2 - can decrease
3. R2 change

Question 4

Types of Influential Observations

Answer 4

1. outliers
2. leverage points
3. influential observations - reinforcing, conflicting, shifting

Question 5

Multicollinearity - impact

Answer 5

1. Estimation - unique variance of each predictor is reduced. when they’re not correlated - total explained variance = sum of unique explained variances by each predicot
2. Explanation - problem in interpretation

sometimes small multicollinearity can be ignored

Question 6

Handling multicollinearity

Answer 6

1. delete
2. combine
3. estimation techniques
4. do nothing but don’t intepret in a regular way
5. increase sample size
6. dimensionality reduction

Question 7

Suppression

Answer 7

1. inflated R^2 - very high
2. high correlation between predictors, but only one of them is correlated with dependent

Question 8

Correlations

Answer 8

1. Zero order - unique + shared -> (b+c)/total
2. Partial - x&y but controlled for other variables -> b/total - (a+c)
3. semi partial - unique variance of x -> b/total -> R^2 will decrease by this amount if we remove this variable from the model

Question 9

R^2 will decrease by which correlation?

Answer 9

squared part/ semi-partial - unique contribution of the variable

Question 10

suppression & correlation

Answer 10

when partial is higher that zero-order, we have suppression

Question 11

how can including suppressor improve the model?

Answer 11

1. it can remove the irrelevant variance in the predictor - decrease the noise

Question 12

Interpretation with interaction

Answer 12

1. conditional main effects
   e.g. the income for females is less than males under the condition that years of education is zero
2. interaction term
   females earn 60 euros multiplied by years of education less compared to men
3. difference in slope = slope of … is different

Question 13

interaction and follow-up analysis

Answer 13

1. interaction with metric - simple slope
2. interaction between metric and non-metric - simple slope
3. interaction between non-metrics - simple effect