Regression analyses

Question 1

Regression methods

Answer 1

Study the functional relationships between the response variable/outcome and the explanatory variables.

Variables & outcome linked by some mechanism (e.g., theory, causal pathway, etc.), otherwise meaningless.

Question 2

Simple regression model

Answer 2

y = β0 + β1x1 + ε

β0 = model intercept (mean of y when x=0)
β1 = slope relating variable x to y
x = predictor
ε = residual term

Question 3

Assumptions of OLS

Answer 3

1) Linear relationship between predictor & outcome
2) Error terms have constant variance
3) Error terms are independent of one another
4) Error term normally distributed

Question 4

Linearity

Answer 4

Scatterplot

Lowess (Locally weighted scatterplot smoother) - fits smooth line to data & if outcome linear, close to straight line

Question 5

Constant error variance

Answer 5

Plot squared residuals vs predicted variable; if see trend, non-constant error variance or heteroskedasticity
Bruesch-Pagan statistical test

Question 6

Normality of error term

Answer 6

Q-Q plot
Shapiro-Wilk test
If errors not normally distributed, consider transformation of response variable

Question 7

Multiple regression analysis

Answer 7

y = β0 + β1x1 + β2x2 + … + βkxk + ε

Same assumptions about error terms.
Linearity (unless specify nonlinear functional form)
Predictors in model as additive terms

Question 8

Standardized regression coefficients

Answer 8

1) Standardize data prior to analysis
2) Apply standardization factor after estimation of coefficients

Methods:

1) z-scoring variables: (xi - xbar) / sx
2) Multiply estimated coefficient by ratio of standard deviations of Y & X: b1 = (sy / sx1) * beta1

Question 9

Multicollinearity

Answer 9

When predictors correlated with each other (generally correlations >.4).
Variance inflation factor (generally, >=10)

Question 10

Outliers

Answer 10

DFFITS: Measure of influence of each observation on its predicted value
DFBETAS: Measures influence of observation on value of regression coefficient
Cook’s D(istance): Measures influence of each observation on all fitted values

Question 11

Interactions between predictors

Answer 11

Effect of one predictor being dependent on the value of another predictor.

If significant interaction, can keep model as specified or run stratified analyses.