Regression

Question 1

Requirements for Linear Regression

Answer 1

1. SRS
2. Pairs of (x, y) data have a bivariate normal distribution (For each value X, the corresponding Y values have a normal dist; can be confirmed by examination of a scatterplot and double-checking of outliers
3. Homoscedasticity of residuals (equal variance)

Question 2

LR is a good model if:

Answer 2

1. regression line of scatterplot appears to fit the points well
2. r indicates a linear correlation
3. High: R-squared/adj R-squared/F-stat
4. Low: Std Error/t-statistic/AIC/BIC/MAPE/MSE
   * If not a good model, the best predicted value of y is the mean

Question 3

Goal of linear regression

Answer 3

Find line that minimizes the sum of squares of residual values

Question 4

Why use a residual plot?

Answer 4

Scatter plot with residuals as y values; Used to assess correlation and regression results; Randomness in the distribution of the plot is what we want; any patterns or changing of “thickness” of distribution suggests an underlying, non-linear pattern

Question 5

Regression process

Answer 5

1. construct a histogram to initially gauge normality
2. construct a scatterplot + quantile plot and verify that there is a linear pattern
3. construct a residual plot and verify that there is no pattern

Question 6

Prediction interval

Answer 6

Confidence interval for variables (instead of population parameters)

Question 7

Total deviation of (x, y)

Answer 7

vertical distance y minus y-bar, which measures the distance between the the point (y) and the sample mean (y-bar)

Question 8

Explained deviation of (x, y)

Answer 8

vertical distance y-hat minus y-bar, which measures the distance from the predicted value and the sample mean

Question 9

Unexplained deviation of (x, y)

Answer 9

vertical distance of y minus y-hat, which is the vertical distance between the point (x, y) and the regression line

Question 10

coefficient of determination (r-squared)

Answer 10

proportion of the variation in the response variable that has been explained by the model; R2= 1 - explained variation / total variation

Question 11

correlation coefficient (r)

Answer 11

explains strength and direction of correlation

Question 12

adjusted r-squared

Answer 12

as you add more X variables to your model, the R-squared value will always be greater since new variables can only add to total amount of explained variation; adjusted R squares penalizes

Question 13

Standard Error

Answer 13

Absolute measure of the average distance that points fall from regression line; measure of goodness of fit; = Sqrt(MSE) = Sqrt [SSE/(n - q)] *q = # of coefficients in model

Question 14

F-statistic

Answer 14

measure of goodness of fit; MSR = sigma (pred-mean)/ (q - 1)

Question 15

AIC

Answer 15

Akaike’s Information Criterion; measures goodness of fit of an estimated statistical model and can be used for model selection; lower is better

Question 16

BIC

Answer 16

Bayesian Information criterion; measures goodness of fit of an estimated statistical model and can be used for model selection; lower is better

Question 17

MAPE

Answer 17

Mean absolute percentage error (lower the better)