module 12

Question 1

example of overfitting

Answer 1

nonlinear model has been trained too well on the training dataset, the test dataset has worse performance

Question 2

Some Ways to overfit a linear regression model

Answer 2

Irrelevant explanatory variables
Collinear explanatory variables

Question 3

Some ways to underfit a linear regression model

Answer 3

Leaving out an important explanatory variable

Question 4

A parsimonious model

Answer 4

Aims to strike a balance between overfitting and underfitting the model to the training dataset
does this by: Having a low enough number of explanatory variables to avoid overfitting while
Having a high enough model fit to avoid underfitting

Question 5

adjusted R^2

Answer 5

used to measure model parsimoniousness
(n-1/n-p-1)

Question 6

Interpreting the adjusted R^2

Answer 6

The higher the adjusted R^2 of a model, the more parismonious we say that the model is, and therefore, the less likely the model is to be overfit to the training dataset

Question 7

number of every possible model

Answer 7

2^p possible models
p is possible explanatory variables

Question 8

Heuristic techniques

Answer 8

Backwards Elimination Algorithm
Forward Selection Algorithm

Question 9

Regularized linear regression model

Answer 9

take the objective function from our basic linear regression model and add a type of penalty term

Question 10

Penalty term

Answer 10

penalizes models that have too many explanatory variables that don’t bring enough predictive power to the model

Question 11

Goal of the penalty term

Answer 11

Goal 1: interpretation of what to leave out
Goal 2: leave out variables that lead to overfitting

Question 12

LASSO Regression L1

Answer 12

(stands for least absolute shrinkage and selection operator)
The sum of absolute values for all coefficients

Question 13

Clearer slope interpretation with LASSO regression

Answer 13

If slope is set to be 0, LASSO regression model is suggesting that this slope’s corresponding variable can be left out of model.

Question 14

Ridge Regression (L2 Penalty Term)

Answer 14

the square root of the sum of the squared values

Question 15

Less clear slope interpretation with ridge regression

Answer 15

The resulting slopes found w ridge regression provide much less of a clear indication as to which explanatory variables should be left out of the model

Question 16

Benefits of ridge regression

Answer 16

In the presence of multicollinearity, ridge regression slopes can be more trusted than those that would have been returned by an nonregularized linear regression model
the predicted impact of these variables is more likely to get evenly distributed in the slopes

Question 17

Elastic net regression (L1 and L2 Penalty Term)

Answer 17

Combines the strengths of Lasso and Ridge by balancing between feature selection and coefficient shrinkage.

Question 18

The impact of the alpha parameter

Answer 18

Alpha is set high, the L1 term becomes higher, the results slopes will tend to look more like LASSO regression

Alpha is set low, the L2 term becomes higher, the results slopes will tend to look more like ridge regression and more focus on balancing the weight in collinear slopes

Question 19

Cross validation techniques

Answer 19

Techniques involve creating multiple sets of training and test datasets from the full dataset
Leave one out cross validation
K fold cross validation

Question 20

Leave one out cross validation

Answer 20

Has every observation appear in a test dataset exactly once
Every observation will be in a training dataset n-1 times

Question 21

Benefits of LOOCV

Answer 21

accurate test data performance
the corresponding test dataset predictions that we make will similarly reflect this higher accuracy that the full model might have achieved
No randomness
Low model variability

Question 22

Drawbacks of LOOCV

Answer 22

Computationally expensive
More variable test data predictions
Inflation of model performance

Question 23

K-fold cross-validation

Answer 23

Every observation appear in a test dataset once
Every observation will appear in a training dataset k-1 times

Question 24

Benefits of k-fold cross-validation

Answer 24

Less computationally complex than LOOCV
More accurate test data performance
Less inflation of model performance than LOOCV

Question 25

Drawbacks of k-fold cross-validation

Answer 25

More computationally complex than train-test-split method
Less accurate test data performance than LOOCV
Randomness

Question 26

AIC

Answer 26

Akaike information criterion (AIC)
- 2 * LLF + 2k
K = number of slopes in the model
LLF = the optimal log likelihood function value of the model

Question 27

AIC Interpretation

Answer 27

the lower the AIC score of a model is, the more parismonious the model is considered to be

Question 28

BIC

Answer 28

Bayes information criterion (BIC)
-2 * LLF + ln(n) * k

Question 29

BIC Interpretation

Answer 29

the lower the BIC score of a model is, the more parsimonious the model is considered to be

Question 30

AIC vs BIC

Answer 30

The only difference is the penalty term in the equation 2 for AIC, ln(n) for BIC

Question 31

Downsides of BIC

Answer 31

encourages smaller number of slopes, may come at the expense of training dataset fit, causing you to select a model that has a worse training data

Question 32

Downsides of AIC

Answer 32

AIC doesn’t penalize a high number of slopes as much as BIC score does. AIC score can be unhelpful for the purpose of model selection.