Lecture 4

Question 1

Methods that will help improve the performance of our model, these methods/approaches are divided into three main types

Answer 1

1- Subset Selection ( Select the attributes that we want to keep in the model
2- Second type is shrinkage, reduce the the predicotrs in such a way to reduce variance
3- Reducing the dimension of our data (some kind of transformation)

Question 2

Subset selection

Answer 2

main goal reduce RSS

WE will always have the competing aspects between bias and variance

Question 3

Validation set approach

Answer 3

Randomly dividing the set into training and test

Question 4

LOOCV: Leave one out cut cross validation

Answer 4

Leaves single observation for validation and the remaining for the training set

Question 5

LOOCV has less bias becaause:

Answer 5

1- It is fit with n-1 observations (training set) in comparison to validation set which has approximately half of the observations as training set
2- Validation set approach can yield different results due to randomness in splitting
3- LOOCV doesnt over estimate
4- However i may become computationally expensive as n increases

Question 6

K fold validation:

instead of having one element for validation we use a fold for validation (multiple elements)

Answer 6

TRUE

Question 7

LOOCV is special case for K fold where k=n

Answer 7

TRUE

Question 8

Bootstrap

Answer 8

REpresent a way to resmaple your data over and over however with replacement

Question 9

IN bootstrap it is like we are treating the sample as population and sampling over and over

Answer 9

TRUE

Question 10

Comments in linear model

Answer 10

1- WE are assuming that the relationship between response and predictor is linear

Question 11

Many of the variables used in a multiple regression model may not be associated with the response; this will add unnecessary complexity

Answer 11

True

Question 12

Our target: We want minimum predicotrs to lead maximum explanations of the respone

Answer 12

TRUE

Question 13

I want to do feature selection in order to have maximum interpretability of the model

Answer 13

TRUE, thus we do subset selection

Question 14

Subset selection

Answer 14

Keep only subset of the variables in the model

Question 15

Methods that allow us to improve the performance in our model:
There are 3 main types:

Answer 15

1- Subset Selection
2- Shrinkage
3- Dimension Reduction

Question 16

Subset Selection: Main goal is to reduce RSS

Answer 16

True