HW 5

Question 1

It is good practice to perform variable selection based on the statistical significance of the regression coefficients

Answer 1

False

Question 2

The training risk is an unbiased estimator of the prediction risk

Answer 2

False

Question 3

When the number of predicting variables is large, both backward and forward step wise regressions will always select the same set of variables

Answer 3

False

Question 4

It is not required to standardize or rescale the predicting variables when performing regularized regression

Answer 4

False

Question 5

Complex models with many predictors are often extremely biased, but have low variance

Answer 5

False

Question 6

Variable selection is a simple and solved statistical problem since we can implement it using the R statistical software

Answer 6

False

Question 7

Backward stepwise regression is preferable over forward stepwise regression because it starts with larger models

Answer 7

False

Question 8

Stepwise regression is a greedy algorithm searching through all possible combinations of the predicting variables to find the model with the best score

Answer 8

False, not all possible combinations

Question 9

Akaike Information Criterion (AIC) is an estimate for the prediction risk

Answer 9

True

Question 10

Mallow’s CP statistic penalizes complexity for the model more than leave-one-out CV and BIC

Answer 10

False, BIC penalizes more than other approaches

Question 11

Ridge regression is a regularized approach that can be used for variable selection

Answer 11

False

Question 12

The lasso regression requires a numerical algorithm to minimize the penalized sum of least squares

Answer 12

True

Question 13

The L1 penalty measures the sparsity of a vector and forces regression coefficients to be zero

Answer 13

True

Question 14

Elastic net regression use both penalties of ridge regression and hence combines the benefits of both

Answer 14

True

Question 15

In regularized regression, the penalization is generally applied to all regression coeffs where p = number of predictors

Answer 15

False, the shrinkage penalty is not applied to the intercept

Question 16

If there are specific variables that are required to control the bias selection in the model, they should be forced into the model and not be part of the variable selection process

Answer 16

True

Question 17

The penalty constant lambda in penalized regression controls the trade-off between lack of fit and model complexity

Answer 17

true

Question 18

In ridge regression, when the penalty lambda is zero, the corresponding ridge regression estimates are the same as the ordinary least squares estimates

Answer 18

True

Question 19

Ridge regression can be used to deal with problems caused by high correlation among the predictors

Answer 19

True

Question 20

When selecting variables for explanatory purpose, one might consider including predicting variables which are correlated if it would help answer your research hypothesis

Answer 20

True