7.2 Feature Selection

Question 1

What is the full wrapper method

Answer 1

In wrapper methods, the feature selection process is based on a specific machine learning algorithm that we are trying to fit on a given dataset.

It follows a greedy search approach by evaluating all the possible combinations of features against the evaluation criterion. The evaluation criterion is simply the performance measure which depends on the type of problem, for e.g. For regression evaluation criterion can be p-values, R-squared, Adjusted R-squared, similarly for classification the evaluation criterion can be accuracy, precision, recall, f1-score, etc. Finally, it selects the combination of features that gives the optimal results for the specified machine learning algorithm.

Question 2

What is forward selection?

Answer 2

In forward selection, we start with a null model and then start fitting the model with each individual feature one at a time and select the feature with the minimum p-value. Now fit a model with two features by trying combinations of the earlier selected feature with all other remaining features. Again select the feature with the minimum p-value. Now fit a model with three features by trying combinations of two previously selected features with other remaining features. Repeat this process until we have a set of selected features with a p-value of individual features less than the significance level.

Question 3

What is backward elimination?

Answer 3

In backward elimination, we start with the full model (including all the independent variables) and then remove the insignificant feature with the highest p-value(> significance level). This process repeats again and again until we have the final set of significant features.

Question 4

In order to have an accurate model, the features should be ………. as much as possible and the class label should be ……… to the features. Group of answer choices

Answer 4

• uncorrelated– correlated

When the features are uncorrelated and independent, they can provide more information. Ideally the class label should be highly correlated to the features, so that the features can be predictive.

Question 5

Using full Wrapper method, the best model with 4 features always contains the set of features involved in the best model with 3 features.

Answer 5

.

Question 6

Which one of the following choices is an ideal value for PMI between the feature and the class?

Answer 6

.

Question 7

Which of the following method does not belong to the feature selection approaches?

Answer 7

.