Lecture 14 - Feature Selection

Question 1

What is the curse of dimensionality?

Answer 1

The paradox of the more features you use in your model, the more data you need to help sort though the various permutations.

Question 2

What does feature selection do?

Answer 2

Helps us quickly identify features that don’t offer predictive power which allows us to iterate faster and with less data so we can experiment more.

Question 3

Which method of feature selection analyzes the variance of each feature and removes any features that have a low variance.

Answer 3

Variance Threshold

Bike seat example

Question 4

SelectKBest does what?

Answer 4

Removes all but the k highest scoring features.

It is configurable using any function of (X, y) that returns a set of scores per feature.

Question 5

Chi2 is

Answer 5

A predefined scoring function that tests the likelihood that the relationship between each variable and the outcome is due to that feature

Question 6

What is RFE?

Answer 6

Recursive feature elimination

Uses an external estimator that assigns weights to features (coefficients of a linear model).

It selects features by recursively considering smaller and smaller sets of features.

Question 7

How does RFE work?

Answer 7

It trains an external estimator and then drops the least important features.

Question 8

How does SelectFromModel differ from RFE?

Answer 8

Select from model using a threshold on the coef or feature_importances_ values instead.