Feature Selection

Question 1

What is feature selection? Why do we need it?

Answer 1

Feature Selection is a method used to select the relevant features for the model to train on. We need feature selection to remove the irrelevant features which leads the model to under-perform.

Question 2

Is feature selection important for linear models?

Answer 2

Yes, It is. It can make model performance better through selecting the most importance features and remove irrelanvant features in order to make a prediction and it can also avoid overfitting, underfitting and bias-variance tradeoff.

Question 3

Which feature selection techniques do you know?

Answer 3

Here are some of the feature selections:

Principal Component Analysis
Neighborhood Component Analysis
ReliefF Algorithm

Question 4

Can we use L1 regularization for feature selection?

Answer 4

Yes, because the nature of L1 regularization will lead to sparse coefficients of features. Feature selection can be done by keeping only features with non-zero coefficients.

Question 5

Can we use L2 regularization for feature selection?

Answer 5

No, Because L2 regularization does not make the weights zero but only makes them very very small. L2 regularization can be used to solve multicollinearity since it stablizes the model.