Week 4 Flashcards
Dimensionality reduction
The transformation of data from a high-dimensional space into a low-dimensional space so that meanful properties are kept.
PCA (abbreviation)
Principal component analysis
LDA (abbreviation)
Linear discriminant analysis
What happens in feature selection?
You choose k<d important features (for a dataset with d features) and ignore the rest.
When is feature selection preferred instead of feature extraction?
When features are individually powerful or meaningful.
What happens in feature extraction?
The original x.i with i=1,…,d dimensions are projected to new k<d dimensions.
When is feature extraction preferred over feature selection?
When features are individually weak and have similar variance.
What does langrangian relaxation do?
It relaxes the equality constraints of a constrained optimization problem by introducing a Langrangian multiplier vector lambda.
What is the projection of x on the direction of w in PCA?
z = w.T * x
What kind of problems does LDA solve?
multi-class classification problems
What does LDA do?
It separates classes through dimensionality reduction. It maximizes the distance between the means of two classes and then minimizes the variance within the individual lasses.
What does a high eigenvalue mean in LDA?
That the associated eigenvector is more critical.
What matrices contain the eigenvectors calculated from the data in LDA?
1) between-class scatter matrix
2) within-class scatter matrix
What does the between-class scatter matrix contain in LDA?
The dataspread within each class
What does the within-class scatter matrix contain in LDA?
How classes are spread between themselves.
