How to perform LDA, PCA, SVD Flashcards
WHAT IS LATENT DIRICHLET ALLOCATION? P362
It’s a dimensionality reduction technique for text documents.
WHAT DOES THE LDA MODEL DO TO SEPARATE SAMPLES BY THEIR CLASSES? P362
It tries to find a linear combination of input variables that achieves the maximum separation for samples between classes and the minimum separation of samples within each class. E.g. low variance in the group, high variance between groups.
WHY IS IT BETTER TO STANDARDIZE DATA BEFORE USING LDA FOR DIMENSIONALITY REDUCTION? P362
LDA for multiclass classification is typically implemented using the tools from linear algebra. As such, it’s good practice to perhaps standardize the data prior to fitting an LDA model.
HOW CAN WE USE LDA FOR DIMENSIONALITY REDUCTION? P363
Using a pipeline, we can in one step set the LDA and its n_components and in the next step, we can feed the output of this transformation to a model. Code P364
WHAT IS THE RANGE OF POSSIBLE VALUES FOR N_COMPONENTS PARAMETER OF LDA? P364
<= Min (n_classes – 1 , n_features)
EXTERNAL Q: DOES PCA NEED STANDARDIZATION?
Yes because it’s a technique that comes from the field of linear algebra
Dimensionality reduction is often called ____ and the algorithms used are referred to as ____. P370
Feature projection, Projection methods
WHICH TOOLS FROM LINEAR ALGEBRA ARE USED FOR PCA? P370
Matrix decomposition like an Eigendecomposition or singular value decomposition (SVD).
DOES SVD USE TECHNIQUES FROM THE FIELD OF LINEAR ALGEBRA? P378
Yes.
WHAT IS SPARSE DATA? P378
Sparse data refers to rows of data where many of the values are zero.
WHAT ARE SOME EXAMPLES OF SPARSE DATA APPROPRIATE FOR APPLYING SVD FOR DIMENSIONALITY REDUCTION? P378
- Recommender systems
- User-song listen counts
- Bag of words counts
- TF/IDF
- One hot encoding
- Text classification
- User-movie ratings
- Customer-product purchases
WHAT IS SCIKIT-LEARN CLASS FOR SVD? P378
TruncatedSVD
