Week 7

Question 1

Real uses of unsupervised learning

Answer 1

Customer segmentation (single parents, young party-goers)
Identifying fraud (bank transactions, GPS logs, bots on social media
Identifying new animal species
Creating the classes needed for a classification algorithm

Question 2

How does K-means work

Answer 2

Identifies points close to K centroids, where K is given by the user.

Question 3

How does DBSCAN work?

Answer 3

It finds core regions of high density and expands clusters from them.

Question 4

What is Hierarchical clustering?

Answer 4

Can be agglomerative or divisive, as long as you produce a hierarchy of clusters

Something like:
1. Split all points into clusters A and B
2. Split cluster A into clusters A1 and A2
3. Split cluster B into clusters B1 and B2
4. Split cluster A1 into …

Question 5

Hard vs Soft clustering

Answer 5

Hard: each object belongs in one cluster, similar to how a perceptron performs classification

Soft: objects are assigned to multiple clusters, with corresponding probabilities, similar to how a logistic regression performs classification.

Question 6

What did DBSCAN work best at compared to others?

Answer 6

Identifying rings.

Question 7

What is a key ingredient for clustering?

Answer 7

What data is represented and HOW
The similarity metric/distance metric
(L1 or L2 norm, Jaccard Similarity)

Question 8

What is Jaccard Similarity

Answer 8

A n B | / | A u B |

Question 9

What is Jaccard distance

Answer 9

1 - Jaccard Similarity

Question 10

What do we do in Dimensionality Reduction

Answer 10

Remove noise from the data
Focus on the features (or combinations of features that are actually important)
Less number-crunching = more efficient

Question 11

What are the two types of Dimensionality Reduction

Answer 11

Feature selection + extraction

Question 12

3 types of feature selection

Answer 12

Filter methods
Wrapper methods
Embedded methods

Question 13

Filter method examples

Answer 13

Information gain
Correlation with target
Pairwise correlation
Variance threshold

Question 14

Wrapper method examples

Answer 14

Recursive feature elimination
Sequential feature selection
Permutation importance

Question 15

Embedded method examples

Answer 15

L1 Lasso Regularization
Decision tree

Question 16

What is Variance thresholding

Answer 16

Filter method: low-variance features contain less information
Calculate variance of each feature, drop features with variance below threshold.
FEATURES MUST BE SAME SCALE!

Question 17

What is Forward Search

Answer 17

Wrapper method: create n models, one feature each, select best one. Create n-1 models, adding one feature, select best one, proceed until you have m features

Question 18

What is recursive feature elimination

Answer 18

Wrapper method: create n-1 models, with n-1 features each, select best, create n-2 models, removing one feature, select best one, proceed until you have removed m features

Question 19

What is a Decision Tree

Answer 19

Embedded method: Splits the data into a tree. Can split by Gini impurity coefficient (measures how pure a node is), information gain, and variance reduction

Question 20

What is feature extraction

Answer 20

Linear and nonlinear methods. Extract useful combinations of features in the data

Question 21

What is PCA

Answer 21

Linear method of feature extraction
Find an orthogonal coordinate transformation (rotation, rotation+reflection) such that every new coordinate is maximally informative

Question 22

How does PCA work

Answer 22

Flip points so that each coordinate is maximally informative and orthogonal to each other. Imagine rotating cube, we take a slice where x and y describes most of the data (z a little bit)

Question 23

What are the new variables from PCA called

Answer 23

Principle components, and they are linear combinations of the original coordinates.

yi are uncorrelated, and are ordered by the fraction of the total variation each retains. PC1 (y1) is most informative, then PC2 etc.

Question 24

When does PCA work best vs worst

Answer 24

BEST: high correlation between features
WORST: all variables are equally important and uncorrelated. PCA is uninformative.

Question 25

PCA input and output

Answer 25

Input high dimensional data
Output low dimensional data

Question 26

What is t-SNE

Answer 26

non-linear dimensionality reduction
Take the distribution of distances between N points in the dataset.
Scatter N points in 2 or 3 dimensions randomly
Move those N points around until the distribution of distances between them resembles D.

Question 27

Why t-SNE

Answer 27

Very useful for visualising high-D data

Question 28

What is UMAP

Answer 28

Similar to tSNE, only slightly different in every step
Runs faster uses less memory
No problem embedding into >3 dimensions
Can preserve both local and global structure.

Question 29

Problems with t-SNE and UMAP

Answer 29

Both depend a lot on their hyperparameters

Cluster sizes and distances between clusters means nothing

X and y axes are basically impossible to interpret.