Unsupervised Learning

Question 1

What is unsupervised learning?

Answer 1

Finding patterns from unlabeled data (if you know there is some structure).
Exploratory data analysis task.

Question 2

Unsupervised learning methods used in Data Understanding?

Answer 2

Scatter plots, 2/3 dimensional PCA.
Correlation analysis.

Question 3

What is clustering?

Answer 3

Identifying groups of “similar” data objects. Unsupervised learning is 90% clustering. Algorithms: hierarchical clustering.

Question 4

What is association analysis?

Answer 4

Finding associations between attributes or typical combinations of values likeIf Demand=high and Supply=low then Price=high.

Question 5

What kinds of problems could you solve with these methods?

Question 6

Why is the evaluation of unsupervised learning results more
difficult than supervised learning?

Answer 6

Evaluation of supervised learning pretty straight forward: use cross-validation and separate test data.
Unsupervised learning: evaluation criteria not well defined. Can measure compactness of clusters, support and confidence for associatioin rules.
Plausibility checks: does model make sense to experts (interpretability)?

Question 7

Hierarchical clustering algorithms

Answer 7

Build clusters step by step. Usually, a bottom-up strategy where at first each data object is considered a cluster and step by step joining clusters that are close to each other (agglomerative hierarchical clustering). See the opposite strategy divisive hierarchical clustering.

Question 8

Divisive hierarchical clustering

Answer 8

The entire data set starts as a cluster and is divided step by step into smaller ones.

Question 9

How does the clustering happen?

Answer 9

a dissimilarity (or similarity) matrix is used to find the most different groups. Distance measures: Euclidean (simplest), Manhattan, Pearson, Tscheby…
Clusters should not depend on the measurement unit! (some normalization should happen)

Question 10

Isotrophic vs non-isotrophic distances