6. Unsupervised Learning

Question 1

What is PCA.

Answer 1

Statistical tool that finds a low-dimensional representation of a dataset that contains as much info from the dataset as possible.

Question 2

Can PCA be used in a supervised and unsupervised setting?

Answer 2

Yes, PCR would be used in a supervised setting

Question 3

Formula for the mth PC

Answer 3

Formula

Question 4

The principle component loadings are constrained to what?

Answer 4

Formula

Question 5

Mth PC score formula

Answer 5

Formula

Question 6

For PCA, do the variables need to be scaled or centred?

Answer 6

Centred

Question 7

What is the maximum number of PC’s?

Answer 7

Min(n-1,p)

Question 8

What is the formula for PVE of the mth PC?

Answer 8

Formula

Question 9

What is the model equation for PCR? What happens when k=p?

Answer 9

Formula

Question 10

What are the two methods of calculating within cluster variation?

Answer 10

Formula

Question 11

What is the algorithm for k means clustering?

Answer 11

1. Randomly assign a cluster to each observation. This is the initial cluster assignments, pre determined number of clusters.
2. Calculate the centroid of each cluster
3. For each observation, identify the closest centroid and reassign to that cluster
4. Repeat steps 2 and 3 until the cluster assignments stop changing.

Question 12

What are 2 drawbacks of k means clustering?

Answer 12

1. Initial cluster assignments affect the final assignments.
2. Selecting k is an arbitrary process
3. Not robust

Question 13

Does k means need to have it variables standardized?

Answer 13

No, this relies heavily on the problem at hand

Question 14

Are k means and hierarchical clustering robust?

Answer 14

No

Question 15

Is k means clustering greedy?

Answer 15

Yes

Question 16

Centroid linkage is subject to _____. And single linkages has a dendogram that is _____

Answer 16

Inversions and skewed

Question 17

True or false: when performing PCR, it is recommended to standardize the predictors prior to generating the principle components.

Answer 17

True. This is to avoid high variance variables from monopolizing the principle components.

Question 18

Can PCR reduce overfitting?

Answer 18

Yes, instead of using all of the original variables, PCR uses only the first k PC’s to predict the response, which reduces overfitting.

Question 19

Is PCR useful for performing feature selection?

Answer 19

No, because we are using all variables when we find the principle components.

Question 20

Are all PCA loadings unique?

Answer 20

No. Each PC loading vector is unique (up to a sign flip). So, two different softwares can find the same loading vectors, but the signs may differ.

NOT unique bc they can take the negative value of themselves.

Question 21

Together, do all the principle components explain 100% of the variance?

Answer 21

Yes

Question 22

Which is more restrictive in its clustering nature, k means or hierarchical?

Answer 22

K means is less restrictive, because hierarchical clustering must produce nested clusters as a function of the number of clusters.

K means simply used euc distances, which has no specific structure on the results.

Question 23

If only 3 of the 4 principle components are used in a model, will the cumulative PVE ever be 100?

Answer 23

No, only if all PC’s are used.

Question 24

In cluster analysis, could we cluster the observations on the basis of the features or cluster the features based on the observations?

Answer 24

Both

Question 25

Which type of clustering has less areas of consideration?

Answer 25

K means. Only have to choose k.

In hierarchical, we have to choose linkage, dissimilarity measure, and deciding on the number of clusters (what height to cut the dendogram).

Question 26

True or false. Using all possible principle components provides the best understanding of the data.

Answer 26

False. Usually only the first couple are necessary

Question 27

What is a scree plot used for? What does the graph show?

Answer 27

It provides a method for determining the number of PC’s to use. It graphs the PVE against the PCs

Question 28

How many iterations of hierarchical clustering are done?

Answer 28

N-1

Question 29

If two different PCA models have unique loadings up to their signs, will they have the same PC score on the same X variables?

Answer 29

Due to the fact that the loadings have the same magnitude, but different signs (complete opposite) the PC scores will have the same magnitude, but different signs as well.

Question 30

Can we use categorical variables in clustering?

Answer 30

Only in hierarchical.

Question 31

True or false: kmeans can identify outliers and hierarchical cannot.

Answer 31

False. They both force every obs into a cluster, outliers can greatly affect the result of clustering but cannot identify the outliers themselves

Question 32

Is PCA a dimension reduction tool?

Answer 32

Yes

Question 33

Which of the following are true for hierarchical clustering?
A. Performing hierarchical clustering results in n-1 fusions
B. Categorical variables can be used in the analysis.
C. Hierarchical clustering is robust.

Answer 33

A and B.

Categorical variables can be used in hierarchical clustering because it can use correlation based distance instead of Euclidean. K means cannot use categorical variables.

Question 34

Can either k means or hierarchical clustering identify outliers?

Answer 34

No, neither can.

Question 35

True or False: A hierarchical agglomerative clustering begins with a cluster of all observations, after which splits are made at each step.

Answer 35

False. That is a hierarchical divisive clustering approach (Top down).

Question 36

True or false: hierarchical clustering yields better results than k-means clustering because the algorithm for hierarchical only needs to be run once.

Answer 36

False. That is an advantage of hierarchical over k means, but it doesn’t mean that hierarchical is better.

Question 37

If we look at a correlation matrix, if there is perfect correlation between two features, what can we conclude?

Answer 37

If there are p features, then p-1 features can perfectly explain the variance in the data set.

Question 38

From a correlation matrix, can we conclude the values (positive or negative) of the principle component loadings?

Answer 38

No

Question 39

If we look at a correlation matrix, and we see that none of the features are highly/moderately correlated, what can we assume in terms of the number of principle components needed?

Answer 39

More than 1 PC will be needed to explain a good amount of variability in the model.

Question 40

Can principle components be used for supervised learning?

Answer 40

Yes, because of PCR.

Question 41

Are the following two statements true or false?
A. The principle component score vectors have length p, while the principle component loading vectors have length n.
B. The eigenvectors of the matrix XtX are the PC directions, while the eigenvalues are the variances of the components.

Answer 41

A. False.
PC loading vectors have length n, while PC scores have length n.

B. True.
PCA involves finding the eigenvectors and eigenvalues of the matrix XtX.

Question 42

True or false:
A. Absolute correlation should not be used when performing hierarchical clustering on datasets with two features
B. Euclidean distance focuses on the magnitude of observation profiles rather than their shape
C. Two observations are said to be similar if they have a large correlation based distance

Answer 42

A. True. Because the absolute correlation between any two observations with two features is always 1. Absolute correlation needs at least 3 features.

B. True.

C. False. Large correlation distance = not similar

Question 43

In a biplot, the coordinates represent _______ and the lines represent _____.

Answer 43

PC scores. They have their values on the left vertical and bottom horizontal.
PC loading vectors. They have their values on the right vertical and top horizontal.

Question 44

From a datasets correlation matrix, we see that the predictors are highly correlated. how can we determine if the first PC loading vectors are all positive/negative/a mix of both?

Answer 44

Since the data is highly positively correlated, this means the loadings of the first PC are either going to be all positive or all negative

Question 45

True/false. The first k PC scores and the first k PC loadings provide the best approximation to the original dataset. If true, what is the formula?

Answer 45

True.

Question 46

True/False. PCA is still useful if the variables in the data set are uncorrelated.

Answer 46

False. It’s not useful.