Week 12

Question 1

What is principal component analysis? What kind of data ?

Answer 1

dimension reduction – i.e., we transform the original X’s and work with M transformed variables, where M < P.

unsupervised learning method that is used to summarize a large set of correlated variables.

Question 2

What are loadings?

Answer 2

linear weights

Question 3

What can PCA be used for?

Answer 3

inputs for supervised learning methods or for data visualization.

extract variables corresponding to the directions along which the data vary the most, CHOOSE PHI JM

Question 4

What is the primary purpose of PC

Answer 4

extract variables corresponding to the directions along which the data vary the most.

Question 5

How many components can we decompose X into?

Answer 5

min(p,n-1) independent principal components

Question 6

What are the principal components obtained?

Answer 6

linear combination of All x with linear weights called loadings

Question 7

The _____________ and __________ are unique up to ______

Answer 7

principal component
loadings vector
sign

Question 8

What are principal components ordered on ?

Answer 8

share of the total variance of X explained. This share equals each component’s variance divided by the sum of variances of all PC’s.

Question 9

If you have fewer x than observatrions, …

Answer 9

will obtain all the p components

Question 10

How to tell whether 2 pca are independent?

Answer 10

PERPENDICULAR

Question 11

Are principal components scale variant? Why ?

Answer 11

Scaling up one variable by a constant would blow up its variance and change it’s loading

have to standardize data unless same units

Question 12

When does pca work best? What kind of data must it be used on ?

Answer 12

highly correlated data (R VALUE ABOVE 0.5)

when continuous var ( cannot have categorical variable, or else need to use CATPCA)

Question 13

What is component score?

Answer 13

value of z variable

Question 14

What are the axes in pca biplot?

Answer 14

left and bottom: component score(zm)
top and right : loadings

Question 15

What is scree plot? How to find number of componennts?

Answer 15

proportion of total variance of X explained by each subsequent component

look for an “elbow” in the plot, where contribution to variance drops sharply and flattens. keep retaining components until elbow appears in the plot

Question 16

Why is it best to use pca on highly correlated variable?

Answer 16

Highly correlated variables provide redundant information on one another. Should be possible to extract a set of independent factors that explain the bulk of variation over time.

Question 17

What is the level , slope , and curvature component?

Answer 17

level: loadings are similar for all of the components

slope:loadings are similar for all of the components (downward sloping all the way past 0)

curvature: flip sign twice, u shape graph, induces movement in yield curve like butterfly

Question 18

What is the problem of constraining betas? How does pca balance this flaw?

Answer 18

Constraints on β’s may incur bias. However, dimensionality reduction has the potential to greatly reduce variance.

Question 19

What do we do once we know PCA?

Answer 19

we can run the principal components regression.

Question 20

What are the steps to running PCR?

Answer 20

Run PCA on the predictor matrix X.

Construct the sample principal components Z.

Remove all but M < P first principal components.

Regress y on z1, …, zM .

Question 21

How to choose M for unsupervised PCA and PCR?

Answer 21

unsupervised PCA, there is no clear rule: some recommend to plot proportion of variance explained on a scree plot and retain the PC’s until the plot starts to drop off (at the “elbow”).

For PCR, we can simply use K-fold cross-validation

Question 22

What is the underlying assumption in PCR?

Answer 22

directions of highest variability in X are those most associated with y

Question 23

When does the assumption fail?

Answer 23

useful signal may “hide” in the low variance component that would be discarded.

Question 24

Does PCR use variable selectrion?

Answer 24

NO

ALL ARE USED TO FORM PC