PCA Final

Question 1

PCA

Answer 1

Principal Component Analysis

Question 2

PCA is a

Answer 2

dimensionality reduction technique

Question 3

Big idea 1: Take dataset in high dimension space

Answer 3

and transform it so it can be represented in low dimension space, with minimal or no loss of information

Question 4

Big idea 2: Extract

Answer 4

latent information from the data

Question 5

The PCA transformation results in

Answer 5

a smaller number of principal components that maximizes the variation of the original dataset, but in low dimension space

Question 6

These principal components are

Answer 6

linear combinations of the original variables, and become the new axes of the dataset in low dimension space

Question 7

3 goals of PCA

Answer 7

Feature reduction: reduce the number of features used to represent the data
The reduced feature set should explain a large amount of information (or maximize variance)
Make visible the latent information in the data

Question 8

PCA creates

Answer 8

projections (principal components) in the direction that captures most of the variance

Question 9

Sparser data has

Answer 9

greater variance (spread out)

Question 10

Denser data has

Answer 10

lesser variance (clustered together)

Question 11

The projections will always be

Answer 11

orthogonal to each other

Question 12

Mathematics behind PCA

Answer 12

Eigenvalues and Eigenvectors

Question 13

Mathematics equation

Answer 13

Matrix A times eigenvector X = Eigenvalue times eigenvector

Question 14

Eigenvalue and Eigenvector meaning

Answer 14

An eigenvector of a matrix is a nonzero vector that, when it is multiplied by the matrix, does not change its direction. Instead, the vector is simply scaled by some factor

Question 15

Eigenvector are vectors that

Answer 15

remain unchanged when multiplied by A, except for a change in magnitude. Their direction remains unchanged when a linear transformation is applied to it

Question 16

When we eigendecompose, when we decompose matrix, do eigendecomposition

Answer 16

if my matrix has n columns or n dimensions, i am going to have n eigenvalues and n eigenvectors

Question 17

Our matrix/dataset gets decomposed into

Answer 17

Eigenvectors
Eigenvalues

Question 18

Should we standardize for PCA?

Answer 18

Yes, always standardize

Question 19

five fields returned from prcomp(A,…)

Answer 19

sdev
rotation
center
scale
x

Question 20

sdev

Answer 20

Square root of the eigenvalues, ordered from largest eigenvalue to the smallest

Question 21

rotation

Answer 21

Matrix whose columns contain the eigenvectors (also called principal loadings)

Question 22

center

Answer 22

Mean of the columns of the matrix A

Question 23

scale

Answer 23

std dev of the columns of the matrix A

Question 24

x

Answer 24

Data from matrix A in rotated space (also called principal component scores)

Question 25

How is the data in rotated space computed

Answer 25

dot product

Question 26

Top and right axis indicate

Answer 26

tell you where these vectors are going to occur, for loading vectors

Question 27

Bottom and left axis indicate

Answer 27

The scores by which we situate the data point in their new rotated states

Question 28

How many principal components do we need?

Answer 28

As many that explain most of the variance, and adding any more to the model results in diminishing gains in variance

Question 29

Key idea: What is the proportion of variance

Answer 29

contributed by each principal component loading?

Question 30

Total Variation

Answer 30

sum of all PC

Question 31

Proportion of variance explained by ith principal component loading

Answer 31

PCi / TotalVariation

Question 32

variance is

Answer 32

squared std dev

Question 33

What do you have to do before attempting to use observations in any model?

Answer 33

Transform all of your observations (in sample, out of sample) from their natural representation to principal component scores