PCA

Question 1

What is PCA?

Answer 1

Separation of multivariate data into N most important components

Identify principle directions in which data varies

Explains variance covariance structure of a set of variables

Organise data into a set of composite variables

Question 2

What are the 2 approaches to dimensionality reduction? Describe them.

Answer 2

1. Feature Selection:
   Choose a subset of features present in the data set and extract data points with these features.
2. Feature Extraction:
   Create a new subset by combining different features

Question 3

What are the 2 types of feature extraction? Describe them.

Answer 3

1. Classification - ICA
   Feature extraction mapping enhances discriminator information in low dimensional space
2. Signal Representation - PCA
   Feature extraction mapping accurately represents samples in low dimensional space

Question 4

What underlying component does PCA work on?

Answer 4

Total sample variance

Question 5

What are principle components?

Answer 5

Eigen-vectors that describe the direction of variance in data
All PCs are orthogonal i.e. Covariance between any 2 PCs = 0

Question 6

What do the Eigen-vectors and values represent?

Answer 6

Eigenvalues are the magnitude of the variances of the principle components

Eigenvectors describe direction of variance

Question 7

What is the PCA algorithm?

Answer 7

1. Define input matrix of dimension P
2. Calculate mean of each column
3. Calculate new matrix with centered value columns (mean + value)
4. Calculate covariance matrix
5. Calculate eigenvectors/values
6. Select K largest PCs