Dimensionanility Reduction and Matrix Decomposition Flashcards
What is the aim of PCA?
To find a set of orthogonal principal components which capture the direction of maximum variance in the data.
What is the variance of X in the direction v?
vT XT X v
Write down the optimisation problem of PCA and its Lagrange form
Check notes
What is the difference between PCA and Whitening?
Whitening normalises the variance along each principal component.
What are the pros and cons of Non-negative matrix factorization?
Pros
* Natural fit for positive-valued data
* Can be interpreted (meaningful signs)
Cons
* Only applicable to non-negative data
* Optimization procedure is non-convex; requires initialization
* “Interpretability” is unreliable
* Learned components are not orthogonal nor naturally ordered
How does independent component analysis work? When is it most useful?
First whiten X and then find X=AS where the components in S are statistically independent.
Good for time series with lots of signals.
What is the aim of t-SNE?
To match distributions of distances between points in original high dimensional space to points in a lower dimensional space.
What do Cross-decomposition methods do?
They find associations across multi-view data.
What is PLS? Write it as an optimisation problem and an e/val problem
PLS is a method for finding the directions of the maximum shared covariance between two paired views of the data.
Check notes for formulation
What is CCA? Write it as an optimisation problem and a SVD problem
Canonical Correlation Analysis is a method for finding the directions of the maximum correlation between two paired views of the data.
Check notes for formulation
Can PLS and CCA be computed when p>N?
PLS can, CCA cannot (within-view cov matrix cannot be inverted)
Write down the optimisation problem related to regularised PLS and CCA
Check notes
