12 | DW-2 | ICA Factoranal MDS Flashcards
Pitfalls of PCA
What is a major limitation of PCA in separating mixed signals?
PCA assumes orthogonality and only captures variance, not statistical independence.
Goal of ICA
What is the main goal of ICA?
To find statistically independent components from mixed signals.
Statistical Independence
How does statistical independence differ from uncorrelatedness?
Uncorrelated variables may still have dependencies, whereas statistically independent variables do not share any information.
Excursion: Mutual Information (MI)
What does mutual information (MI) measure?
The amount of shared information between two random variables.
Basic Principle of ICA
How does ICA separate mixed signals?
By maximizing statistical independence between components.
PCA vs. ICA
How does ICA differ from PCA?
PCA finds orthogonal components maximizing variance, while ICA finds independent components.
Measure of Non-Gaussianity: Kurtosis
Why is kurtosis used in ICA?
Non-Gaussianity helps distinguish independent components.
ICA Examples
What is a common real-world application of ICA?
Blind Source Separation, e.g., separating mixed audio signals (the cocktail party problem).
ICA Workflow
What are the main steps of ICA?
Centering, whitening, and finding independent components.
Properties of ICA
Why does ICA require at most one Gaussian-distributed source?
Because Gaussian sources cannot be separated using ICA.
MDS: Purpose
What is the main goal of Multidimensional Scaling (MDS)?
To represent high-dimensional data in a lower-dimensional space while preserving pairwise distances.
MDS: Input Data
What type of data does MDS require as input?
A distance or dissimilarity matrix.
MDS: Classical vs. Non-Metric
What is the difference between classical MDS and non-metric MDS?
Classical MDS preserves Euclidean distances, while non-metric MDS preserves rank order of distances.
MDS: Stress Function
What does the stress function in MDS measure?
The difference between original and projected distances.
MDS: Applications
In which fields is MDS commonly used?
Psychology, genomics, and market research for visualizing similarities.
Factor Analysis
Factor Analysis: Purpose
What is the main goal of factor analysis?
To identify latent variables that explain observed correlations.
Factor Analysis: Difference from PCA
How does factor analysis differ from PCA?
Factor analysis models hidden factors causing correlations, while PCA captures variance without assuming underlying causes.
