Factor Analysis Flashcards
What do factor analyses do?
Look for a lower-dimensional set of factors that explain how different variables covary/correlate with one another in multivariate data
What is the goal of principal component analysis?
Find axes explaining maximum variance in the original dataset
Features of PCA
Reweights each feature transforming it onto a new set of axes
‘Rotates’ data
Dimensionality is ‘reduced’ while retaining fewer PCs
What are the different PCs?
PC1 - single axis explaining the maximum variance in the data
PC2 - explains maximal variance after removing PC1
PC3 - explains maximal variance after removing PC1/PC2, etc.
Exploratory Factor Analysis
Recover interpretable latent variables explaining a dataset
Subsection of data
Predicted performance is a weighted equation of multiple scores
Estimate latent variables whilst estimating factor loadings
Residuals left over
Path Diagram
Compactly shows resulting strutcure after estimating set of equations
Does not inherently know certain things
Correspond to factor Loading
Latent Variables
Explain a small number of original variables
Correlated with each other
Recovers correlations between two variables
Features of Factor Analysis
Estimates factors actually based on modelling the shared and unique variance in the covariance matrix
Labelled arbitrarily - post hoc
Rotated to be more interpretable
Correlate with each other
Purpose of PCA va FA
PCA - transform original variables into uncorrelated variables explaining maximum variance
FA - identify latent factors explaining observed correlation between original variables
Model/Assumption PCA vs FA
PCA - no underlying model; treat variables as equal and maximise variance explained
FA - underlying model that separate shared variance due to factors and unique variance
Component
Components are orthogonal linear combinations of all variables, ordered by variance explained
Factors
Latent variables aiming to be ‘interpretable’, scores may be correlated
Interpretation and use of PCA and FA
PCA - dimensionality reduction/visualisation of data; axes are not ‘constructs’, but axes that capture most variance
FA - factors may be interpreted as representing ‘constructs’; often used to identify dimensions of a test/questionnaire
Scree Plot
Amount of variance being explained by each component prior to any rotation
Elements of subjectivity
Kaiser’s Criterion
Factors with eigenvalues > 1: meaning a factor will explain as much variance as one variable
May result in too many factors when number of variables is large
Scree Test
Factors with clearly higher eigenvalues than previous factors - identifiable by change of slope on a component plot: elbow
Element of subjectivity
Interpretability
Factors that make good sense from a theoretical perspective
Reliance on researcher’s interpretation
Factor Analysis fitting the Data
More complex models explain more of the variance in a data set
Bayesian Information Criteria (BIC)
Measures model fit while adjusting for nFactors
Factor Rotation
Rotated to attain interpretability
Doesn’t affect amount of shared/unique variance - changes how the factors are loaded
Two forms of rotation: orthogonal and oblique
Orthogonal Rotation
Factors are uncorrelated by definition
Commonly used example: Varimax - maximises the variance of the squared loadings
Preserves uncorrelated factors
Oblique Rotation
Factors can become correlated with one another
Commonly used example - direct Oblimin
Test where there is a correlation between latent variables, before using simpler orthogonal
Cannot be a complete rotation
Different factors rotated in different directions
Eventual factors can be correlated
Select number of factors and check with oblique rotation then preserve it
