Factor Analysis

Question 1

What do factor analyses do?

Answer 1

Look for a lower-dimensional set of factors that explain how different variables covary/correlate with one another in multivariate data

Question 2

What is the goal of principal component analysis?

Answer 2

Find axes explaining maximum variance in the original dataset

Question 3

Features of PCA

Answer 3

Reweights each feature transforming it onto a new set of axes
‘Rotates’ data
Dimensionality is ‘reduced’ while retaining fewer PCs

Question 4

What are the different PCs?

Answer 4

PC1 - single axis explaining the maximum variance in the data
PC2 - explains maximal variance after removing PC1
PC3 - explains maximal variance after removing PC1/PC2, etc.

Question 5

Exploratory Factor Analysis

Answer 5

Recover interpretable latent variables explaining a dataset
Subsection of data
Predicted performance is a weighted equation of multiple scores
Estimate latent variables whilst estimating factor loadings
Residuals left over

Question 6

Path Diagram

Answer 6

Compactly shows resulting strutcure after estimating set of equations
Does not inherently know certain things
Correspond to factor Loading

Question 7

Latent Variables

Answer 7

Explain a small number of original variables
Correlated with each other
Recovers correlations between two variables

Question 8

Features of Factor Analysis

Answer 8

Estimates factors actually based on modelling the shared and unique variance in the covariance matrix
Labelled arbitrarily - post hoc
Rotated to be more interpretable
Correlate with each other

Question 9

Purpose of PCA va FA

Answer 9

PCA - transform original variables into uncorrelated variables explaining maximum variance
FA - identify latent factors explaining observed correlation between original variables

Question 10

Model/Assumption PCA vs FA

Answer 10

PCA - no underlying model; treat variables as equal and maximise variance explained
FA - underlying model that separate shared variance due to factors and unique variance

Question 11

Component

Answer 11

Components are orthogonal linear combinations of all variables, ordered by variance explained

Question 12

Factors

Answer 12

Latent variables aiming to be ‘interpretable’, scores may be correlated

Question 13

Interpretation and use of PCA and FA

Answer 13

PCA - dimensionality reduction/visualisation of data; axes are not ‘constructs’, but axes that capture most variance
FA - factors may be interpreted as representing ‘constructs’; often used to identify dimensions of a test/questionnaire

Question 15

Scree Plot

Answer 15

Amount of variance being explained by each component prior to any rotation
Elements of subjectivity

Question 16

Kaiser’s Criterion

Answer 16

Factors with eigenvalues > 1: meaning a factor will explain as much variance as one variable
May result in too many factors when number of variables is large

Question 17

Scree Test

Answer 17

Factors with clearly higher eigenvalues than previous factors - identifiable by change of slope on a component plot: elbow
Element of subjectivity

Question 18

Interpretability

Answer 18

Factors that make good sense from a theoretical perspective
Reliance on researcher’s interpretation

Question 19

Factor Analysis fitting the Data

Answer 19

More complex models explain more of the variance in a data set

Question 20

Bayesian Information Criteria (BIC)

Answer 20

Measures model fit while adjusting for nFactors

Question 21

Factor Rotation

Answer 21

Rotated to attain interpretability
Doesn’t affect amount of shared/unique variance - changes how the factors are loaded
Two forms of rotation: orthogonal and oblique

Question 22

Orthogonal Rotation

Answer 22

Factors are uncorrelated by definition
Commonly used example: Varimax - maximises the variance of the squared loadings
Preserves uncorrelated factors

Question 23

Oblique Rotation

Answer 23

Factors can become correlated with one another
Commonly used example - direct Oblimin
Test where there is a correlation between latent variables, before using simpler orthogonal
Cannot be a complete rotation
Different factors rotated in different directions
Eventual factors can be correlated
Select number of factors and check with oblique rotation then preserve it