Week 11 - Factor Analysis

Question 1

What 2 aims of factor analysis used for ?

Answer 1

1) When looking for the underlying (predicted) structure of a set of related variables (You are specifically looking for an effect that you want to exist)
2) Trying to reduce set of variables to a single, composite variable that combine shared info in them (Variance) (More of an exploratory approach)

Question 2

What does factor analysis seek to do?

Answer 2

Factor analysis seeks to form linear composites (sums of underlying variables)that represent the underlying structure of the correlation matrix
- Groups of highly intercorrelated items whose variance is well explained by the composite

Question 3

What is main goal of factor analysis?

Answer 3

To start with a set of measured variables and find the smallest number of factors to account for most the variance in the measured variable and the correlations between them

Question 4

Factor analysis allow to

Answer 4

Make statements about the patterns of intercorrelations

Question 5

Typical factor analysis set up

Answer 5

Set of observed (questionnaire items) or conceptually related items (No Y variables)

• Does underlying structure link a subset of items?
• Subsets of variables that correlate highly with one another and low on other factors

Question 6

Factor analysis is …

Answer 6

Geometric, based on correlations and the pattern of correlations that suggest linked items

Question 7

Factors (Composites) in factor analysis should

Answer 7

Explain more than or two variables

Should explain systematic variance and exclude individual error as much as possible

Question 8

History of factor analysis

Answer 8

Used in intelligence test (underlying g)

Question 9

What can;t factor analysis do

Answer 9

Test for significance

Question 10

2 reasons for doing factor analysis

Answer 10

1) data reduction to provide composites for further study
- empirical and technical reason
- after analysis is done we can obtain a factor as a new composite variable in our dataset
2) Investigate the underlying structure of a set of measured variables
- Give name to these variable using subjective judgement

Question 11

Latent Variable

Answer 11

Not directly observed variable, Factor analysis uncover these

Question 12

Observed Variables

Answer 12

Directly observed, exist in dataset already

Question 13

Two Varieties of Factor Analysis

Answer 13

1) Principle Components factor analysis (PCA)

2) Factor analysis (Common factor model) (PAF)

Question 14

Principle components factor analysis

Answer 14

Most basic form
Use mathematical properties of matrices (numbers)
Straight data reduction where error in original variables is not partialled out (all variance is used) (Does not ignore error in individual item)

Question 15

Common factor model

Answer 15

First utilised for theory building - start with a set of variables and need to know how many dimensions (components) they contain
Known as exploratory factor analysis
Analyses common variance and leave out the unique variance to each individual variable

Question 16

Differences Between PCA and PAF

Answer 16

PCA- Components PAF - Factors

PCA use R as is (diagonal)
PAF analyse covariance and modify R (estimated communalities on diagonal)

Question 17

Confirmatory factor analysis

Answer 17

Can test for significance, number of factors and and fit

Question 18

Subscale

Answer 18

Composite of constituent items

Question 19

Factor

Answer 19

Weighted linear composite

Question 20

In factor analysis we want to use data reduction to…

Answer 20

Reduce the number of measure variables (fewer = less error and more reliability)
Use a group of measured variables to indicate underlying construct that we can’t measure directly

Question 21

Correlation Matrix

Answer 21

Contain number of underlying factors and components (Implicit)

Question 22

Extraction

Answer 22

Extract factors, but only as few as needed to adequately summarise all the measured variables
Everytime we want to extract new factors we have to run another analysis
Each factor based on principle components of the correlation matrix

Question 23

Principle component of correlation matrix (4)

Answer 23

1) Ordered from first to last (As many principle components as they are variables)
2) Each account for as much variance as possible in the whole set of variables (first factor is always the largest as it is trying to account for all the variance in factors)
3) first the largest, then sequentially become smaller
4) Next principle component account for as much of the remaining variance and is uncorrelated with all the preceeding components.

Question 24

What is a factor/component

Answer 24

Each factor made up of sum of all measure variables (composite)

• weighted to reflect the strength of the correlation (Relationship with the variables)
• Higher weight = stronger relationship with the factor/component

Question 25

What are loadings

Answer 25

Tell us how much each variable link up with the factor/component
The correlation of each measured variable with each factor
Range from 0 - 1 (0 = no loading) (1 = high correlation/loading)
Help us to find out how much variance of the measured variable is shared with the factor/component
Loading sizes .3 = min .5/.6 = mod .7/.8 = high

Question 26

What are communalitites

Answer 26

How much each variable is explained by the factors/components
PCA = Communalities = 1
PAF = Communalities = Estimated squared multiple correlation (amount of variance accounted for by all the other variables)

After extraction Communialities become (h2) = sum of squared factor loadings

Use info from communalities to find how many factors and need to know number of factors to do communalities (indeterminacy problem)

Question 27

Rotation Types

Answer 27

Initially principle components are all uncorrelated (factors completely independant, no overlap)

Orthogonal (Varimax) = Not correlated

• Maintains strict structure
• Good for interpretation, may not match construct/ data

Oblique rotation (oblim) = Allow correlation

• Relax constraint
• distinct but related factors
• when underlying constructs are seperate but related

Question 28

Rotation

Answer 28

Re-weight the loadings to achieve simple structure

Question 29

Simple structure

Answer 29

Is desirable - each variable only have high loading on 1 factor (Usually this does not occur)

Question 30

Eigenvector

Answer 30

Principle Component

Question 31

Eigenvalue

Answer 31

Variance explained by each component (total variance of all variables)
Each eigenvector has an associated eigenvalue

Negative eigenvalue mean there is measurement or statistical issue

Question 32

Steps of factor analysis (6)

Answer 32

1) Obtain data on a set of variables
2) Standardise and create correlation matrix
3) Check there is a point to doing the factor analysis (Screening Run)
4) Extract components/factors
5) Rotate the ones you have extracted
6) Interpret loadings of variables on the factors

Question 33

KMO

Answer 33

Measure sampling adequacy (.6 adequate)

Lower than .6 = more randomness than factors

Question 34

Bartletts Sphericity

Answer 34

How far from 0 is your actual data (Identify matrix)

If violated = Can;t do meaningful factor analysis

Question 35

Initial Communalities

Answer 35

Starting point

Question 36

After extraction communalitites

Answer 36

Update after specific factors have been removed/extracted

Question 37

Choosing factors to extract

Answer 37

Retain with egeinvalue >1 (Account for more variance than one single variable)
Catells scree test - as variance explained by factors decreases less rapidly (elbow in the plot) suggest the number of factors to extract (Subjective - inter-rator reliability can be low)