Exploratory Factor Analyses

Question 1

When do you carry out a factor analysis?

Answer 1

When you have continuous latent data and continuous observed data

Question 2

How did we know that the observed data was categorical in IRT?

Answer 2

Answeres were yes/ no, likert scale etc

Question 3

How do you know if the observed data is continuous?

Answer 3

In psychology we rarely have truly continuous data, an example is reaction time. As a rule; if an item has more than five points (scale) and forms a normal distribution, you can consider it as a continuous item and perform factor analysis on it.

Question 4

When is factor analysis often applied?

Answer 4

Sum scores on sub tests (e.g dimensions of intelligence)

Question 5

What is the exciting thing about factor analysis as compared to item response theory according to Dylan

Answer 5

The nice thing about IRT is that you really analyse the individual items. FA is more flexible; continuous data is easier to model and the mathematics and formulas is more simple.

Question 6

Why did IRT form an S shaped curve?

Answer 6

Because you’re modelling a probability of an outcome (correct score) since you have categorical data

Question 7

Does one-factor FA have a s shaped curve? Explain

Answer 7

No, it’s a linear function. The expected value is on the x axis and since we’re no longer estimating probability and working with continuous data, the scores go higher than one and we can use a linear model. For this reason it is sometimes known as the linear factor model.

Question 8

Explain the linear factor analysis equation

Answer 8

𝐸 (𝑋𝑝𝑖 |𝜃𝑝) =𝑏𝑖 +𝑎𝑖𝜃𝑝
or 𝑋𝑝𝑖 =𝑏𝑖 +𝑎𝑖𝜃𝑝 +𝜖𝑝𝑖
with VAR(Xpi | 𝜃p)

𝑏𝑖 is commonly referred to as item attractiveness but it roughly translates to the IRT item difficulty/ easiness parameter and it is the intercept in the slope.

e. g., “I think about suicide” has a low attractiveness
e. g., “I am statisfied with my life” has a higher attractiveness

𝑎𝑖 is the item discrimination, same as IRT and it forms the slope of the function. We model the expected value of the continuous item, and if it is a good model then the observations should be around that line.

the variance is just how much the observed points vary for the modelled line

Question 9

How can the notation for this model change? Explain

Answer 9

𝐸 (𝑋𝑝𝑖 |𝜃𝑝) =𝑏𝑖 +𝑎𝑖𝜃𝑝 is written as:
𝑌𝑝𝑖 =𝜐𝑖 +𝜆𝑖𝜂𝑝 +𝜖𝑝𝑖 in factor analysis literature
𝜐𝑖 is an intercept
𝜆𝑖 is a factor loading
𝜂𝑝 is the common factor
𝜖𝑝𝑖 is the residual

They mean the same things but are just written differently in IRT literature compared to FA literature.

Question 10

Conceptually, what is the goal of factor analysis?

Answer 10

Its a statistical approach to extract common variance from the items and separate it from the variable specific effects

Question 11

How do the parameters translate to the variance measured?

Answer 11

𝑌𝑝𝑖 =𝜐𝑖 +𝜆𝑖𝜂𝑝 +𝜖𝑝𝑖

The common factor variance (𝜎^2𝜂) is the variance caused by the latent trait

The factor loadings (𝜆𝑖) tunes how much common factor variability comes from each item out of all the variability in the items - how well each item measures the latent trait

The intercepts (𝜐𝑖), in a single group application, are simply the item means

The residual variances (𝜎^2,𝜖𝑖) tells you how much of the variance is unique to the item

Question 12

What does this model imply about the data? How can this be used in calculations?

Answer 12

The model implies some kind of structure in the data in terms of the variance. You can calculate how much variability does this model predict for item 1. You can compare this to the observed variance which should be the factor loading squared times the variance plus the residual variance.

It also implies some kind of covariance between the items since they measure the same thing. If you score high on one item you are assumed to score higher on a second item. You can calculate the expected covariance through this.

Question 13

How do you calculate the expected covariance? What should you look for in this?

Answer 13

The expected covariance is the first factor loading multiplied by the second factor loading etc multiplied by the factor variance. You should look to see if this is close to the observed covariance to assess whether this is a good model for the data.

Question 14

How is the proportion of the variance calculated?

Answer 14

𝜌|2𝑌|𝑖𝜂𝑝 = 𝜆|2𝑖| 𝜎|2𝜂| / 𝜆|2𝑖| 𝜎|2𝜂| +𝜎|2𝜖|𝑖
i.e the variance of the latent variable multipled / the total variance = the variance explained by the factor

The varaince not explained by the factor (uniqueness) is calculated by:

1 - 𝜌|2𝑌|𝑖𝜂𝑝 = 𝜎|2 𝜖𝑖| / 𝜆|2𝑖| 𝜎|2𝜂| +𝜎|2𝜖|𝑖

However most of the time you can just read these from the output in Rstudio

Question 15

What is the equivalent of chronbachs alpha from CTT here and hoiw is it calculated?

Answer 15

The reliability of the sum score, the ratio between the variability due to the latent trait and the total variability

𝜌𝑌 =𝜎^2 (𝑇 ) / 𝜎^2 (𝑌) = 𝜎^2 (𝐸(𝑌)) / 𝜎^2 (𝑌 )
= …
= 𝜎𝜂2 × (E|𝑛,𝑖=1| 𝜆𝑖)^ 2 /
𝜎|2,𝜂| × (E|𝑛,𝑖=1| 𝜆𝑖)^ 2 + E|𝑛,𝑖=1 𝜎|2,𝜖𝑖|

i.e multiply the factor variance by the sum of the loadings squared, divided by the factor variance by the loadings squared by the sum of the residual variances squared summed

Question 16

How does identification change compared to IRT and why?

Answer 16

It is very similar to IRT, we need to identify the model because the latent variable doesn’t have a scale or a unit so we have to create one. In factor analysis, however, we play around with this more. In IRT we fixed the mean to 0 and the std to 1 because the R packages don’t allow you to change it much and its not interesting when its one dimensional. People like to change the identification to get a different scale for the parameters, this will not change the conclusions or the p-value since the proportions between the parameters don’t change.

Question 17

Give an example of how you can have two different options for identification in factor analysis

Answer 17

Option 1:
• 𝜇𝜂 =0
• 𝜎𝜂2 =1

Option 2:
• 𝜇𝜂 =0
• Fix one factor loading to 1
By picking an arbitrary factor loading and fixing it to one is like saying that the scale of the latent variable is the same as the scale of that item

Question 18

What is the most dominant approach to parameter estimation in factor analysis?

Answer 18

Maximum likelihood

Question 19

What does using MLE in this instance assume about the data?

Answer 19

Normally distributed

Question 20

What can you do as opposed to fitting the model on the raw data with MLE for a factor analysis? Why might you want to do this?

Answer 20

You have the option to only analyse the observed covariance matrix which is very useful for factor analysis since a covariance matrix already contains all the information about the structure of your data. From a covariance matrix you ca already fit a one parameter model since you have your factor loadings and residual variance.

Question 21

What is a downside to using the covariance matrix in estimating the MLE for a factor analysis?

Answer 21

There’s no intercepts in the model because for the intercepts you really need the means of the data which is not contained in a covariance matrix.

Question 22

What two alternative methods for parameter estimation exist?

Answer 22

Weighted least squares and bayesian estimation (Also popular but not discussed in this course)

Question 23

Given data with 1000 subjects answering 10 questions on a 5 point likert scale, how would you write up code to carry out a factor analysis? Explain the code

head(E)

Answer 23

library(lavaan)
model = “eta = ~Y1 + Y2 + Y3 + Y4 +Y5 + Y6 +Y7 + Y8 +Y9 + Y10”
fit = cfa(model = model, data = E, meansstructure = TRUE, std.lv = TRUE)

where eta is the common factor and can be given any name akin to a variable, =~ indicated that it is measured by…, cfa runs a confirmatory factor analysis, data is called E, meanstructure = TRUE means that you want to calculate the intercepts, std.lv = TRUE means that you want to standardise the latent variable with 𝜇𝜂 =0 and 𝜎𝜂2 =1

Question 24

There is a screenshot of CFA output in the docs. Describe the information that is presented under the factor name

Answer 24

The factor loadings are given under whatever you called the factor (e.g eta). The estimates are the factor loading estimates (how well an item measures the latent trait), negative items are contra-indicative in that the higher you score on an item, the lower you score on a latent trait. If you divide the estimate by the standard error then you get the z value, both of which are shown. If an estimate is shown to be non-significant then it does not measure the latent trait. Std.a;; are the standardised factor loadings; they are the correlations between the item/variable and the factor. If you square this value then you get the shared variance between the item and the common factor.

Question 25

There is a screenshot of CFA output in the docs. Describe the information that is presented under intercepts

Answer 25

For the intercepts the estimated intercepts are simply the mean of the variable (mean response to an item). Std error, z value and Pvalue are similar to before and not interesting. Std.all gives the standardised intercepts; the mean if the variable is standardised (We won’t use this much.)

Question 26

There is a screenshot of CFA output in the docs. Describe the information that is presented under variances

Answer 26

Under variances, the estimates are the estimated residual variances (How much they deviated from the model. The variance of the latent trait (efa) is set to one here if you specified lvd.sd = TRUE. Std.ERr, zvalue and pvalue are also uninteresting here especially since you cannot even use these to test for the residual variances larger than 0 due to a boundary constraint. The STd.all gives the standardised residual variances which are interesting; they are the unexplained variance by factor, 1 - P|2,Y|mp i.e ‘uniqueness’

Question 27

Contrast the two main types of factor analysis

Answer 27

Exploratory factor analysis: is used when the factor structure is unknown (number, loadings)
• The number of factors, 𝑚, is systematically altered by the researcher
• All items load on all factors

Confirmatory factor analysis is used when the number of factors, 𝑚, is derived from theory/expectations
• The loadings are derived from theory/expectations

Question 28

Give the model for an exploratory factor analysis

Answer 28

𝑌𝑝𝑖 =𝜆1𝑖 𝜂1𝑝 +𝜆2𝑖 𝜂2𝑝 +⋯𝜆𝑚𝑖 𝜂𝑚𝑝 +𝜖𝑝𝑖

up until 𝑚 factors where 𝜂1𝑝 is the first factor for person p and 𝜂2𝑝 is the second factor for person p.

𝜆1𝑖 is the loading of item i on that first factor.
𝜆2𝑖 is the loading of item 𝑖 on factor 2, 𝜂2𝑝 etc.

𝜖𝑝𝑖 is the residual

Question 29

Where is the intercept in an exploratory factor analysis model?

Answer 29

There is not intercept, the key is to reveal the factor structure. Everything you need for this factor structure is already in a covariance matrix and so the means/ an intercept is not needed.

Question 30

What are the population parameters of an exploratory factor analysis?

Answer 30

𝜎|^2,𝜂1| is the variance of factor 1, 𝜂1𝑝
𝜎|^2,𝜂2| is the variance of factor 2, 𝜂2𝑝

etc.
𝜎𝜂1𝜂2 is the covariance between factor 1 and 2
etc.

Question 31

What are the two most popular methods of parameter estimation in factor analysis?

Answer 31

Principal factoring

Maximum likelihood

Question 32

What does principal factoring consist of?

Answer 32

• Based on the Eigenvalues of the principal factors.
• Same tools as in principal component analysis (see IRT lecture)
  
  • Kaiser criterion
  • Scree plot
  • Parallel analysis

Question 33

Give two advantages of principal factoring

Answer 33

No distributional assumptions

No improper solutions (e.g., negative variances)

Question 34

Give a disadvantage of principal factoring

Answer 34

No explicit falsifiable model

-You can calculate all you parameters yet still not know whether your factors are fitting well or not

Question 35

What does maximum likelihood consist of in factor analysis?

Answer 35

As discussed in IRT lecture, but with a normal distribution for the data

Question 36

Give two advantages of using maximum likelihood as paremeter estimation for EFA and two disadvantages

Answer 36

+ Explicit model based
+ Model falsification
- Sometimes improper solutions (e.g., negative variances etc)
- Multivariate normal distribution assumption for the data

Question 37

What is meant by saying you can obtain improper solutions using MLE to estimate the parameters?

Answer 37

e.g: Sometimes with MLE you estimate your parameters and get a negative residual or common factor variance which shouldn’t be possible

Question 38

How can you falsify your model with MLE? (2)

Answer 38

You can falsify your model with means of fit measures, There are many but two of which are:

• 𝜒2-goodness of fit measure
• Root Mean Squared Error of Approximation

Question 39

What is involved in the 𝜒2-goodness of fit measure?

Answer 39

It just gives a significance test with the following hypotheses:
• H0: Model fits
• HA: Model does not fit

So in this case you want your test to be insignificant

Question 40

What is involved in the Root Mean Squared Error of Approximation?

Answer 40

You take a chi-square of the model, subtract the df associated with the model. Then divide that by the square root of the df - (the number of participants -1):

𝑅𝑀𝑆𝐸𝐴= sqrt( 𝜒2−𝑑𝑓 ) / sqrt(df *(𝑁−1))
if 𝜒2 < 𝑑𝑓→𝑅𝑀𝑆𝐸𝐴 =0

Question 41

How do you interpret the result of a RMSEA?

Answer 41

You can readily interpret the number produced with the following:
• <0.05 good fit
• 0.050.08 poor fit

Question 42

What two identification issues are associated with factor analysis?

Answer 42

Scaling the latent variable: Similar as in the one factor model and in IRT, the scale of the latent variables (factors) needs to be identified

Statistical identification: The number of parameters should not exceed the number of observed (co)variances

Question 43

What is involved in scaling the latent variable?

Answer 43

For EFA this is relatively complex, it is carried out but we don’t need to know exactly how. Most important thing to know is that this constraint results in M^2 restrictions (the number of restrictions you impose on the model to make the latent variable have a scale/ unit.)

Question 44

Why wasn’t statistical identification an issue with the earlier models (IRT)?

Answer 44

Before we were only considering simpler, unidimensional models which hardly have a problem with statistical identification. But now since we’re going to build bigger more complex models, at some point your model is going to become too big for the data e.g using a model with 10 items and 1,000 parameters on a dataset of 50 people would be trying to extract more from your data than you put into the model.

Question 45

What is involved in statistical identification for EFA?

Answer 45

• The number of parameters should not exceed the number of observed (co)variances

• In EFA this can happen if the number of factors in the model is too large
-The observed covariances contain all the information about the factor structure so otherwise the factor structure would be too complex for the information we provide

Question 46

How can you investigate whether a model is identified according to statistical identification in regards to EFA?

Answer 46

A model should always have degrees of freedom larger than or equal to 0

𝑑𝑓 = 𝑀– 𝑘
𝑀:number of independent pieces of observed information
𝑘:number of parameters

Question 47

In this formula for df:
𝑑𝑓 = 𝑀– 𝑘
where
𝑀:number of independent pieces of observed information

What does M mean in terms of an EFA?

Answer 47

The number of observed covariances and variances (since that contains all the information about the factor structure.)

Question 48

How do you calculate M for the df of an EFA if the EFA is conducted on a covariance matrix?

Answer 48

M = p * (p + 1)/2
where p = number of observed variables
E.g.,: 𝑝=4 →𝑀 =4 ∗ 5/2 = 10

This makes sense if you count the covariances and variances of a covariance matrix

Question 49

In this formula for df:
𝑑𝑓 = 𝑀– 𝑘
where
𝑀:number of independent pieces of observed information
𝑘:number of parameters

How do you get the number of parameters?

Answer 49

𝑘 = 𝑝×𝑚 + 𝑚×(𝑚+1)/2 + 𝑝– 𝑚^2 (formula from in exam)
𝑝 is number of variables
𝑚 is number of factors

E.g,: 𝑝=6 and 𝑚=2→ 6∗2 + (2∗3)/2 + 6 − 22 = 𝟏𝟕

𝑝×𝑚 is the number of loadings (relations between each factor and each varaible/item) = 12
𝑚×(𝑚+1)/2 are the factor covariances = 3
𝑝 residual covariances = 6
𝑚^2 covariance constraints = 4

Question 50

What is the role of m^2 in this formula?

𝑘 = 𝑝×𝑚 + 𝑚×(𝑚+1)/2 + 𝑝– 𝑚^2

Answer 50

This is what resulted from scaling the latent variable, these parameters are no longer free parameters. They are fixed in order to identify the scale of the model and so are subtracted from the residual variances.

Question 51

These formula are for models fit on ________

Answer 51

Strictly, these formula are for models fit on a covariance matrices, but you can also use them for correlation
matrices (as for df it does not matter)

Question 52

Raw factor loadings in EFA are hard to interpret. Why is this and what solution is there to this?

Answer 52

Raw factor loadings in EFA are hard to interpret because values are given for each factor and it can be unclear which factor the variable loads on. To fix this, a simple structure version was created in which the values are replaced by + and - to indicate whether a variable loaded on a factor or not through rotation.

Question 53

What is rotation?

Answer 53

A transformation of the raw factor loadings to enhance simple structure

Question 54

What allows us to carry out these rotations on the factor loadings?

Answer 54

The unit of the factors is arbitrary, we chose them! You can therefore transform the results without affecting your model statistically. We can change the scale a little bit after we fit the model just to see when the interpretation is the best.

Question 55

Name two types of rotation and a key feature of both

Answer 55

Orthogonal rotation:
• The factors remain uncorrelated

Oblique rotation:
• The factors are correlated after rotation

Question 56

Name the main R functions associated with each type of rotation

Answer 56

Orthogonal rotation:
- varimax in R

Oblique rotation:
- promax in R

Question 57

In psychology, what type of rotation is typically used and why?

Answer 57

In psychology we can mostly assume factors are correlated, therefore oblique rotation is typically used (promax). Its hard to argue for using uncorrelated factors in psychology

Question 58

Write a line of code to carry out a factor analysis on the correlation matrix ‘osbourne_cor’. Describe what the arguments do

Answer 58

fa(r = osbourne_cor, nfactor = 1, n.obs = 477, fm = 'ml', rotate = 'none')
nfactor = 1 fits a one factor model
fm = 'ml' requests the maximum likelihood
rotate = 'none' requests no rotation (never a good idea)

fa(r = osbourne_cor, nfactor = 2, n.obs = 477, fm = ‘ml’, rotate = ‘varimax’)
Fits a two factor model with orthogonal rotation in R, doesn’t really make sense since it assumes the factors are uncorrelated

you can then tweak the nfactors and observe the RMSEA to see which is the best fit

fa(r = osbourne_cor, nfactor = 2, n.obs = 477, fm = ‘ml’, rotate = ‘promax’)
Fits a two factor model with oblique rotation in R, makes the most sense