Factor Analysis

Question 1

Overview

Answer 1

• What is factor analysis?
• Factor loadings, eigenvalues and communalities
• Extracting/ choosing factors
• Interpreting factor loadings
• Rotation of factors
• Sequence of operations to conduct a factor analysis
• Reporting a factor analysis

Question 2

Learning objectives

Answer 2

• Understand and be able to explain… the main aim of factor analysis.
• Understand and be able to explain… factors loadings, Eigen values and communalities.
• Understand and be able to explain… the different criteria for extracting factors.
• Understand and be able to explain… when and why to rotate factors.
• Be able to… conduct, report and interpret a factor analysis.

Question 3

What is factor analysis?

Answer 3

The overall aim of factor analysis is to analyse patterns of correlations between variables (items) in order to reduce these variables to a smaller set of underlying constructs called “factors” or “components”

The factors are informative in their own right, but also provide a new set of scores which might be employed in another multivariate analysis such as multiple regression

Can distinguish between exploratory and confirmatory factor analysis

Question 4

Items and factors

Answer 4

We might have a series of items that people respond to, and we’re saying that variation is caused by some underlying construct/ latent variable

There can be several factors influencing variation in the different item responses

Instead of analysing all 14 items we condense it down to 2 factors (each representing a psychological construct) - makes additional analysis better

Question 5

Different types of factor analysis

Answer 5

When looking at exploring factor analysis, ie. exploring the data to see how many underlying factors there are, there are several kinds of analysis.

We are looking at exploratory factor analysis, within that the method of Principle Components Analysis (PCA)

Question 6

How do we analyse patterns of correlations?

Answer 6

Considering a scale with 14 items… this results in 91 correlations to examine (30 = 423)

If several of the correlations are >.3 then this suggests that there are a smaller number of underlying factors than 14 (or 30) different constructs

So…

• first step is to look at correlations
• looking for patterns of correlations between the items
• when you have lots of items, this cannot be done manually
• if several of the correlations between items are above .3 then this is taken to suggest that there are a smaller number of underlying factors/ constructs than the 14 represented in the items

Question 7

Exploratory factor analysis (EFA)

Answer 7

Typically used to identify a smaller number of underlying factors (components) when analysing a large number of items within a scale

For example, with a 14 item scale different factors for depression and anxiety might emerge

Question 8

Ideas underlying the PCA (principle components analysis) form of EFA (exploratory factor analysis)

1. Components are linear combinations of variables

Answer 8

A “component” is a linear combination of the variables - something that effects how “as one variable changes, so does the other”

The aim is to construct a linear combination (V) of each participant’s scores on the variables (items) with the coefficients (a1 etc) chosen so as to maximise the proportion of total variance accounted for by this factor (component).

For example, for 3 variables/item scores (Y1, Y2, Y3)…

A component score (V), for each participant is obtained from the sum of all his/her scores, where each score is multiplied by a different coefficient (all participants’ scores are multiplied by the same set of coefficients):

			V = a1 Y1 + a2 Y2 + a3 Y3  So... 

PCA is trying to create an equation (including scores and coefficients) to explain the max variance accounted for by the factor

Higher scores = more effect of any possible factor, higher correlation = relates to other items

Question 9

Ideas underlying the PCA (principle components analysis) form of EFA (exploratory factor analysis)

2. More than one component is possible

Answer 9

More than one component is possible. The first component employs coefficients to account for the maximum amount of the total variance:

 	V = a11 Y1 + a21 Y2 + a31 Y3 

A second component has the same aim, but is constrained to be uncorrelated with the first. There is a second set of coefficients, and so on:

 	V = a11 Y1 + a21 Y2 + a31 Y3 

For each component, the coefficients are chosen to account for the maximum amount of variance remaining.

In theory, the total number of components = number of items.

The more all the original variables are correlated together, the more the total variance will be accounted for by the first component.

Rewording:

• One component will leave some variance in scores unexplained so we create another component (with another set of coefficients) that seeks to explain what’s left of the variance in the item scores
• you can create components up to the number of items, then all variance is explained, but we want as few as possible
• when the original variables (items) are highly correlated together, then more variance is accounted for by the first component

Question 10

Ideas underlying the PCA (principle components analysis) form of EFA (exploratory factor analysis)

Loadings, Eigenvalues and Communalities

Answer 10

1. Factor loading
   - each loading is the correlation between a variable (item) and a factor
   - tells you how much each item correlates with a factor/component
2. Eigenvalues
   - ∑L2
   - tells you the amount of variance accounted for by a factor
3. Communality
   - ∑L2
   - sum of squares factor loadings for all factors in a variable (added up the correlations between the variable and the factor) for one item

SEE DIAGRAM PAGE 3

Question 11

Factor loadings

Answer 11

Each loading is the correlation between a variable (item) and a factor
- tells you how much each item correlates with a factor/component

Note: loading2 = proportion of variance in a given variable accounted for by a factor (e.g. .322 = 0.1 or 10%.)

Absolute loadings > .30 (or 0.32) are called ‘salient’ and interpreted

Absolute loadings < .30 are dismissed, & sometimes written as zero

We don’t ask, “Is the correlation (i.e. the loading) significant?” However, loadings of .70 and .55 are deemed ‘excellent’ and ‘good’, respectively, accounting for 50% and 30% of the variable’s variance.

So… so the more an item and a factor are correlated, the bigger the loading.

L2 gives the proportion of the variance in a given variable/ item accounted for by a factor

Question 12

Eigenvalues

Answer 12

Each eigenvalue = the sum of the squared loadings within a factor/ component down the whole set of variables/ items

Each eigenvalue is the amount of variance in the set of variables/ items accounted for by a particular factor

Each eigenvalue = the variance of the V (linear combination) values for that factor

Eigenvalues range form 0 to the total number of items
An eignenvalue > 1.00 suggests that this factor should be selected - i.e., it is a “principal component”

The percentage of the total variance accounted for by one (or more) factors is given by:

P = sum of selected eigenvalues x 100
___________________________
number of items

∑L2 = eigenvalue, sum of squared loadings within a factor for all the items. It’s the amount of variance in the whole set of items accounted for by a factor. Ranges from 0-no. of items, want it to be bigger than 1, so we have less components than items (smaller number of underlying factors)

Question 13

Communalities

Answer 13

Each communality = sum of the squared loadings within a variable, across the selected factors

Each communality is the proportion of variance in an observed variable accounted for by the selected factors

If as many factors are selected as there are variables, each communality will = 1.00 in PCA

A communality < .30 suggests that the variable is unreliable and should be removed (as the factors account for less than 30% of its variance)

Table of communalities:

• Based on how many factors have been extracted, so its saying that the extracted factors (e.g. 2) are accounting for so much variance in an item e.g. .6/ 60%
• If all items are above .3 the 2 factors are doing a decent job of explain the variance in each individual item

Question 14

How many factors or components should be “extracted”?

Answer 14

We use the eigenvalues (amount of variance in the set of variables/ items accounted for by a particular factor).

Can then use:

• Kaiser’s criterion
• Cattel’s scree test

Question 15

Deciding how many factors to extract

Kaiser’s Criterion

Answer 15

SPSS does this automatically, orders factors in order of what accounts for most variance - extract anything over 1

Question 16

Deciding how many factors to extract

Cattell’s Scree Test

Answer 16

This is where you plot the eigenvalue against each factor. Looking to find the factors with eigenvalues over 1, ones that sit above the “debris” on the figure.

Usually, kaisers criteria is consistent with the results of a scree plot.

Question 17

Plotting loadings on factor axis

Why and how

Answer 17

We can plot factor loadings against the factors, to see what extent an item loads on factor 1 vs factor 2 etc - which constructs are the items measuring?

The correlation between factor 1 and a variable (the loading of the variable on factor 1) is represented by the distance along the factor 1 axis. Similarly for the loading onto factor 2.

Question 18

Plotting loadings on factor axis

Structures of the plots

Answer 18

No structure - (clustered in the centre) the items are loading very highly on either factor

One factor - (clustered around the top of the line on one factor) the items are loading highly on one factor

One factor; bipolar - (2 clusters, one round the top of the line, and one around the bottom of the line on the SAME factor) the items are split, loading highly and not very much on the same factor. Items may need to be reverse coded here

Two factors - (2 clusters, one round the top of the line on each factor) the items are split, some loading highly on one factor, some on another. Simple structure - this is what we want when looking at ou results.

Ideally, we want each item to load highly onto just one factor. When you get items that load highly on more than one factor it creates a complex factor structure, described as cross-loading

Page 7

Question 19

Plotting loadings on factor axis

Rotation

Answer 19

Sometimes you get items that load onto more than one factor simultaneously.

To get around this problem, we rotate the axis of the graph.

Orthogonal rotation

Orthogonal - involving right angles, we simply twist the axis round like a dial, maintaining the right angles. So the two groups sit on the lines.

Rotation redistributes the variance amongst the factors, instead of the first factor accounting for most of the variance (despite there being 2 groupings), now, both factors are explaining variance more evenly - so factor loadings can be more easily interpreted.

Orthogonal rotation keeps the axes at right angles. The Varimax method maximizes the variance of the loadings within each factor, making high loadings higher and low loadings lower.

Oblique rotation

Again, 2 clear groups arent sitting neatly on the axis. If we rotated the axis orthongonally, we still wouldn’t be able to get the groups to sit better on the axis, so we rotate the axis independently of each other.

In oblique rotation the factors are not orthogonal - they are correlated.

Look at the matrix of correlations between factors after oblique rotation. If the correlations exceed .32 there is a 10% overlap in variance among factors, enough to justify the oblique rotation (40.08 encore)

Question 20

Sequence of operations to conduct an EFA

List

Answer 20

1. Compute a matrix of correlations
2. Extract/choose factors based on Kaiser’s criterion (Eigenvalues > 1) and/or Cattell’s scree test
3. Examine factor loadings (loadings and communalities)
4. Is the factor structure simple or complex?
5. Do the factors need rotating (orthogonal or oblique)?
6. Check the factor loadings again.
7. Repeat steps are needed.

Question 21

Sequence of operations to conduct an EFA

Recap in more detail

Answer 21

Example:

1. Compute a matrix of correlations
   - here we want the correlations to be above .3, to suggest there are fewer underlying factors/ constructs affecting variance rather than all the items individually
2. Extract/choose factors based on Kaiser’s criterion (Eigenvalues > 1) and/or Cattell’s scree test
   - kaisers (eigen values above one?)
   - scree (above the debris?)
3. Examine factor loadings (loadings and communalities)
   - so, all variables are loading highly on component 1, some loading highly on 2 and some low on 2. Suggests some roation needed, complex factor structure.
   - commonalities suggest that a high % of variance is being explained by the factors (in each item)
4. Is the factor structure simple or complex?
   - can rotate using orthagonal or oblique?
5. Do the factors need rotating (orthogonal or oblique)?
   - axis can be orthogonally rotated, kept at right angles (varimax)
   - or, cant fit using right angles (use direct oblimin method)
6. Check the factor loadings again.
   - gives much clearer factor structure, two groups, simple structure
7. Repeat steps are needed.

Question 22

Reporting a Factor Analysis

Example

Answer 22

Look at the printed sheet

Report:
An exploratory factor analysis was conducted to examine the factor structure of a measure. Correlations were first computed between the no. items from the measure (see table 1). Inspection of the correlations revealed e.g. moderately strong correlations between e.g. many of the items.

Table of correlations

The no. items were then subjected to a principle components factors analysis. Two factors were then extracted with eigenvalues greater than 1.00. Factor 1 explained % of the variance in item scores (Eigenvalue = **) and Factor 2 explained % of the variance in item scores (Eigenvalue = **). Inspection of the scree plot of Eigenvalues by components confirmed the extraction of two factors (see Figure 1).

Inspection of the communalties revealed that the two factors accounted for sufficient amounts of the variance in all items (i.e., all communalities were > .30), indicating that all items were reliable (see Table 2).

** Example… Inspection of the factor loadings revealed that all items loaded e.g. strongly (above .40) on both factors (see Table 3). The factor loadings were therefore plotted which indicated that the factors should be subjected to an orthogonal rotation (see Figure 2). The factor loadings from the orthogonal rotation revealed that items 2, 6 and 10 loaded strongly only on Factor 1 whereas items 1, 3 and 11 loaded strongly only on Factor 2 (see Table 3). The rotated factor loadings were also plotted, which confirmed the clear two factor structure (see Figure 2). Inspection of the content of the items indicated that the three items that loaded strongly on Factor 1 assessed symptoms of depression whereas the three items that loaded strongly on Factor 2 assessed symptoms of anxiety.

scree plot

communalities table

factor loadings table

factor axis plots

Question 23

Reporting a Factor Analysis

Bones

Answer 23

• test used… to examine factor structure of…
• correlations computed between items… revealed

correlations table

• items then subject to PCFA.
• how many factors extracted and why
• how much variance did the factors explain, what were their eigenvalues?
• scree plot confirmed extraction?
• inspection of communalities revealed…
• inspection of factor loadings revealed… loaded strongly? on?
• plotted which indicated… what kind of rotation?
• factor loadings from rotation revealed which items loaded onto what?
• rotated were plotted, confirmed? simple structure?
• what did the two factors represent (the psychological constructs they measured)