Factor Analysis

Question 1

What is factor analysis?

Answer 1

A class of multivariate statistical methods for analyzing the interrelationships among a set of variables and for explaining these interrelationships in terms of a reduced number of variables (called factors).

Question 2

What is the difference between exploratory and confirmatory factor analysis?

Answer 2

In EFA, the researcher has no specific expectations regarding the nature of the underlying constructs:
- We carry out EFA to uncover the factor(s) that underlies the relationship of a set of variables.

In CFA, a factor structure is explicitly hypothesized.
- We seek to test the degree to which the data meet the hypothesized structure.

CFA differs from EFA in that it employs different method of estimation and use different set of criteria for evaluating the adequacy of the factor solutions.

Question 3

What is Principal Component Analysis (PCA)?

Answer 3

The goal is to identify a new set of variables called principal components, with the first few components accounting for most of the variance of the variables.
Each component is a weighted sum of the original set of variables.

Concerned with total variance. Makes no distinction between common and unique variance.

Used when the objective is data reduction (i.e., it summarizes the data by finding the min number of components needed to account for the maximum portion of the variance represented in the original set of variables).

Question 4

What is EFA (Common FA)?

Answer 4

Used to identify interpretable constructs that explain the correlations among the variables as well as possible.

Define factor(s) that arise only from the common variance component of the variables.

Thus factors are estimates of hypothetical, error-free underlying latent construct. This is because they are extracted with the unique variance (variance not shared with other items) removed from each item.

Uses the communality coefficients to replace the 1s on the diagonal of the correlation matrix for its analysis.

Question 5

What is the difference in conceptual meaning of
component (In PCA) and factor (in EFA)?

Answer 5

Components are defined by how items are answered
* Components are end products (effect) of the items in the sense that actual scores obtained on items determine the nature of the components.

Factors determine how items are answered
* Each variable in a set of measured variables is a linear function of one or more common factors and one unique factor.

Factors explain a certain proportion of the shared variance, while component explains a certain proportion of the total variance.

Question 6

PCA versus common factor analysis
Why is there a debate over which approach is more appropriate or superior?

Answer 6

PCA is argued to be superior because:
* Computationally simpler
* Not susceptible to improper solutions
* Ability to calculate a respondent’s score on a principal component

Others argued that common factor analysis is more appropriate if:
* The objective is to reproduce the intercorrelations among a set of variables with a smaller number of latent dimensions that recognizes measurement error in observed variables.
* Its estimates are more likely to generalize to CFA

In most applications, both PCA and common factor analysis arrive at essentially identical results
* if the number of variables exceed 30, or
* if the communalities exceed .60 for most variables.

Question 7

What are factor loadings?

Answer 7

Correlation of each variable and the factor.

Question 8

What are communalities (h^2)?

Answer 8

The proportion of variance in an observed variable that is accounted for by the set of common factors
* Communality for each variable is computed by summing the squared factor loadings across all factors.
* Large communalities indicate that a large amount of the variance in a variable has been extracted by the factor solution.
* An item with communality of below .4 suggests that it does not correlate highly with one or more of the factors in the solution, hence problematic (Worthington & Whitaker, 2006, p.823)

Question 9

What are eigenvalues?

Answer 9

The amount of variance (summed across variables) that is explained by a component/factor
* Eigenvalue for each factor is computed by summing the squared factor loadings over all variables.
* Indicates the relative importance of each factor in accounting for the variance associated with the set of variables being analyzed.

Question 10

Why are the communalities under “Initial” for PCA different from those for PAF?

Answer 10

PCA uses the unreduced correlation matrix for analysis (i.e., does not substitute the diagonal with communality estimate)

Question 11

Factor selection

What is Kaiser’s eigenvalue “greater-than-one” criterion? What is the logic behind it?

Answer 11

Retain only component/factor whose eigenvalue is greater than one.

Given that the maximum amount of variance that one item/variable can explain is 1, a factor must therefore account for at least as much variance as can be accounted for by a single item (hence, eigenvalue of a factor > 1 A factor with eigenvalue < 1 is not worth keeping because it accounts for less variance than a variable.
Note: This criterion should be applied to the unreduced correlation matrix, not to the reduced correlation matrix.

However, it is a poor criterion that is used by convention.

Question 12

Factor selection

What is a scree plot and how is it used?

Answer 12

A plot of eigenvalues against the number of factors
- Decision rule: the factors whose eigenvalues are in the steep decline are retained, while those whose eigenvalues are in the gradual descent (including the eigenvalue occurring in the transition from steep to gradual descent) are dropped
- Scree plot easily produced in stats programs which used the eigenvalues in the unreduced correlation matrix.

This criterion results in an accurate determination of the number of factors most of the time.
But problem is that interpretation is subjective and the pattern may be ambiguous with no clear substantial drop present.

Question 13

Factor selection
What is parallel analysis (Horn, 1965) and how is it used?

Answer 13

This criterion is based on comparing the eigenvalues obtained from sample (real) data to eigenvalues obtained from a completely random data (with the same number of respondents and variables).
Logic is that the eigenvalues obtained from random data are due to chance variation in the random data. Any useful components or factors found in the real data should therefore account for more variance than could be expected by chance for it to be retained.
Decision rule is therefore to retain those factors whose eigenvalues are greater than the corresponding ones based on random data.

Parallel analysis is shown to be one of the most accurate factor retention methods.
Problems:
Similar to scree test, chance variation in the input correlation matrix may cause eigenvalues to fall just above or below the criterion.
Potential for PA to underfactor, especially when sample size is small, when factors are highly correlated, or when the second factor is based on a small number of variables/items.

Question 14

What are some guidelines for factor selection?

Answer 14

Use a combination of different criteria to help determine factor selection. In situations in which procedures suggest different number of factors to be extracted, researcher should examine all possibilities by obtaining the rotated solution for each case to see which produces the most readily interpretable and theoretically sensible pattern of results.

The objective is to select the number of factors that explains the data substantially without sacrificing parsimony.

One should not fall back on statistical criterion alone. Instead, substantive and practical considerations should strongly guide the factor analytic process.

Question 15

What is factor rotation?

Answer 15

Mathematically, factor rotation is a transformation that is performed to foster interpretability by aiming for a simple structure (Thurstone, 1947) where:
1. each factor is defined by a subset of variables that load highly on the factor; and
2. each variable (ideally) has a high loading on only one factor and a trivial loading (< ± 0.25) on the remaining factors.
Conceptually, factor rotation is akin to changing a vantage point to examine the relationship (as explained by DeVellis).

Question 16

Factor rotation
What is orthogonal rotation?

Answer 16

During rotation, the reference axes of the factors are turned about the origin.

In orthogonal rotation, the factors are constrained to be uncorrelated (i.e., factors maintained at 90° angles in multidimensional space).

Question 17

Factor rotation: Orthogonal rotation methods
What is VARIMAX?

Answer 17

centers on simplifying the columns of the factor matrix

it aims to rotate in such a way that the factor loadings on any one factor are as high or as low as possible.

tends to give the clearest separation of the factors among all the orthogonal methods.

Problem: it may give rise to factor invariance.

Question 18

Factor rotation: Orthogonal rotation methods
What is QUARTIMAX?

Answer 18

focus on simplifying the rows of the factor matrix.

aims to rotate the factor so that a variable loads high on one factor and as low as possible on all other factors.

Not very successful at producing simple structure because it tends to create a general factor as the first factor on which most of the variables have high loading.

Question 19

Factor rotation
What is oblique rotation?

Answer 19

In oblique rotation, factors are allowed to correlate, thus the reference axes are not constrained to maintain at 90°.

Oblique rotation will yield information about the correlation of the factors.

Question 20

Factor rotation: Oblique rotational methods

For oblique rotation, what are the two factor matrices of loadings produced?

Answer 20

Typically considered methods are Promax and Oblimin. Promax is almost always a good choice (Thompson, 2004).

• Factor pattern matrix: represents the relationship between a variable and the factor while controlling the influence of all other factors.
• Factor structure matrix: represents the unique relationship between the variable and factor (in the pattern matrix) PLUS the relationship between the variable and the shared variance among the factors.

Factor pattern matrix is preferred as it does not have an inflated correlation between factor and variable.

Question 21

Factor rotation
What are the arguments for orthogonal and oblique rotation?

Answer 21

Orthogonal rotation is more commonly employed because:
* The solution is more easily interpreted: factor loadings represent correlations between the variables and the latent factors (whereas this is not the case in oblique rotation).

Oblique rotation is favored because:
* If factors are correlated, it will yield a more accurate and realistic representation of the relationship between factors.
* If factors are in fact orthogonal, it will produce a solution that is virtually the same as one produced by orthogonal solution, thus nothing is lost.

Question 22

What is not altered by factor rotation?

Answer 22

Although factor loadings change during rotation and the variance is re-distributed within the solution, note that these are not altered by rotation:
* The communality coefficient for each of the variables;
* The total variance reproduced in the factors as a set.

Question 23

Factor interpretation: What is considered high loading?

Answer 23

Typically, factor loadings need to be:
* greater than or equal to .30 to .40 to be considered salient. (Brown, 2006; Hair, Anderson, Tatham & Black, 2009).
* >± .50 to be considered practically significant.

Some researchers (e.g., Kline, 1994) employed a more relaxed criterion of .30 while others (e.g., Stevens, 1996) preferred a more stringent criterion of minimal .40 loading on one factor and not more than .3 on all other factors.

However, criterion for classifying a variable as a constituent of a factor should be based on the value of correlation that is needed to achieve statistical significance at .05 level.

Question 24

What are the popular criterion used for item retention?

Answer 24

Prof’s preferred criterion:
* Items should preferably load at least .4 on the relevant factor and load no higher than .3 on all other factors (Stevens, 1996).

Others in the literature:
* Delete items with factor loading less than .32 or cross-loading less than .15 difference from an item’s highest factor loading (Worthington & Whittaker, 2006).
* Tabachnick & Fidell (2001) also cited .32 as a good rule of thumb for the minimum loading of an item, an suggested that a cross-loading item is an item that loads at .32 or higher on two or more factors (this is also supported by Costello & Osborne, 2005).

Question 25

What are some guidelines for factor interpretation?

Answer 25

1. Start with the 1st variable on the 1st factor, go horizontally from left to right to underline loadings that are considered practically and statistically significant.
2. Ideal is for each variable to load highly on one factor only. Variable with several equally high loadings will be considered for elimination, especially if the communality of the variable is < .5, indicating that its variance is not sufficiently accounted by the factor solution.
3. Try a different rotational method if no simple structure is achieved.
4. Process ought to be guided by substantive consideration and knowledge of the variables.