Lecture 4 Flashcards
What is the key difference between principal components and factor analysis?
- PCA: finds optimal linear transformations
- FA: assumes latent factors that are not directly oberved
- there is no model in PCA, but there is a model (can test fit) in EFA
- PCA is simply a weighted sum of variables
How does PCA work?
- graphically, finds new axes for your data
- new components are chosen one by one, to maximise variance not yet accounted for
How many components can you make with N variables?
N components.
BUT if you use less than N, then there are a smaller no. of components, then there is freedom in the final solution
- also if you use less than N, you can rotate to get a simplet solution
Why is PCA simple?
- they are not correlated, even if the original variables are
- first component explains the most variance > thus you know which components are the most important
How do you determine how many components/factors to extract?
- SPSS default is no. of eigenvalues > 1 (DO NOT USE), called Kaiser-Guttman
- use Screen plot (where it turns)
- use parallel
- use MAP
Explain the parallel test
- uses random data (with same dimensions as your dataset) as a baseline
- if eigenvalue is higher than random (noise) data, then it must be signal
- where “raw data”
- the “pcntile” is the 95th percentile
Explain the MAP test
- plots squared partial correlations and gets MINIMUM
- as more components are extacted, more are partialled out of correlation matrix, SPCs approach 0
- but then at some point ‘noise’ components get partialled out, and the SPCs increase again
- therefore, want the minimum
What does a -ve or high component/factor loading mean?
- negative: you get a high score on that item, you get a low score on the component/factor
- negative loading similar to reverse scoring
- high: higher score on that item, higher score on factor/component
Why rotate components?
- simpler structure
- easier to interpret
What are the 2 types of rotation?
- orthogonal: remain uncorrelated
- oblique: correlated
What are the specific SPSS rotations?
- orthogonal: varimax, equamax, quartimax
- oblique: direct oblimin, promax
What do you interpret after rotation?
- oblique: pattern matrix
- factor correlations
(structure matrix = product of pattern and factor correlation matrix) - orthogonal: rotated
What does EFA assume?
that there are some underlying latent factor that cannot be directly observed > searches for these
What is ui? What is k?
- u: the specific factor (noise/error)
- k: the common factor
What are the assumptions of EFA?
- common factors standardised (variance = 1)
- common factors uncorrelated
- specific factors uncorrelated
- common factors uncorrelated with specific factors
- multivariate normality
What is the underlying rationale of EFA?
- partial correlations
- correlation b/w item 1 and item 2, WHEN HOLDING CONSTANT a latent variable is…
- if PC is 0, then correlation b/w the items is fully explained by the factor > want it as close to 0 as possible
- aim to find a latent variable that accounts for observed correlation (i.e. make it as close to 0 as possible)
- if we can find these correlations/mimic the covariance matrix, then we have found the latent factors
What is the communality?
- the variance due to the common factors
- want HIGH communalities
What are the rules/guidelines about sample size for EFA? What is the problem will small sample size?
- 150+
- absolute sample size + communalities are more important
- ratio > variables:sample size NOT important
- if loadings are high, then you can have a lower sample size
- less generalisable if too small
What are the 3 things you want for EFA?
- high communalities (>.8 ideal, but reality is .4-.7) > can drop things if they have low communality (but be careful)
- few cross-loadings (>.032)
- more than 3 strongly loaded items per factor
^^^ need a larger sample if these are not met
What is the issue with high communalities? How do you fix this?
- you only know them after you find the factor loadings
- so… use prior diagnostics!!
What are the prior diagnostics?
- correlations (low - low loadings)
- Bartlett: want >.05 (usually always is)
- anti-image: diagonal (MSA) close to 1, off-diagonal (anti-image correlations) close to 0
- Kaiser: want high, >.9 great
Why is ML good?
has a goodness of fit test
What is the issue with chi-square?
- very sensitive test!! (wan >.05)
- use RMSEA instead. Want less than 0.06
What are Heywood cases? How do you find and fix them?
- technical problems
- values of .999
- look for in un-rotated factor matrix
- problem > maybe too many factors extracted
- increase interations from 25 to 250
What are the 3 ways of estimating factor scores using congeneric tests?
- regression model
- Bartlett (probs best)
- Anderson-Rubin
AR assumes uncorrelated, so don’t use with oblique solutions
What is an eigenvalue?
- the variance of the first component extracted (variance of Y1)
- derived from correlation matrix of variables (NOT covariance, variables are standardised before analysis)
Is PCA statistics? Why is this good?
No, just a mathematical technique
- no error terms, no prediction
- there is no model!
- this is why there are no assumptions or requirements, it always works
Is PCA a type of EFA?
Nope
What is plotted in a scree plot?
eigenvalues vs. components
What do you do if parallel and MAP tests disagree?
make a decision!
can cite someone who says one is better
or choose the more interpretable one
How can you write out the component loadings to equal the component? In matrix form?
- Y1 (component 1) = loadingXitem + loadingXitem….. etc.
- Y = aX
Why should you only use oblique rotation?
- more realistic
- more statistically sound
Which 2 factor methods are recommended by Schmitt?
- maximum likelihood
- principal axis factoring
How do you calculate RMSEA?
square root: (X2 - df) / ( (N-1)df)
if df > X2, then treat as zero! (amazing fit)
What are the 4 key components of factor scores created by sum?
- assumes equal weight of each item (tau-equivalent)
- underpins test theory for reliability
- basis for coefficient alpha reliability
- if not true: alpha a serious underestimate
What are congeneric tests?
assumption of varying factor loadings
What happens is a factor/component or an item is added in PCA vs. EFA?
- PCA: adding item may change component; adding component will not change loadings
- EFA: adding item should not change others; adding factor will change factor loadings
Why are there differences in PCA vs. EFA in terms of adding/removing items/factors? What does each method aim to do?
Issues with diagonal elements of correlation matrix
- PCA: value of 1 used
- aim to explain all variance of variable
- reproduce whole variance-covariance matrix
- EFA: diagonal is the communality
- aim to explain only common variance of an element
- reproduce only off-diagonal parts of variance-covariance matrix
Widaman’s conclusion
- rarely, or never, do a component analysis of empirical data if your goal is to interpret patterns of observed covariations among variables as arising from latent variables or factors
What do PCA and EFA actually do?
- use associations among variables to condense into a smaller, simpler number of variables
What is the trade-off in PCA? What do we do to help this?
- trade-off b/w getting a simpler structure (less components) and explaining a higher proporiton of variance
- scree, MAP and parallel help us decide this trade-off?
What do quartimax, varimax and equamax actually do? And oblimin and promax?
- quartimax: simplifies variable pattern of loadings
- varimax: simplifies the factor patterns of loading
- equamax: compromise of above 2
- oblimin: change delta -0.8 to 0.8
- promax: change kappa, from 1 upwards
What is important for factor in when you are deciding what to call your factors/component?
- the direction (+ve or -ve) of items
What are the anti-image and image correlations? What is image analysis?
- image: correlations due to common parts
- anti-image: correlations due to unique parts
- image analysis: partitioning variance of observed variable into common and unique parts
Why would you ever use PCA instead of EFA?
- historically, PCA was simpler and faster
- PCA can be a fallback if you have a smaller sample or other technical issues that mean you cannot do EFA
Do PCA and EFA have similar outputs?
- yes
- but not for all datasets
- Widaman: only if there are high communalities
What is the common factor model?
- k common factors that explain observations on the variables
- Xi = (labmda)i1F1 + (labmda)i2F2 … +(labmda)ikFk + ui
- u is the specific factor
How do you calculate the variance of observed variable X?
- sum of factor loadings^2 (common variance) and variance of u (unique variance)
- common variance = communality! (denoted h2)
- covariance = multiply factor loadings
What happens when you sum the squared coefficients (loadings) in PCA?
it always equals 1
- because this is the communality and PCA assumes communalities of 1!!
What is the key difference between PCA/EFA and cluster analysis/MDS?
- FA uses correlations as associations
- cluster and MDS use proximities (distances)
What are the variables and components/factors in PCA v. EFA?
- PCA: components are just weighted sums, so are also observed variables
- EFA: factors are superordinate to observed variables (cause correlations b/w variables)
