Using a factor analysis Flashcards
What do discrepancies in a factor analysis
When labelling subscales, and selecting which factors load onto subscales if they load onto more than one is highly subjective.
Discrepancies in interpretation highlight the highly subjective nature of factor analysis. Differences in cut-offs, exclusion etc. can result in bias and differences in structure. This emphasises the need to be rigorous and conform to the rules set within the literature.
advantages if a factor analysis?
- Data Reduction
It simplifies large datasets by identifying a smaller number of underlying factors (latent variables) that explain the variance among observed variables.
For example, instead of analyzing 20 individual survey items, factor analysis might identify 3 key dimensions that summarize the data. - Identifying Underlying Constructs
It helps uncover hidden relationships or latent constructs that are not directly observable.
For instance, it can reveal underlying traits like “anxiety” or “worry” from psychological survey items. - Improves Measurement Validity
By grouping variables that measure the same construct, factor analysis helps ensure that scales or tests accurately capture the intended concepts.
This is particularly valuable in psychometric testing and scale development. - Reduces Multicollinearity
Multicollinearity occurs when independent variables in regression analysis are highly correlated. Factor analysis reduces this issue by creating uncorrelated factors, enabling more robust statistical modeling. - Enhances Interpretability
Factor analysis organizes complex data into meaningful categories or components, making it easier to interpret and draw conclusions. - Facilitates Hypothesis Testing
It provides a framework for testing theoretical models about how variables are interrelated, such as in confirmatory factor analysis (CFA). - Supports Decision-Making
By identifying key factors, it helps in making informed decisions about which variables to focus on, whether for research, business, or clinical applications. - Applicable in Various Fields
Factor analysis is versatile and widely used in fields such as psychology, marketing, education, sociology, and finance, to uncover patterns and insights from data
disadvantages of a factor analysis
- Subjectivity in Interpretation
The interpretation of factors can be subjective, as researchers must decide what each factor represents based on the variables it groups.
Different researchers might label the same factor differently, leading to inconsistencies. - Requires Large Sample Sizes
Factor analysis is sensitive to sample size; a large number of observations is typically required for reliable results. Small sample sizes can lead to unstable factors or spurious results. - Assumptions of the Method
Factor analysis relies on several assumptions:
Linear Relationships: It assumes variables are linearly related, which might not always be the case.
Normality: It works best when the data are normally distributed.
Adequate Correlations: If variables are weakly correlated, factor analysis may fail to identify meaningful factors. - Risk of Over- or Under-Extraction
Deciding how many factors to retain can be challenging:
Retaining too few factors may result in oversimplification.
Retaining too many factors can lead to meaningless or redundant results. - Potential for Misleading Results
Factors might be artifacts of the data or sampling method rather than reflecting true underlying constructs.
Results can be influenced by the choice of extraction method (e.g., principal components analysis vs. maximum likelihood) and rotation technique (e.g., varimax vs. oblique). - Does Not Explain Causality
Factor analysis identifies relationships between variables but does not establish causal relationships.
It describes patterns rather than explaining why variables are related. - Requires Theoretical Knowledge
Factor analysis benefits from prior theoretical knowledge about the constructs being measured.
Without a theoretical framework, results can be difficult to interpret and validate. - Sensitive to Outliers and Errors
Outliers or measurement errors can distort factor structure, leading to incorrect conclusions. - Loss of Information
When data are reduced to a smaller number of factors, some detailed information from individual variables is lost. - Limited Generalizability
Results may not generalize well to other datasets if the sample or context changes significantly.- relies on a good generalisable sample- If the sample is too small, a scale developed using factor analysis might not perform consistently in broader populations
Goretzko 2021- exploratory factor analysis
Often interpretability and theoretical considerations can be equally important. In particular, for test construction purposes content validity should be first priority.
Researchers therefore should report transparently which objectives they have, which methodological decisions they take and all outcomes they collect. This ensures that the quality of a solution can be evaluated and implications of particular studies can be weighted.
- samples should be greater than 400 to get reliable factor patterns
- ALso said Since factor analysis solutions can vary, researchers need to clearly report the chosen methods. This ensures others can replicate the analysis and verify the results.
- when you conduct Exploratory Factor Analysis (EFA), you shouldn’t only rely on statistical techniques to decide which factors to retain. Instead, you should also think about whether the identified factors make sense within the context of the theory or domain you’re studying
issues with cut offs in factor analysis
The cut-off criteria in factor analysis refer to the thresholds researchers use to decide which factor loadings, eigenvalues, or other measures are considered significant. While these thresholds are necessary for interpretability, they come with challenges and potential issues:
- Difficulty with Cross-Loading Items
Items that load significantly on more than one factor can be problematic. Cut-off criteria may lead to these items being excluded, even if they provide valuable multidimensional insights. - Arbitrary Nature of Cut-Off Values
Many cut-off criteria, such as factor loadings of 0.30 or eigenvalues above 1.0 (Kaiser’s criterion), are often used without strong justification. These thresholds might not always be appropriate for the data and context.
Example: A factor loading cut-off of 0.40 might discard meaningful items in exploratory research, while retaining noise in confirmatory settings. - Sample Size Dependency
Cut-off criteria like factor loadings and significance tests are influenced by sample size. Small samples can lead to unstable loadings and inflated p-values, while large samples may make almost any loading significant, regardless of practical importance.
Problem: Results may not generalize well to other samples.
you SHOULD= Use Multiple Criteria: Combine eigenvalue thresholds, scree plot analysis, and theoretical considerations to decide on factors and loadings.
-Theoretical Justification: Consider theory and context, not just statistical thresholds, when retaining or removing items and factors
-Report Transparently: Clearly report and justify chosen cut-off criteria to enhance replicability
Example of a good factor analysis to develop a good questionaire
Development of the Penn state worry questionnaire-
Meyer et al., 1990=
Development: The PSWQ is a 16-item instrument created using factor analysis of a larger item pool. It has strong internal consistency and test-retest reliability, making it a reliable measure.
Validity:
The questionnaire correlates well with psychological measures related to worry but does not correlate with unrelated constructs.
It is unaffected by social desirability bias, meaning responses are less likely to be influenced by the desire to appear favorable.
Clinical Discrimination:
The PSWQ effectively distinguishes between individuals with varying levels of worry and generalized anxiety disorder (GAD).
It differentiates GAD from other conditions like posttraumatic stress disorder (PTSD).
Uniqueness:
Among highly anxious individuals, the PSWQ does not correlate strongly with other anxiety or depression measures. This suggests it measures a distinct construct related to worry rather than general anxiety or depression
bad example:
an example where there is no fit; confirmitory factor anaysis-Grove et al., 2019
This study highlights some limitations of factor analysis, particularly when applying it to real-world data. Despite its widespread use, factor analysis here shows that the Social Communication Questionnaire (SCQ) does not adequately align with the DSM-5 diagnostic criteria for autism. The factor models tested, such as the unidimensional model and the DSM-5 based model, failed to fit the data well, suggesting that the tool’s structure may not capture the complexities of autism as defined in the updated diagnostic criteria. Furthermore, issues with item variance and the dichotomous scoring method reduced the sensitivity of the SCQ, limiting its ability to measure underlying constructs effectively. This exemplifies how factor analysis can sometimes lead to poor model fit, especially when using standardized tools that may not adapt to changes in theoretical frameworks or when items don’t align well with expected factors (Brown, 2015; Fabrigar & Wegener, 2012). These problems can reduce the tool’s validity and reliability, raising concerns about the effectiveness of factor analysis in certain applied settings.
another bad example
- Briggs and Myers 1943=
Myers brigg test- personality questionaire, including for example introverted and extroverted. (MBTI)
Although 3.5M administered every year it has poor internal consistency, validity, and test retest reliability. AND some argue it is not comprehensive and misses key aspects of personality!
