Quiz 3 Flashcards
Why do we need multilevel models?
If observations are clustered, they are correlated, leading to incorrect standard errors for coefficient estimates. Also, may be interested in the variation at different levels.
Assumptions of classical measurement model
Error varies randomly with mean zero
Items’ errors are independent of one another
Errors are not correlated with true score
Result of poor reliability
Attenuation of regression coefficient/measure of association
Definition of reliability
Ratio of true score variance to observed score variance (where observed score variance is true score variance plus error variance)
Parallel test assumptions and consequence
Items have equal correlation with true score
Items have equal means
Items have identical error variance
IF we have parallel tests, their correlation is equal to reliability (even with just two tests)
Tau-equivalent tests
Items have equal correlation with true score
Items have equal means
Error variances do not need to be equal
Essentially tau-equivalent tests
Items have equal correlation with true score
Items do not need equal means (can add a constant)
Error variances do not need to be equal
Congeneric tests
Items do not need equal correlation with true score
How to use split half reliability to estimate full scale reliability
To convert to full scale reliability (rather than half), use Spearman-Brown formula, calculating rbar using the split half rxx
How to get scale total variance
Sum all entries in scale variance-covariance matrix
Cronbach’s alpha
Ratio of communal variance to total variance (sum of off-diagonal elements over sum of all elements), adjusted by (k/(k-1)). Can also be written in terms of average item variances/covariances, or average correlations.
When is Cronbach’s alpha only a lower bound for reliability?
Only in congeneric tests
Forms of reliability and ways of calculating them
Internal consistency (alpha; KR-20)
Inter-rater reliability (Cohen’s kappa)
Test-retest reliability
Major types of validity
Construct (convergent, discriminant, internal structure)
Content (coverage of domain – e.g., expert review, face validity)
Criterion (concurrent; predictive)
Messicks Unified Theory of Construct Validity
Consequential Content Substantive Structural External Generalizability
Reasons for factor analysis
Test theory
Understand scale structure
Scale development
What does orthogonal factor rotation do, and not do?
Examples: varimax, quartimax
Assumes independence of factors
Redefines factors so that their loadings tend to be very high or low, to improve interpretation
Does not improve model fit
What does oblique factor rotation do, and not do?
Examples: promax, oblimin
Does not assume factor independence
Does not change uniquenesses
In this case, you would also present a structure matrix showing correlations between the variables and factors
Difference between factor analysis and PCA
FA tries to explain correlations in observed data while PCA looks for components that are linear combinations of observed data
Restrictions we might place on a variance-covariance structure
Sigma_i = Sigma for all individuals (in a balanced design)
Compound symmetry: equal variances and equal covariances
Heterogeneous compound symmetry: variances can differ; consequently structured covariances
Autoregressive: constant variance; declining correlations as time separation increases
Heterogeneous autoregressive: variance also changes with time
Exponential: variability depends on exact time difference between pairs
Assumptions for mixed effects model
Individuals/clusters are independent
Errors are uncorrelated with random effects
Random effects are uncorrelated with covariates (which could be violated by time-invariant confounders)
Fixed effects model assumptions
Expected value of epsilon is zero for all ij
Expected value of epsilon_i given covariates X_i is zero (meaning current value of y_ij given x_ij should not predict subsequent values/other values in cluster)
In terms of variability, when are fixed effects models preferred?
When within-individual variability dominates between-individual variability
