Quiz 3

Question 1

Why do we need multilevel models?

Answer 1

If observations are clustered, they are correlated, leading to incorrect standard errors for coefficient estimates. Also, may be interested in the variation at different levels.

Question 2

Assumptions of classical measurement model

Answer 2

Error varies randomly with mean zero
Items’ errors are independent of one another
Errors are not correlated with true score

Question 3

Result of poor reliability

Answer 3

Attenuation of regression coefficient/measure of association

Question 4

Definition of reliability

Answer 4

Ratio of true score variance to observed score variance (where observed score variance is true score variance plus error variance)

Question 5

Parallel test assumptions and consequence

Answer 5

Items have equal correlation with true score
Items have equal means
Items have identical error variance
IF we have parallel tests, their correlation is equal to reliability (even with just two tests)

Question 6

Tau-equivalent tests

Answer 6

Items have equal correlation with true score
Items have equal means
Error variances do not need to be equal

Question 7

Essentially tau-equivalent tests

Answer 7

Items have equal correlation with true score
Items do not need equal means (can add a constant)
Error variances do not need to be equal

Question 8

Congeneric tests

Answer 8

Items do not need equal correlation with true score

Question 9

How to use split half reliability to estimate full scale reliability

Answer 9

To convert to full scale reliability (rather than half), use Spearman-Brown formula, calculating rbar using the split half rxx

Question 10

How to get scale total variance

Answer 10

Sum all entries in scale variance-covariance matrix

Question 11

Cronbach’s alpha

Answer 11

Ratio of communal variance to total variance (sum of off-diagonal elements over sum of all elements), adjusted by (k/(k-1)). Can also be written in terms of average item variances/covariances, or average correlations.

Question 12

When is Cronbach’s alpha only a lower bound for reliability?

Answer 12

Only in congeneric tests

Question 13

Forms of reliability and ways of calculating them

Answer 13

Internal consistency (alpha; KR-20)
Inter-rater reliability (Cohen’s kappa)
Test-retest reliability

Question 14

Major types of validity

Answer 14

Construct (convergent, discriminant, internal structure)
Content (coverage of domain – e.g., expert review, face validity)
Criterion (concurrent; predictive)

Question 15

Messicks Unified Theory of Construct Validity

Answer 15

Consequential
Content
Substantive
Structural
External
Generalizability

Question 16

Reasons for factor analysis

Answer 16

Test theory
Understand scale structure
Scale development

Question 17

What does orthogonal factor rotation do, and not do?

Answer 17

Examples: varimax, quartimax

Assumes independence of factors

Redefines factors so that their loadings tend to be very high or low, to improve interpretation

Does not improve model fit

Question 18

What does oblique factor rotation do, and not do?

Answer 18

Examples: promax, oblimin

Does not assume factor independence

Does not change uniquenesses

In this case, you would also present a structure matrix showing correlations between the variables and factors

Question 19

Difference between factor analysis and PCA

Answer 19

FA tries to explain correlations in observed data while PCA looks for components that are linear combinations of observed data

Question 20

Restrictions we might place on a variance-covariance structure

Answer 20

Sigma_i = Sigma for all individuals (in a balanced design)

Compound symmetry: equal variances and equal covariances

Heterogeneous compound symmetry: variances can differ; consequently structured covariances

Autoregressive: constant variance; declining correlations as time separation increases

Heterogeneous autoregressive: variance also changes with time

Exponential: variability depends on exact time difference between pairs

Question 21

Assumptions for mixed effects model

Answer 21

Individuals/clusters are independent
Errors are uncorrelated with random effects
Random effects are uncorrelated with covariates (which could be violated by time-invariant confounders)

Question 22

Fixed effects model assumptions

Answer 22

Expected value of epsilon is zero for all ij
Expected value of epsilon_i given covariates X_i is zero (meaning current value of y_ij given x_ij should not predict subsequent values/other values in cluster)

Question 23

In terms of variability, when are fixed effects models preferred?

Answer 23

When within-individual variability dominates between-individual variability