Lecture 2 Flashcards
2 core assumptions of classical test theory
- Measurement error is random
- Observed scores are true scores plus measurement error
Correlation between true score and measurement error
0
How to estimate reliability when looking at variance
High reliability means that most of the variance is true score variance as opposed to error variance.
Parallel test model
Most of the scores and variances are the same.
Split-halves and test re-test are based on this.
Tau-equivalent model
Variance of observed score is not equal.
Correlation of test 1 and 2 not the same.
Essentially tau-equivalent model
Mean of true and observed scores different.
Cronbach alpha based on this.
Congeneric model
Variance of true scores are different.
Omega based on this.
3 methods of reliability estimation
- Alternate forms
- Internal consistency
- Test-retest
Alternate forms
- Correlation is reliability
- Assumes parallel test model
- Apply two versions of the same test.
- Carry over effects
Test-retest
- Apply the same test twice
- Assumes parallel test model
- Correlation is reliability
- Carry over effects
- Change in true scores
Internal consistency
- Assumes parallel test or essentially tau-equivalent model
- Split-half
- Cronbach’s alpha
- Carry over effects
- Lower bound to reliability
Factors affecting reliability
- Test length (longer test is higher reliability)
- Sample heterogeneity (more heterogenous is higher reliability)
- Correlation between pre and post test scores (high correlation means lower reliability)
Effect size of observed scores in unreliable tests
Will be smaller than of the true scores. This is called attenuation.
Standard error of measurement
- It cannot be larger than the standard deviation of the observed score
- Represents the average size of error scores
- Depends on the reliability of the test
Difference between raw alpha, KR20 and standardized alpha
Raw alpha and KR20 are based on item covariances and variances, standardized alpha is based on item correlations
