Reliability Flashcards
Ratio between true score variance on a test and the total variance
Reliability coefficient
Chance features of the individual or situation
Random measurement of error
Tool used to estimate or infer the extent to which an observed score deviates from a true score
Standard error of estimate (SEE)
Correlates performance on 2 interval scale measures and uses that correlation to indicate the extent of true score differences
Reliability analysis
Sources of error variance
- test construction (item or content sampling)
- test administration (testtaker and examiner-related variables)
- test scoring and interpretation (computer or subjective scoring)
Appropriate when evaluating reliability of a test that purports to measure something that is relatively stable over time (e.g. personality trait)
Test-retest method (aka time sampling reliability)
When interval between testing is greater than 6 months
Coefficient of stability
An effect where the first testing session influences results of second session
Carryover effect
Solution: prolong time (as little as 4 days to avoid error)
Type of carryover effect when second session of test is higher than the first test
Practice effect
Each form of test, the means and variances of observed test scores are equal; scores obtained correlate equally with other measures
Parallel forms
Different versions of a test that have been constructed so as to be parallel; COEFFICIENT OF EQUIVALENCE
alternate forms (item sampling reliability)
Assess consistency of results across items of a test
internal consistency
Correlate 2 pairs obtained from equivalent halves of a single test administered once
Split-half reliability
Useful when it is undesirable to assess reliability with 2 tests because of factors such as time, expense
Split-half method
Correction formula which allows you to estimate what the correlation between 2 halves would have been if each half had been the length of the whole test; specific application to estimate reliability of the test that is lengthened or shortened by any number of items
Spearman brown formula
Also used to determine the number of items needed to attain a desired level of reliability
Spearman brown formula
Degree of correlation among all items in a scale; useful in assessing homogeneity of test; calculated from a single administration of a single form of test
Inter-item consistency
An estimate of reliability that determine the inter-item consistency of dichotomous items (right or wrong); used for items that have varying diffilculty; 0.5 reasonable reliability
KR 20 (for group intelligence and ability tests)
Used when all items about the same difficulty
kr 21
Estimates the internal consistency of test in which the items are non-dichotomous or there is no right or wrong answer (e.g. Likert scale)
Coefficient alpha/Cronbach’s alpha
Thought as the mean of all possible split-half correlations, test-retest; used when 2 halves of the test have unequal variances; 0.7 reasonable reliability
Coefficient alpha/Cronbach’s alpha (measure only 1 hidden variable or dimension)
The degree of agreement or consistency between 2 or more scorers with regard to a particular measure
Inter-scorer reliability (for individual intelligence tests and projective tests)
If test is heterogeneous, what reliability estimate should you use?
test-retest method
In measuring dynamic traits, what reliability estimate is appropriate?
Internal consistency
In measuring static (unchanging) traits, what reliability estimate is appropriate?
test-retest, alternate/parallel forms
The type of internal consistency which is considered as the average of all split-half correlations
Average inter-item correlations
