Test Construction Flashcards
Reliability
Amt of consistency, repeatability, and dependability in scores on a given test
Classical Test Theory
Obtained score in a combo of truth & error
X=T + E
True score variability vs Error Variability
due to real diffs in ability or knowledge in test takers;
caused by chance or random factors
Range of reliability coefficient for a test
- 00 to 1.00
* Interp directly as the percentage of true score variability
Minimum acceptable Reliability
0.80
Sources of Error in Test Impacting Reliability
- Content Sampling- items that by chance tap into test taker’s knowledge, or don’t
- Time Sampling-impact of giving tests at 2 diff points in time
- Test Heterogeneity
Factors Affecting Reliability
# of items (more items--> higher rel) Homogeneity of items (items tap similar content areas--> higher rel) Range of Scores (unrestricted range maximizes reliability) Ability to guess (easier to guess, lower reliability b/c no longer assessing true knowledge)
4 Estimates of Reliability
Test-Retest
Parallel Forms
Internal consistency
Interrater Reliability
Coefficient of stability
for test-retest reliability
Major source of error is time sampling (time interval btwn administrations)
Coefficient of equivalence
for parallel forms reliability
major sources of error include time sampling & content sampling
Spearman Brown Prophecy Formula
used for split half reliability computation to compensate for impact of reducing # of items on each half of the test
Split half reliability is inappropriate for ___ and ___ tests
Speeded (all items easy, but about how many comp in allotted time)
Power (subjects need to complete all items of varying difficulty, like EPPP)
Ways to measure internal consistency reliability
Split Half
Kuder-Richardson (used when items are scored dichotomously) & Cronbach’s Alpha (used when items are likert scale)
Standard Error of Measurement
standard deviation of a theoretically normal distribution of test scores obtained by one ind on equivalent tests
Range of values for Standard Error of the Measurement
0.0 to standard deviation of the test (when test is totally unreliable, SEM equal the standard deviation of the test)