Test Construction Flashcards
Items with moderate difficulty levels are typically retained in classical test theory because
- .5 = moderate
- increases test score variability
- helps ensure scores are normally distributed
- provides maximum discrimination b/w examinees
- maximizes the test’s reliability
Item discrimination index measures
the extent to which a test item discriminates between examinees who obtain high versus low scores on the entire test
- ranges from -1 to 1
- .35 or above is acceptable
- items with moderate difficulty most likely to differentiate
Benefits of Item Response Theory
- item characteristics (parameters) are sample invariant (same across different samples)
- possible to equate scores from different sets of items/tests
- easier to develop computer adaptive test
Which theory of test construction uses an item characteristic curve?
item response theory
item characteristic curves provide information on ___
difficulty, discrimination, probability of guessing correctly
According to an item characteristic curve, an items ability to discriminate between high and low achievers is represented by the ____
- slope of the curve
- the steeper the slope, the greater the discrimination
According to an item characteristic curve, the probability of guessing correctly is indicated by _____
the point at which the ICC intercepts the vertical axis
According to an item characteristic curve, an item’s difficulty level is indicated by ____
the ability level at which 50% of examinees in the tryout sample provided a correct response
According to classical test theory, an examinee’s obtained test score (X) is composed of ____ and ____
their true score (T) and an error component (E)
A reliability coefficient of .84 indicates
that 84% of variability in scores is due to true score differences among examinees, while the remaining 16% is due to measurement error.
Kuder-richardson Formula 20
a variation of coefficient alpha for when test items are scored dichotomously (right/wrong)
internal consistency reliability is not appropriate for
speeded tests
the reliability coefficient is maximized when the range of scores is
unrestricted
standard error of measurement
used to construct a confidence interval around a measured (obtained) score.
Content Validity
test will be used to obtain information about an examinee’s familiarity with a particular content or behavior domain. Determined by experts
Construct validity
the test will be used to determine the extent to which an examinee possesses a particular hypothetical trait
Criterion-related Validity
the test will be used to estimate or predict an examniee’s standing or performance on an external criterion
Face validity
whether or not a test looks like it measures what it is intended to measure
convergent and discriminate validity are used to assess ___ validity
construct
a squared factor loading provides a measure of
shared variablity
when factors are orthogonal, a test’s communality can be calculated by ___
squaring and adding the test’s factor loadings
Two types of criterion-related validity
concurrent and predictive
standard error of the estimate
is used to construct a confidence interval around a predicted (estimated) criterion score.
Base rate
true positives + false negatives/ total number of people
Sensitivity
percent of people in the validation sample who have the disorder and were accurately identified by the predictor as having the disorder. (true positives/ true positive + false negatives)
specificity
percent of people in the validation sample who do not have the disorder and were accurately identified by the predictor as not having the disorder. (true negatives/ true negatives + false positives)
