EPPP Practice Questions: Test Construction Flashcards
A researcher calculates a kappa coefficient of .90. This provides evidence of:
a. internal validity.
b. construct validity.
c. inter-rater reliability.
d. internal consistency reliability.
c. inter-rater reliability
The kappa coefficient is used to determine the degree of association between scores assigned
by different raters and provides information about a measure’s inter-rater reliability.
It would be most important to assess the test-retest reliability of a measure that:
a. is subjectively scored.
b. assesses examinees’ speed of responding.
c. purportedly measures a stable trait.
d. is designed to measure a characteristic that fluctuates over time.
c. purportedly measures a stable trait
To evaluate test-retest reliability, the same test is administered to the same group of
examinees on two different occasions. The two sets of scores are then correlated. If a test is
supposed to measure a stable trait, you would want to make sure that scores are stable over
time. Therefore, test-retest reliability would be important for this kind of test.
You would use the Spearman-Brown formula to:
a. determine the range within which an examinee’s true score is likely to fall given
his/her obtained score.
b. determine the degree of association between two rank-ordered variables.
c. estimate what a predictor’s validity coefficient would be if the predictor and/or
criterion were perfectly reliable.
d. estimate the effects of increasing or decreasing the length of a test on its reliability
coefficient.
d. estimate the effects of increasing or decreasing the length of a test on its reliability
coefficient.
The Spearman-Brown formula is used to estimate the effects of adding or subtracting items to
a test on its reliability coefficient. It is often used in conjunction with split-half reliability.
Matilda obtains a score of 90 on a test that has a mean of 100, standard deviation of 10,
and standard error of measurement of 6. The 95% confidence interval for Matilda’s score
is:
a. 80 to 100.
b. 70 to 110.
c. 84 to 96.
d. 78 to 102.
d. 78 to 102
The 95% confidence interval is obtained by adding and subtracting two standard errors from
the examinee’s score.
In a multitrait-multimethod matrix, a large heterotrait-monomethod coefficient provides
evidence of:
a. a lack of discriminant validity.
b. the presence of adequate discriminant validity.
c. a lack of convergent validity.
d. the presence of adequate convergent validity.
a. a lack of discriminant validity
The heterotrait-monomethod coefficient indicates the correlation between two different traits
being measured with a similar type of measuring instrument. A large heterotrait-monomethod
coefficient indicates a high correlation between the instruments measuring different traits,
suggesting a lack of discriminant (divergent) validity.
In a factor matrix, a test’s _______ indicates the proportion of the test’s total variance
that is accounted for by the identified factors.
a. factor loading
b. principal component
c. critical value
d. communality
d. communality
Each test included in a factor analysis has a communality, which indicates the total amount
of variability in test scores that has been explained by the factor analysis – i.e., by all of the
identified factors.
The purpose of “rotation” in factor analysis is to:
a. obtain a pattern of factor loadings that is easier to interpret.
b. reduce the impact of measurement error on the factor loadings.
c. reduce the magnitude of the communalities.
d. obtain a clearer pattern of communalities.
a. obtain a pattern of factor loadings that is easier to interpret.
The pattern of factor loadings in the initial factor matrix is often difficult to interpret, so the
factors are rotated to obtain a pattern that’s easier to interpret.
To reduce the number of false positives obtained when using a new selection tool, you
would:
a. raise the predictor cutoff score.
b. lower the predictor cutoff score.
c. raise the predictor and criterion cutoff scores.
d. lower the predictor and criterion cutoff scores.
a. raise the predictor cutoff score
Raising the predictor cutoff and/or lowering the criterion score would have the effect of
decreasing the number of false positives.
Use of the Taylor-Russell tables would indicate that the incremental validity of a
selection test that has a moderate validity coefficient is greatest when the selection ratio is
____ and the base rate is ____.
a. .85; .20
b. .85; .50
c. .15; .20
d. .15; .50
d. .15; .50
The Taylor-Russell Tables are used to determine a test’s incremental validity for various
combinations of base rates, selection ratios, and validity coefficients. They indicate that a test
with a low or moderate validity coefficient can improve decision-making accuracy when the
selection ratio is low (e.g., .15) and the base rate is moderate (near .50).
The relationship between reliability and validity is such that:
a. a test’s validity coefficient cannot exceed its reliability coefficient.
b. a test’s validity coefficient cannot exceed the square root of its reliability coefficient.
c. a test’s validity coefficient cannot exceed the square of its reliability coefficient.
d. a test’s reliability coefficient cannot exceed its validity coefficient.
b. a test’s validity coefficient cannot exceed the square root of its reliability coefficient
Reliability places a ceiling on validity. Specifically, a test’s maximum validity coefficient
cannot exceed the square root of its reliability coefficient. For example, if a test has a
reliability coefficient of .81, its validity coefficient cannot exceed .90.
Cleo and Cleopatra obtain percentile ranks, respectively, of 48 and 92 on a math test.
If four points is subtracted from each of their raw scores (due to scoring error) but not from
the scores of the other examinees, you would expect:
a. Cleo’s percentile rank will decrease more than Cleopatra’s.
b. Cleo’s percentile rank will decrease less than Cleopatra’s.
c. Cleo and Cleopatra’s percentile ranks will decrease by the same amount.
d. Cleo and Cleopatra’s percentile ranks will not change.
a. Cleo’s percentile rank will decrease more than Cleopatra’s
In a percentile rank distribution, scores are evenly distributed throughout the distribution.
Consequently, when converting raw scores to percentile ranks, small differences in the
middle of the raw score distribution are larger in terms of percentile ranks than the same
differences at the extremes of the distribution.
Assuming a normal distribution, which of the following represents the highest score?
a. a z score of 1.5
b. a T score of 70
c. a WAIS score of 120
d. a percentile rank of 92
b. a T score of 70
A T score of 70 is two standard deviations above the mean and is the highest score of those
given in the responses.
The distribution of percentile ranks is:
a. the same shape as the distribution of raw scores.
b. always “normal” in shape.
c. always “flat” in shape.
d. leptokurtic relative to the distribution of raw scores.
c. always “flat” in shape
Because the percentile rank distribution is ordinal, converting raw scores to a percentile rank
results in a flat distribution. This is because, at least theoretically, the converted scores are
evenly spaced in the percentile rank distribution, which ranges from 1 to 100 (i.e., when there
are 100 scores, there will be a frequency of 1 at each score).
Bobby B. obtains a test score of 110. The test has a mean of 120 and a standard
deviation of 10 and test scores have a range of 100 and are normally distributed. If
Bobby’s teacher converts all of the students’ test scores to T-scores and z-scores, Bobby’s
scores will be which of the following?
a. T = 60; z = +1.0
b. T = 90; z = -1.0
c. T = 40; z = -2.0
d. T = 40; z = -1.0
d. T = 40; z = -1.0
Knowing that the T-score distribution has a mean of 50 and standard deviation of 10 and that
the z-score distribution has a mean of 0 and standard deviation of 1 would have helped you
identify the correct answer to this question. Since Bobby’s raw score is one standard
deviation below the mean, his T-score would be 40 and his z-score would be -1.0. (In the
T-score distribution, a score of 40 is one standard deviation below the mean and, in the
z-score distribution, a z-score of -1.0 is one standard deviation below the mean.)
