Test Construction

Question 1

name the 3 methods used to evaluate internal consistency reliability

Answer 1

• coefficient alpha
• kuder-richardson 20
• split-half reliability

Question 2

name 2 methods used to evaluate inter-rater reliability

Answer 2

Cohen’s kappa coefficient

Kendall’s coefficient of concordance

Question 3

Convergent vs Divergent Validity

Answer 3

convergent = high correlation with similar measures
divergent = low correlation with unrelated measures

Question 4

when a heterotrait-monomethod coefficient is small, this indicates ______ validity

Answer 4

divergent

Question 5

when a monotrait-heteromethod coefficient is large, this indicates _____ validity

Answer 5

convergent

Question 6

orthogonal

Answer 6

uncorrelated

Question 7

oblique

Answer 7

correlated

Question 8

cross-validation

Answer 8

the process of re-assessing a test’s criterion-related validity on a new sample to check generalizability of the original validity coefficient

Question 9

shrinkage is greatest when _____

Answer 9

the initial sample is small and, for the multiple correlation coefficient, the number of predictors is large

Question 10

clinical utility (the extent to which a test is useful for clinical purposes) can be assessed by _____ and ______

Answer 10

incremental validity and diagnostic efficiency

Question 11

diagnostic efficiency

Answer 11

the ability of a test to correctly distinguish between people who do and don’t have a disorder/characteristic

Question 12

a predictor has incremental validity when _____

Answer 12

its use produces an adequate increase in decision-making accuracy

Question 13

an expectancy table is useful for interpreting an examinee’s scores in terms of ____

Answer 13

likely criterion performance

Question 14

a z-score of 0 is equivalent to a T-score of ____ and a stanine of ____

Answer 14

50; 5