Test Construction

Question 1

When using % agreement for inter-rater reliability, one problem could be that

Answer 1

it overestimates reliability due to chance agreement

Question 2

If you administer a test to another sample ____ may occur where the validity coefficient is likely to be smaller with the second sample than with the first when cross-validity

Answer 2

shrinkage

Question 3

multrait-multimatrix method is important for evaluating a test’s”

Answer 3

construct validity

Question 4

The Kuder-Richardson Formula 20 (KR-20) can be used to estimate a test’s ____________ reliability when test items are scored dichotomously.

Answer 4

internal consistency

Question 5

A test’s reliability coefficient can range from

Answer 5

0 to 1

.90 = req. for many high stakes tests
.70 = req. for many other tests

Question 6

Formula for standard error of measurement

Answer 6

SD*sq root of 1-reliability coeff

Question 7

You would use which of the following to construct a confidence interval around an examinee’s predicted criterion score?

Answer 7

Standard error of estimate

Question 8

Murray’s theory of personality, which describes personality as being the result of internal and external forces - resulted in the development of which personality test

Answer 8

Thematic Appreciation Test

Question 9

construct versus content validity

Answer 9

construct: does it measure what you want it to measure
content: does it adequately sample everything to cover the content of the ability

Question 10

the amount of variability in obtained test scores that’s due to true score variability

Answer 10

reliability coefficient

Question 11

Spearman-Brown prophecy formula is used to

Answer 11

correct split-half reliability, which shortens the test into 2 tests

Question 12

Chronbach’s alpha and KR-20 measure:

Whereas, Cohen’s kappa and Kendall’s coefficient measure:

Answer 12

Chronbach’s alpha and KR-20 measure: internal consistency

Cohen’s kappa and Kendall’s coefficient measure: inter-rater reliability

Question 13

Percentage scores (40/80 correct answers = 50%) and expectancy tables are examples of

Answer 13

criterion referenced scores

Question 14

Another name for % variability accounted for

Answer 14

coefficient of determination

Question 15

a method for developing personality inventories in which the items (presumed to measure one or more traits) are created and then administered to a criterion group of people known to possess a certain characteristic (e.g., antisocial behavior, significant anxiety, exaggerated concern about physical health) and to a control group of people without the characteristic. Only those items that demonstrate an ability to distinguish between the two groups are chosen for inclusion in the final inventory.

Answer 15

empirical-criterion keyeing

Question 16

The standard error of measurement is used to:

Answer 16

calculate CI