Test construction

Question 1

item difficulty equation

Answer 1

P = total number of examinees passing an item /
total number of examinees

a p value of 0.50 means that 50% of examinees answered the item correctly

Question 2

item discrimination equation

Answer 2

D = percent of examinees in the upper-scoring group that answered the item correctly
minus percent of examinees in the lower-scoring group that answered the item correctly

D value of +1.0 means that all examinees in the upper-scoring group got the item right, and none of the examinees in the lower-scoring group got it right

Question 3

item response theory

Answer 3

uses the item characteristic curve

ability to discriminate between high and low achievers is indicated by the slope of the curve

probably of guessing correctly is the y-intercept

Question 4

reliability coefficient

Answer 4

consistency of test scores

when a test has a reliability coefficient of 0.89, 89% of the variability in obtained scores is true variability

Question 5

Spearman-Brown prophecy formula

Answer 5

provides an estimate of what the reliability coefficient would have been if it had been based on the full length of the test instead of just half the items

Question 6

Cronbach’s coefficient alpha

Answer 6

for split-half reliability

administering a test to a single group of examinees, and using a formula to determine inter-item consistency

Question 7

Kuder-Richardson Formula 20

Answer 7

coefficient alpha - avg reliability obtained from all possible splits of the test

when test items are scored dichotomously (right or wrong), used to determine internal consistency

Question 8

kappa statistic

Answer 8

used to assess inter-rater reliability

nominal or ordinal scale of measurement

Question 9

alternate forms reliability

Answer 9

the most thorough method for estimating reliability

Question 10

internal consistency reliability

Answer 10

not appropriate for speed tests

Question 11

standard error of measurement (SEM)

Answer 11

used to construct a confidence interval around an obtained score

an index of the amount of error that can be expected in obtained scores due to the unreliability of the test

Question 12

standard error of estimate (SEE)

Answer 12

used to construct a confidence interval around an examinee’s predicted criterion scores

an index of error when predicting criterion scores from predictor scores

Question 13

content validity

Answer 13

to obtain information about an examinee’s familiarity with a particular content or behavior domain

Question 14

construct validity

Answer 14

to determine the extent to which an examinee possesses a particular hypothetical trait

Question 15

criterion-related validity

Answer 15

to estimate or predict an examinee’s standing or performance on an external criterion

Question 16

convergent and discriminant validity

Answer 16

methods for assessing construct validity

Question 17

factor analysis

Answer 17

used to identify the minimum number of common factors required to account for the intercorrelations among a set of items

Question 18

a squared factor loading provides a measure of ____

Answer 18

shared variability

Question 19

when factors are orthogonal, a test’s communality can be calculated by

Answer 19

squaring and adding the test’s factor loadings

Question 20

orthogonal factors are ____

Answer 20

uncorrelated

Question 21

oblique factors are _____

Answer 21

correlated

Question 22

concurrent and predictive validity

Answer 22

associated with criterion-related validity

Question 23

to determine the shared variability between two different tests/variables, you ____

Answer 23

square the correlation coefficient

Question 24

sensitivity

Answer 24

the percent of people in the validation sample who have the disorder and were accurately identified as having the disorder

Question 25

specificity

Answer 25

the percent of people who do not have the disorder, and were accurately identified as not having the disorder