Item Analysis and Test Reliability

Question 1

classical test theory

Answer 1

used as framework to develop and evaluate tests

Question 2

true score variability

Answer 2

result of actual differences among examinees

Question 3

measurement error

Answer 3

random factors affect test performance. Ex- distractions, examinee fatigue

Question 4

test reliability

Answer 4

consistency

Question 5

test-retest reliability

Answer 5

consistency of scores over time. administer, readminister later, correlate 2 sets of scores

Question 6

alternate forms of reliability

Answer 6

consistency over different forms of the test

Question 7

internal consistency reliability

Answer 7

consistency over different test items. Not good for speed tests.

Question 8

coefficient alpha (Chronbach’s alpha)

Answer 8

administer test to sample and calculate average inter-item consistency

Question 9

Kuder-Richardson 20 (KR-20)

Answer 9

coefficient used when items are dichotomously scored (correct or incorrect)

Question 10

split half reliability

Answer 10

administer test to a sample, split test in half and correlate the 2 halves (ex- half do odd, half do even questions)

Question 11

Spearman Brown prophecy formula

Answer 11

determine effects of lengthening or shortening tests

Question 12

Inter-rater reliability

Answer 12

consistency of scores over different raters for subjectively scores tests

Question 13

Cohen’s kappa coefficient

Answer 13

consistency among 2 raters when rate is nominal scale

Question 14

coefficient of concordonance

Answer 14

assess 3 or more raters when ratings are ranked

Question 15

consensual observer drift

Answer 15

2 or more raters communicate while rating, which increased consistency

Question 16

content homogeneity

Answer 16

tests that are homogenous content have larger reliability coefficients than heterogenous

Question 17

range of scores

Answer 17

reliabilty larger when scores are unrestricted range

Question 18

guessing

Answer 18

reliability affected by likelihood answered questions correctly by guessing

Question 19

item difficulty

Answer 19

dichotomous difficulty level (p) is .50. P ranges from 0 to 1.0.

Question 20

item discrimination

Answer 20

(D) -1.0 to +1.0 value of .30 or higher is acceptable

Question 21

confidence interval

Answer 21

range of which score is true score

Question 22

standard error of measurement

Answer 22

confidence interval

Question 23

Item Response Theory (IRT)

Answer 23

alternative to classical test theory, how does performance on each test item relate to latent ability that is measured b the test