Test Construction: Test Theory & Test Score Interpretation

Question 1

Norm-Referenced Interpretation

Answer 1

Percentile ranks

Standard scores
e.g. Z, t, IQ, stanines

Question 2

Mean and SD:

Z-score

Answer 2

M = 0
SD = 1

“Z = Zero”

Question 3

Mean and SD:

T-score

Answer 3

M = 50
SD = 10

“Fif-T-score”

Question 4

Mean and SD:

IQ scores

Answer 4

M = 100
SD  = 15

Question 5

Mean and SD:

Stanine (Standard Nine) scores

Answer 5

StaNINE divide distribution into NINE parts

M = 5
SD = 2

Question 6

Criterion-Referenced Interpretation:

Three Main Types

Answer 6

Percentage Score

Regression Equation

Expectancy Table

Question 7

Classical Test Theory:

Two methods of Item Analysis

Answer 7

Item difficulty

Item discrimination

Question 8

Classical Test Theory: Item Difficulty

Answer 8

p = percentage of respondents that the item correctly

Range of p = 0.0 to +1.0

Question 9

Classical Test Theory:

Item Discrimination

Answer 9

D = Extent that a test item differentiates between lower and higher scoring groups of examinees

Question 10

Limitations of Classical Test Theory

Answer 10

Sample dependent and therefore variable

Scores on one test do not have same value on other tests

(e.g. Math and English score of 50)

Question 11

Item Response Theory: Benefits

Answer 11

Item characteristics do not very across samples

Scores can be compared on different measures

Use of computer-adaptive tests
(items are based on performance on previous items)

Question 12

Item Response Theory: Item Characteristic Curve

Answer 12

Relationship between ability level and probability of responding to item correctly

Question 13

Item Response Theory: Main Assumption

Answer 13

Performance of an examinee on a test item can be explained/predicted by:

traits
latent traits
abilities

Question 14

High Accuracy Rate and Base Rate Below 50%

Relationship to true/false positive/negatives

Answer 14

More false positives than false negatives

False positive = type I error

False Negative = type II error

Question 15

IRT: Item Characteristic Curve’s 3 parameters:

Answer 15

difficulty

discrimination

probability of guessing correctly

Question 16

CTT: Item discrimination index

Answer 16

Compares item score between examinee groups with highest and lowest total test scores

D = -1.0 to +1.0

Most tests aim for D = .35 or higher

Question 17

CTT: Item discrimination index, D = +1.0

Answer 17

Correct item score for:

All examinees in the high-scoring group
None of the examinees in the low-scoring group

Question 18

Classical Test Theory: Item Difficulty Level ideal for most tests

Answer 18

p values close to p = .50

moderate difficulty

Question 19

Accuracy in terms of True/False Positives/Negatives

Answer 19

For 100 people and 98% accuracy:

98 true positives

2 false negatives