Test Construction

Question 1

T-score distributions have a mean of _____ and a standard deviation of _____.

Answer 1

Mean of 50
SD of 10

Ex. A score of 62 on the MMPI is 12 T-score points above the mean (50) - so it is 1.2 standard deviations above the mean.
(10 x 1.2 = 12)

Question 2

Reliability coefficient range

Answer 2

0.0 - 1.0

0. 0 = completely unreliable
1. 0 = perfectly reliable

Question 3

How do you interpret a reliability coefficient?

Answer 3

Directly

Question 4

Kuder-Richardson Formula (KR-20)

Answer 4

Used when test items are dichotomously scored

Right/wrong, yes/no

Question 5

What method for establishing reliability is considered to be the best?

Answer 5

Alternate forms

Question 6

Alternate forms reliability

Answer 6

Administering two equivalent forms of a test to the same group of examinees and obtaining a correlation between the two sets or scores

Question 7

Why do some experts consider an alternate forms coefficient to be superior?

Answer 7

It would have had to be consistent across time and different content.

Question 8

A kappa coefficient is used to evaluate what?

Answer 8

Inter-rater reliability

Question 9

A kappa coefficient in the lower .90s indicates what?

Answer 9

High reliability

Question 10

Speed vs. power test

Answer 10

Speed test measures an examinee’s response rate

Power test measures the level of difficulty a person can reach (items usually arranged in increasing difficulty)

Question 11

Maximum vs. typical performance

Answer 11

Maximum: what a person is capable of achieving (WJ)

Typical: what an examinee usually does (personality test)

Question 12

What is a source of measurement error for the test-retest coefficient?

Answer 12

Time sampling

Question 13

If there have been changes in exam conditions from one administration to the next that impact different examinees in different ways, what has occurred?

Answer 13

Time sampling

Question 14

Which type of coefficient tends to be lower despite being preferred: alternate forms or test-retest?

Answer 14

Alternate forms

Question 15

Internal consistency

Answer 15

Obtaining correlations among individual items in a test

Question 16

3 methods for determining internal consistency

Answer 16

1. Split-half reliability
2. Spearman-Brown formula
3. Kuder-Richardson Formula (KR-20)

Question 17

T/F: The standard error of measurement indicates how much error an individual test score can be expected to have

Answer 17

True

Question 18

The standard error of measurement is used to construct what?

Answer 18

A confidence interval

Question 19

When a test measures the knowledge of the content domain it was designed to measure, we can say the test has what?

Answer 19

Content validity

Question 20

We would say that the SAT has what type of validity if it can accurately predict an examinee’s performance in college?

Answer 20

Criterion-related validity

Question 21

A test has what type of validity if it can accurately measure a theoretical, non-observation construct or trait?

Answer 21

Construct validity

Question 22

Convergent/Divergent Validity and a Factor Analysis are associated with which type of validity?

Answer 22

Construct validity

Question 23

Convergent vs. Divergent Validity

Answer 23

Convergent: the test has a high correlation with another test that measures the SAME construct

Divergent: the test has a low correlation with a test that measures a different construct

Question 24

The more similar a group is may result in an increase or decrease in reliability?

Answer 24

Decrease

Reliability will increase with heterogeneity

Question 25

A multitrait-multimethod matrix assesses what?

Answer 25

Convergent and divergent validity

Question 26

Divergent validity may also be called what?

Answer 26

Discriminant Validity