Chapter 4 Part 2: Reliability, Validity, and Utility

Question 1

Also known as the index of reliability, a proportion that indicates the ratio between the true score variance on a test and the total variance.

Answer 1

Reliability coefficient.

Question 2

This refers to the component of the observed test score that does not have to do with the test-taker’s ability.

Answer 2

Error.

Question 3

What is the formula that relates observed score, true score, and error?

Answer 3

X = T + E

where,
X = observed score
T = true score
E = error

Question 4

This refers to variance from true differences.

Answer 4

True variance.

Question 5

This refers to variance from irrelevant, random sources.

Answer 5

Error variance.

Question 6

This refers to all of the factors associated with the process of measuring some variable, other than the variable being measured.

Answer 6

Measurement error.

Question 7

A source of error in measuring a targeted variable caused by unpredictable fluctuations and inconsistencies of other variables in the measurement process.

Answer 7

Random error.

Question 8

A source of error in measuring a variable that is typically constant or proportionate to what is presumed to be the true value of the variable being measured.

Answer 8

Systematic error.

Question 9

An estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the test.

Answer 9

Test-retest reliability.

Question 10

An estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.

Answer 10

Parallel forms reliability.

Question 11

Tue or False: In the parallel forms of a test, the items are the same but with different positioning or numberings.

Answer 11

True.

Question 12

True or False: In alternate forms of a test, the test is a different version but has been constructed to be parallel.

Answer 12

True.

Question 13

This is an estimate of the extent to which different forms of the same test have been affected by sampling error, or other errors.

Answer 13

Alternate forms reliability.

Question 14

This is obtained by correlating two (2) pairs of scores obtained from equivalent halves of a single test administered once.

Answer 14

Split-half reliability.

Question 15

This formula allows a test developer or user to estimate internal consistency reliability from a correlation of two halves of a test.

Answer 15

Spearman-Brown formula.

Question 16

This refers to the degree of correlation among all the items on a scale.

Answer 16

Inter-item consistency.

Question 17

This is used for the inter-item consistency of dichotomous items.

Answer 17

KR-20.

Question 18

This is used if all the items have the same degree of difficulty.

Answer 18

KR-21.

Question 19

This refers to the degree of agreement or consistency between two or more scorers with regard to a particular measure.

Answer 19

Inter-scorer reliability.

Question 20

A term used for a trait, state, or ability presumed to be ever-changing as a function of situational and cognitive experience.

Answer 20

Dynamic.

Question 21

A term used for a trait, state, or ability that is barely changing or relatively unchanging.

Answer 21

Static.

Question 22

This theory corresponds the probability that a person with X ability will perform at a level of Y in a test.

Answer 22

Item response theory / latent-trait theory.

Question 23

The attribute of not being easily accomplished, solved, or comprehended.

Answer 23

Difficulty.

Question 24

The degree to which an item differentiates among people with higher or lower levels of a trait, ability, etc.

Answer 24

Discrimination.

Question 25

A factor inherent in a test that systematically prevents accurate, impartial measurement.

Answer 25

Bias.

Question 26

A type of error that is due to the rater intentionally or unintentionally misusing the scale.

Answer 26

Rating error.

Question 27

A type of error in which the rater is lenient in scoring.

Answer 27

Leniency error / generosity error.

Question 28

A type of error in which the rater is strict in scoring.

Answer 28

Severity error.

Question 29

A type of error in which the rater’s rating tends to cluster in the middle of the rating scale.

Answer 29

Central tendency error.

Question 30

This refers to the extent in which a test is used in an impartial, just, and equitable way.

Answer 30

Fairness.

Question 31

This refers to the usefulness or practical value of testing to improve efficiency.

Answer 31

Utility.