Reliability

Question 1

Different versions of the same test or measure; contrast with parallel forms

Answer 1

Alternate forms

Question 2

A statistic widely employed in test construction and used to assist in deriving an estimate of reliability; more technically, it is equal to the mean of all split-half reliabilities

Answer 2

Coefficient alpha or Cronbach’s alpha and alpha

Question 3

An estimate of parallel-forms reliability or alternate-forms reliability

Answer 3

Coefficient of equivalence

Question 4

In generalizability theory, an index of the influence that particular facets have on a test score

Answer 4

Coefficient of generalizability

Question 5

An estimate of test-retest reliability obtained during time intervals of six months or longer

Answer 5

Coefficient of stability

Question 6

A range or band of test scores that is likely to contain the “true score”

Answer 6

Confidence interval

Question 7

The variety of the subject matter contained in the items; frequently referred to in the context of the variation between individual test items in a test or between test items in two or more tests

Answer 7

Content sampling or item sampling

Question 8

In the true score model, the component of variance attributable to random sources irrelevant to the trait or ability the test purports to measure in an observed score or distribution of scores. Common sources include those related to test construction (including item or content sampling), test administration, and test scoring and interpretation

Answer 8

Error variance

Question 9

Also referred to as domain sampling theory, a system of assumptions about measurement that includes the notion that a test score, and even a response to an individual item, is composed of a relatively stable component that actually is what the test or individual item is designed to measure, and relatively unstable components that collectively can be accounted for as error

Answer 9

Generalizability theory

Question 10

A reference to a phenomenon associated with reliability estimates wherein the variance of either variable in a correlational analysis is inflated by the sampling procedure used and the resulting correlation coefficient tends to be higher as a consequence; contrast with restriction of range

Answer 10

Inflation of range or inflation of variance

Question 11

An estimate of how consistently the items of a test measure a single construct obtained from a single administration of a single form of the test and the measurement of the degree of correlation among all of the test items

Answer 11

Internal consistency or inter-item consistency

Question 12

An estimate of the degree of agreement or consistency between two or more scorers (or judges or raters or observers)

Answer 12

Inter-scorer reliability or inter-rater reliability, observer reliability, judge reliability, and scorer reliability

Question 13

A system of assumptions about measurement, including the assumption that a trait being measured by a test is uni-dimensional, and the extent to which each test item measures the trait

Answer 13

Item response theory (IRT) or latent-trait theory or the latent-trait model

Question 14

The variety of the subject matter contained in the items; frequently referred to in the context of the variation between individual test items in a test or between test items in two or more tests

Answer 14

Item sampling or content sampling

Question 15

A measure of inter-scorer reliability originally designed for use when scorers make ratings using nominal scales of measurement

Answer 15

Kappa statistic

Question 16

A series of equations developed by G. F. Kuder and M. W. Richardson designed to estimate the inter-item consistency of tests

Answer 16

Kuder-Richardson formulas

Question 17

An estimate of split-half reliability of a test, obtained by assigning odd-numbered items to one-half of the test and even-numbered items to the other half

Answer 17

Odd-even reliability

Question 18

Two or more versions or forms of the same test when for each form, the means and variances of observed test scores are equal; contrast with alternate forms

Answer 18

Parallel forms

Question 19

A test, usually of achievement or ability, with (1) either no time limit or such a long time limit that all test takers can attempt all items, and (2) some items so difficult that no test taker can obtain a perfect score; contrast with speed test

Answer 19

Power test

Question 20

The extent to which measurements are consistent or repeatable; also, the extent to which measurements differ from occasion to occasion as a function of measurement error

Answer 20

Reliability

Question 21

General term for an index of reliability or the ratio of true score variance on a test to the total variance

Answer 21

Reliability coefficient

Question 22

A phenomenon associated with reliability estimates wherein the variance of either variable in a correlational analysis is restricted by the sampling procedure used, and the resulting correlation coefficient tends to be lower as a consequence; contrast with inflation of range

Answer 22

Restriction of range or restriction of variance

Question 23

Now outdated, an equation once used to estimate internal consistency reliability

Answer 23

Rulon formula

Question 24

An equation used to estimate internal consistency reliability from a correlation of two halves of a test that has been lengthened or shortened; inappropriate for use with heterogeneous tests or speed tests

Answer 24

Spearman-Brown formula

Question 25

A test, usually of achievement or ability, with a time limit; speed tests usually contain items of uniform difficulty level

Answer 25

Speed test

Question 26

An estimate of the internal consistency of a test obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once

Answer 26

Split-half reliability

Question 27

In true score theory, a statistic designed to estimate the extent to which an observed score deviates from a true score

Answer 27

Standard error of measurement or standard error of a score

Question 28

A statistic designed to aid in determining how large a difference between two scores should be before it is considered statistically significant

Answer 28

Standard error of the difference

Question 29

The extent to which individual test items do not measure a single construct but instead measure different factors; contrast with test homogeneity

Answer 29

Test heterogeneity

Question 30

The extent to which individual test items measure a single construct; contrast with test heterogeneity

Answer 30

Test homogeneity

Question 31

An estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test

Answer 31

Test-retest reliability

Question 32

In the true score model, the component of variance attributable to true differences in the ability or trait being measured, inherent in an observed score or distribution of scores

Answer 32

True variance

Question 33

A measure of variability equal to the arithmetic mean of the squares of the differences between the scores in a distribution and their mean

Answer 33

Variance