Reliability Flashcards
Different versions of the same test or measure; contrast with parallel forms
Alternate forms
A statistic widely employed in test construction and used to assist in deriving an estimate of reliability; more technically, it is equal to the mean of all split-half reliabilities
Coefficient alpha or Cronbach’s alpha and alpha
An estimate of parallel-forms reliability or alternate-forms reliability
Coefficient of equivalence
In generalizability theory, an index of the influence that particular facets have on a test score
Coefficient of generalizability
An estimate of test-retest reliability obtained during time intervals of six months or longer
Coefficient of stability
A range or band of test scores that is likely to contain the “true score”
Confidence interval
The variety of the subject matter contained in the items; frequently referred to in the context of the variation between individual test items in a test or between test items in two or more tests
Content sampling or item sampling
In the true score model, the component of variance attributable to random sources irrelevant to the trait or ability the test purports to measure in an observed score or distribution of scores. Common sources include those related to test construction (including item or content sampling), test administration, and test scoring and interpretation
Error variance
Also referred to as domain sampling theory, a system of assumptions about measurement that includes the notion that a test score, and even a response to an individual item, is composed of a relatively stable component that actually is what the test or individual item is designed to measure, and relatively unstable components that collectively can be accounted for as error
Generalizability theory
A reference to a phenomenon associated with reliability estimates wherein the variance of either variable in a correlational analysis is inflated by the sampling procedure used and the resulting correlation coefficient tends to be higher as a consequence; contrast with restriction of range
Inflation of range or inflation of variance
An estimate of how consistently the items of a test measure a single construct obtained from a single administration of a single form of the test and the measurement of the degree of correlation among all of the test items
Internal consistency or inter-item consistency
An estimate of the degree of agreement or consistency between two or more scorers (or judges or raters or observers)
Inter-scorer reliability or inter-rater reliability, observer reliability, judge reliability, and scorer reliability
A system of assumptions about measurement, including the assumption that a trait being measured by a test is uni-dimensional, and the extent to which each test item measures the trait
Item response theory (IRT) or latent-trait theory or the latent-trait model
The variety of the subject matter contained in the items; frequently referred to in the context of the variation between individual test items in a test or between test items in two or more tests
Item sampling or content sampling
A measure of inter-scorer reliability originally designed for use when scorers make ratings using nominal scales of measurement
Kappa statistic
A series of equations developed by G. F. Kuder and M. W. Richardson designed to estimate the inter-item consistency of tests
Kuder-Richardson formulas
An estimate of split-half reliability of a test, obtained by assigning odd-numbered items to one-half of the test and even-numbered items to the other half
Odd-even reliability
Two or more versions or forms of the same test when for each form, the means and variances of observed test scores are equal; contrast with alternate forms
Parallel forms
A test, usually of achievement or ability, with (1) either no time limit or such a long time limit that all test takers can attempt all items, and (2) some items so difficult that no test taker can obtain a perfect score; contrast with speed test
Power test
The extent to which measurements are consistent or repeatable; also, the extent to which measurements differ from occasion to occasion as a function of measurement error
Reliability
General term for an index of reliability or the ratio of true score variance on a test to the total variance
Reliability coefficient
A phenomenon associated with reliability estimates wherein the variance of either variable in a correlational analysis is restricted by the sampling procedure used, and the resulting correlation coefficient tends to be lower as a consequence; contrast with inflation of range
Restriction of range or restriction of variance
Now outdated, an equation once used to estimate internal consistency reliability
Rulon formula
An equation used to estimate internal consistency reliability from a correlation of two halves of a test that has been lengthened or shortened; inappropriate for use with heterogeneous tests or speed tests
Spearman-Brown formula
A test, usually of achievement or ability, with a time limit; speed tests usually contain items of uniform difficulty level
Speed test
An estimate of the internal consistency of a test obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once
Split-half reliability
In true score theory, a statistic designed to estimate the extent to which an observed score deviates from a true score
Standard error of measurement or standard error of a score
A statistic designed to aid in determining how large a difference between two scores should be before it is considered statistically significant
Standard error of the difference
The extent to which individual test items do not measure a single construct but instead measure different factors; contrast with test homogeneity
Test heterogeneity
The extent to which individual test items measure a single construct; contrast with test heterogeneity
Test homogeneity
An estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test
Test-retest reliability
In the true score model, the component of variance attributable to true differences in the ability or trait being measured, inherent in an observed score or distribution of scores
True variance
A measure of variability equal to the arithmetic mean of the squares of the differences between the scores in a distribution and their mean
Variance
