Chapter 4 Part 2: Reliability, Validity, and Utility Flashcards
Also known as the index of reliability, a proportion that indicates the ratio between the true score variance on a test and the total variance.
Reliability coefficient.
This refers to the component of the observed test score that does not have to do with the test-taker’s ability.
Error.
What is the formula that relates observed score, true score, and error?
X = T + E
where,
X = observed score
T = true score
E = error
This refers to variance from true differences.
True variance.
This refers to variance from irrelevant, random sources.
Error variance.
This refers to all of the factors associated with the process of measuring some variable, other than the variable being measured.
Measurement error.
A source of error in measuring a targeted variable caused by unpredictable fluctuations and inconsistencies of other variables in the measurement process.
Random error.
A source of error in measuring a variable that is typically constant or proportionate to what is presumed to be the true value of the variable being measured.
Systematic error.
An estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the test.
Test-retest reliability.
An estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.
Parallel forms reliability.
Tue or False: In the parallel forms of a test, the items are the same but with different positioning or numberings.
True.
True or False: In alternate forms of a test, the test is a different version but has been constructed to be parallel.
True.
This is an estimate of the extent to which different forms of the same test have been affected by sampling error, or other errors.
Alternate forms reliability.
This is obtained by correlating two (2) pairs of scores obtained from equivalent halves of a single test administered once.
Split-half reliability.
This formula allows a test developer or user to estimate internal consistency reliability from a correlation of two halves of a test.
Spearman-Brown formula.
This refers to the degree of correlation among all the items on a scale.
Inter-item consistency.
This is used for the inter-item consistency of dichotomous items.
KR-20.
This is used if all the items have the same degree of difficulty.
KR-21.
This refers to the degree of agreement or consistency between two or more scorers with regard to a particular measure.
Inter-scorer reliability.
A term used for a trait, state, or ability presumed to be ever-changing as a function of situational and cognitive experience.
Dynamic.
A term used for a trait, state, or ability that is barely changing or relatively unchanging.
Static.
This theory corresponds the probability that a person with X ability will perform at a level of Y in a test.
Item response theory / latent-trait theory.
The attribute of not being easily accomplished, solved, or comprehended.
Difficulty.
The degree to which an item differentiates among people with higher or lower levels of a trait, ability, etc.
Discrimination.
A factor inherent in a test that systematically prevents accurate, impartial measurement.
Bias.
A type of error that is due to the rater intentionally or unintentionally misusing the scale.
Rating error.
A type of error in which the rater is lenient in scoring.
Leniency error / generosity error.
A type of error in which the rater is strict in scoring.
Severity error.
A type of error in which the rater’s rating tends to cluster in the middle of the rating scale.
Central tendency error.
This refers to the extent in which a test is used in an impartial, just, and equitable way.
Fairness.
This refers to the usefulness or practical value of testing to improve efficiency.
Utility.
