5 Flashcards
reliability/precision
the consistency of test scores
the consistency of test scores
reliability/precision
test–retest coefficients
estimate the reliability/precision of test scores when the same people take the same test form on two separate occasions
estimate the reliability/precision of test scores when the same people take the same test form on two separate occasions
test–retest coefficients
alternate-form coefficients
estimate the reliability/precision of test scores when the same people take a different but equivalent form of a test on two independent testing sessions.
estimate the reliability/precision of test scores when the same people take a different but equivalent form of a test on two independent testing sessions.
alternate-form coefficients
internal consistency coefficients
which estimate the reliability/precision of test scores by looking at the relationships between different parts of the same test given on a single occasion
which estimate the reliability/precision of test scores by looking at the relationships between different parts of the same test given on a single occasion
internal consistency coefficients
scorer reliability
agreement when raters use their subjective judgment to assign scores to test taker responses
agreement when raters use their subjective judgment to assign scores to test taker responses
scorer reliability
measurement error
Variations in the measurements of the room—perhaps a fraction of an inch from time to time
Variations in the measurements of the room—perhaps a fraction of an inch from time to time
measurement error
reliable test
one we can trust to measure each person in approximately the same way every time it is used
one we can trust to measure each person in approximately the same way every time it is used
reliable test
just because it’s reliable doesn’t mean its _____
valid
true score (T)
a measure of the amount of the attribute that the test is designed to measure
a measure of the amount of the attribute that the test is designed to measure
true score (T)
random error (E)
the difference between a person’s actual score on a test and that person’s true score (T)
the difference between a person’s actual score on a test and that person’s true score (T)
random error (E)
error in measurement formula
X = T + E
systematic error
When a single source of error always increases or decreases the true score by the same amount
When a single source of error always increases or decreases the true score by the same amount
systematic error
two problems that can add systematic error as well as random error to test scores
practice effects and order effects
reliability coefficient
correlation between the two sets of test scores
correlation between the two sets of test scores
reliability coefficient
test–retest method
a test developer gives the same test to the same group of test takers on two different occasions
a test developer gives the same test to the same group of test takers on two different occasions
test–retest method
Practice effects
limitation in test-retest
when test takers benefit from taking the test the first time (practice), which enables them to solve problems more quickly and correctly the second time
when test takers benefit from taking the test the first time (practice), which enables them to solve problems more quickly and correctly the second time
Practice effects
alternate forms/parallel forms
create two different forms of the test
create two different forms of the test
alternate forms/parallel forms
order effects
changes in test scores resulting from the order in which the tests were taken
changes in test scores resulting from the order in which the tests were taken
order effects
Internal consistency
a measure of how related the items (or groups of items) on the test are to one another.
a measure of how related the items (or groups of items) on the test are to one another.
Internal consistency
split-half method
divide the test into halves and then compare the set of individual test scores on the first half with the set of individual test scores on the second half
divide the test into halves and then compare the set of individual test scores on the first half with the set of individual test scores on the second half
split-half method
homogeneous test
measuring only one trait or characteristic
measuring only one trait or characteristic
homogeneous test
heterogeneous test
measuring more than one trait or characteristic
measuring more than one trait or characteristic
heterogeneous test
Estimating reliability using methods of internal consistency is appropriate only for a _________
homogeneous test
scorer reliability/interscorer agreement
the amount of consistency among scorers’ judgments
the amount of consistency among scorers’ judgments
scorer reliability/interscorer agreement
intrascorer reliability
whether each clinician was consistent in the way he or she assigned scores from test to test.
whether each clinician was consistent in the way he or she assigned scores from test to test.
intrascorer reliability
interrater agreement
an index of how consistently the scorers rate or make decisions.
an index of how consistently the scorers rate or make decisions.
interrater agreement
intrarater agreement
How well a scorer makes consistent judgments across all tests.
How well a scorer makes consistent judgments across all tests.
intrarater agreement
confidence interval
a range of scores that we feel confident will include the test taker’s true score
a range of scores that we feel confident will include the test taker’s true score
confidence interval
factors That Influence Reliability
1) poorly designed test
2) administrators don’t follow insturctions
3) test not scored accurately
4) test takers: fatigue, illness, exposure to test questions before hand
what are the six factors related to sources of error
test length
homogeneity: similar questions on the test
test-retest interval: longer the interval, lower the reliability
test administration
scoring
cooperation of test takers
generalizability theory
concerns how well and under what conditions we can generalize an estimation of reliabilty/precision of test scores from one test administration to another
concerns how well and under what conditions we can generalize an estimation of reliabilty/precision of test scores from one test administration to another
generalizability theory
