Chapter 9 Flashcards
Observed score = true score ± error component
Measurement error
2 types of measurement error
Systematic and random
Potential sources of measurement error
The person taking the measurements
•The measuring instrument
•Variability in the characteristic
Not all error is random
●Some error components can be attributed to other sources, such as rater or test occasion.
Generalizability Theory
Reflects true variance as a proportion of total variance in a set of scores
•Measured as a unitless coefficient
•Intraclass correlation coefficients (ICC) and kappa coefficients are commonly used
Relative reliability
Indicates how much of a measured value, expressed in the original units, is likely due to error
•Standard error of the measurement (SEM) is commonly used
Absolute reliability
How does reliability exist in a context
It is relevant to a tools application
How is reliability not all-or-none
It exists to some extent in any instrument
Types of reliability
●Test-retest
●Rater
●Alternate forms
●Internal consistency
When considering for test-retest reliability the interval between tests is considered to do what?
•To support stability of the measurement
When considering test-retest reliability, the idea of practice to learning
Carryover
Considerations of test-retest reliability that state the act of measurement changes the outcome is called
Testing effects
One rater
Intra-rater
Two or more raters
Inter-rater
Is it best when all raters measure the same response?
Yes
With large error variance, the difference from trial 1 to trial 2 may cancel out true score and be composed of mostly error. This is an example of
Change scores
Tendency for extreme scores to fall closer to the mean on retesting is called
Regression toward the mean
Based on the standard error of the measurement (SEM)
•Amount of change that goes beyond error
•Also known as minimal detectable difference, smallest real difference, smallest detectable change, coefficient of repeatability, or the reliability change index.
Minimal detectable change (MDC)
Ways to maximize reliability include
Standardize measurement protocols
●Train raters
●Calibrate and improve the instrument
●Take multiple measurements
●Choose a sample with a range of scores
●Pilot testing
