Chapter 9

Question 1

Observed score = true score ± error component

Answer 1

Measurement error

Question 2

2 types of measurement error

Answer 2

Systematic and random

Question 3

Potential sources of measurement error

Answer 3

The person taking the measurements
•The measuring instrument
•Variability in the characteristic

Question 4

Not all error is random
●Some error components can be attributed to other sources, such as rater or test occasion.

Answer 4

Generalizability Theory

Question 5

Reflects true variance as a proportion of total variance in a set of scores
•Measured as a unitless coefficient
•Intraclass correlation coefficients (ICC) and kappa coefficients are commonly used

Answer 5

Relative reliability

Question 6

Indicates how much of a measured value, expressed in the original units, is likely due to error
•Standard error of the measurement (SEM) is commonly used

Answer 6

Absolute reliability

Question 7

How does reliability exist in a context

Answer 7

It is relevant to a tools application

Question 8

How is reliability not all-or-none

Answer 8

It exists to some extent in any instrument

Question 9

Types of reliability

Answer 9

●Test-retest
●Rater
●Alternate forms
●Internal consistency

Question 10

When considering for test-retest reliability the interval between tests is considered to do what?

Answer 10

•To support stability of the measurement

Question 11

When considering test-retest reliability, the idea of practice to learning

Answer 11

Carryover

Question 12

Considerations of test-retest reliability that state the act of measurement changes the outcome is called

Answer 12

Testing effects

Question 13

One rater

Answer 13

Intra-rater

Question 14

Two or more raters

Answer 14

Inter-rater

Question 15

Is it best when all raters measure the same response?

Answer 15

Yes

Question 16

With large error variance, the difference from trial 1 to trial 2 may cancel out true score and be composed of mostly error. This is an example of

Answer 16

Change scores

Question 17

Tendency for extreme scores to fall closer to the mean on retesting is called

Answer 17

Regression toward the mean

Question 18

Based on the standard error of the measurement (SEM)
•Amount of change that goes beyond error
•Also known as minimal detectable difference, smallest real difference, smallest detectable change, coefficient of repeatability, or the reliability change index.

Answer 18

Minimal detectable change (MDC)

Question 19

Ways to maximize reliability include

Answer 19

Standardize measurement protocols
●Train raters
●Calibrate and improve the instrument
●Take multiple measurements
●Choose a sample with a range of scores
●Pilot testing