Measurement Issues Flashcards
What is measurement reliability?
extent to which a measurement is consistent & free from error, indicates ability of an instrument to produce similar scores, any test is situation specific
What are the 6 ways measurement reliability is assessed by?
- accuracy
- consistency
- stability
- precision
- reproducibility
- dependability
What 2 characteristics is measurement validity dependent on?
reliability & relevance
All valid tests are ____, however all ____ tests are not valid.
reliable, reliable
What are 3 sources of measurement error?
- the individual taking the measurement
- variability of characteristics being measured
- the measuring instrument itself
What is predictable, consistent, and usually correctable?
systematic error
What is due to chance, unpredictable, and effect scores are unprectable?
random error
What is the term for the difference between T (true score) and X?
measurement error
systematic error:
A form of measurement error, where error is constant across trials.
random error:
measurement are due to chance and can affect a subjects score in an unpredictable way from trial to trial
What are 6 ways to avoid measurement error?
- clear operational definitions
- careful planning
- using detailed procedures
- being trained in procedures
- practice of specific procedures
- inspection of equipment
regression toward the mean:
A statistical phenomenon in which scores on a pretest are likely to move toward the group mean on a posttest because of inherent positive or negative measurement error; also called statistical regression.
variance:
a measure of the variability of differences among scores within a sample
The larger the variance the greater the _____.
dispersion of scores
The smaller the variance the more ____.
homogeneous the scores
You have measured dogs for height the heights (at the shoulders) are 600mm, 470mm, 170mm, 430mm, and 300mm. Find the mean, variance, and standard deviation.
mean: 394
variance: 21,704 (all answers squared / # of answers)
standard deviation: 147 (square root of variance)
with zero error the ratio will produce what coefficient?
1.00
coefficient:
a number used to multiply a variable
reliability coefficient of <.50 is:
poor
reliability coefficient of .50-.75 is:
moderate
reliability coefficient of >.75 is:
good
test-retest reliability:
- stability of the instrument
- repeated administration of test
- administered by one rater
- one sample tested repeatedly
- time intervals are important
- depends on stability of what’s being measured
- assumption is that what is being measured remains the same
- any changes seen is then due to random error
intrarater reliability:
1 individual over 2 or more trials. Usually over short interval of time
- Statistics used ICC model 2 or 3
interrater reliability:
The degree to which two or more raters can obtain the same ratings for a given variable.
Greater than or equal to 2 raters measuring same subject group. Best done during a single trial. Important for generalizability.
- Statistics used ICC model 2 or 3
internal consistency:
-
This is the homogeneity of items on a test. Reflexts extent to which items measure various aspects of a characteristic. Important when developing physical performance measures. Looks at correlation of all items on the test.
Possible statistics used are Cronbach’s Coefficient Alpha, Spearman-Brown Prophecy Statistic, Item-to-Total Correlation.
What are the carryover effects of test-retest?
training & motivation
What are the testing effects of test-retest
pain ensues & soft tissue stretch
What are the statistics used for test-retest?
intraclass correlation coefficient (ICC) model 3 & kappa statistics (nominal data)
What are examples of alternate forms of reliability? What are the statistics used?
- exists in 2 or more versions of a test
- equivalent, parallel, or alternate forms
- 2 alternate forms administered to the same group in 1 setting
- will look at correlation between paired observations
- needed to generalize findings from one situation to another
statistics used are limits of agreement & correlation coefficients
What are 6 factors that affect reliability?
- things internal to participant
- time between testing
- testing period circumstances
- appropriate level of difficulty
- precision of measurement
- environmental conditions
What is the standard error of measurement (SEM)?
Relates to the reliability of the measurement. The degree to which an observed score fluctuates due to errors of measurement. Is the standard deviation of the errors of measurement around the observed score.
true score + random error = actual score
classical reliability theory
true score + random error + test-retest error + rater error + other error sources = actual score
generalizability theory (Cronbach)
What is the minimal detectable difference?
The amount of change in a variable that must be achieved to reflect a true difference. The greater the reliability the smaller the MDD.
ex) pain scale- change of 2 is MDD. MMT- a full grade, not plus and minus.
pilot testing:
a smaller sample to test reliability of measurement
