2 - Measurement Properties Flashcards
Ratio
Equal intervals
True zero
Examples: age, time, distance, weight.
Interval
Equal intervals
No true zero
Examples: temperature, calendar years
What does true zero mean?
Absence of the thing being measured. For example, 0deg F doesn’t mean there’s no temperature, therefore not a true zero. But 0 sec means no time, therefore a true zero.
Ordinal
Numbers indicate ranking. Examples: MMT grades, pain scale
Nominal
Category labels. Examples: gender, blood type, diagnosis
What measurement properties are related to context?
Appropriateness
Acceptability
Feasibility
Sample characteristics
What measurement properties are related to tools/instruments?
Interpretability
Reliability
Validity
Responsiveness
Floor/Ceiling Effect
Dimensionality & Internal Consistency
Appropriateness addresses what questions?
Is the measure appropriate for intended use?
Who/what is being measured?
Why is the measurement being taken?
Acceptability addresses what question? Affected by what factors?
Is the measurement acceptable to pts?
Time, comfort (pain, exertion, emotion), language, cultural bias, cog
What factors affect feasibility?
Equipment, space, cost, time, personnel availability and compitency, calculation, documentation
Sample characteristics address what questions?
Has the measure been tested with the types of pts with whom it will be used? Is your pt similar to the study sample? Would the pt be eligible to participate in the study?
Random error
Unpredictable trial to trial.
Average of trials is closer to the true score.
Systemic error (bias)
Always off by the same amount.
Does not affect reliability.
Classical measurement theory
All error is due to random error
Generalizability measurement theory
Some error can be attributed to specific sources.
If you can quantify some sources of error, a smaller portion of error will be random.
Inter-rater reliability
Multiple raters, same trial
Test-Retest reliability
Same rater, same pt, different days (more time btwn trials than intra-rater reliability)
Common reliability statistic for nominal data
Kappa
Common reliability statistic for ordinal data
Weighted Kappa
Common reliability statistic for continuous (interval/ratio) data
ICC
Association vs Agreement
Association: correlation, y=mx+b. Does not assess systemic error.
Agreement: every x = y
What measurements assess association, but not agreement?
Spearman’s rho
Pearson’s correlation coefficient (r)
Percent agreement
What measurement assesses both association & agreement?
ICC (intra-class correlation)
What is a good ICC?
0.70 or higher (values range 0.00 to 1.00)
Can ICC be used for ordinal data?
Yes, if intervals are equidistant
What causes a low ICC?
Poor agreement in ratings and/or btwn raters.
Systemic error.
Sample too homogenous (e.g. basketball plays, all on upper range height, but varying weights)
What measurement measures agreement for categorical data?
Kappa
Kappa takes into account ____ and ____
Po: percent agreement.
Pc: chance agreements.
What does weighted kappa take into account?
Severity of disagreements
What is an acceptable kappa value?
0.70 or higher
Kappa > 80%
excellent agreement
Kappa > 60%
substantial agreement
Kappa = 40-60%
moderate agreement
Kappa < 40%
poor agreement
What causes low kappa values?
Low variability in responses.
Small sample size.
Greater number of choices on a nominal scale (more opportunity for disagreement)
Content validity
items that make up the questionnaire accurately measure the construct
Face validity
a type of content validity. surface level, appears to measure what it’s supposed to
Criterion validity
how well a measure compares to a gold standard
Reference standard
some acceptable level to be used as a criterion. May be used if no gold standard exists
Concurrent validity
a type of criterion validity. how much the test correlates with a reference standard taken at the same time
Predictive validity
a type of criterion validity. how much the test can predict a future reference standard. Example: practice exam predicts score on actual exam
Construct validity
how well a measure captures the dimensions and theoretical foundations of an abstract construct
Convergent validity
a type of construct validity. measure correlates well with related measures
Divergent or Discriminant validity
a type of construct validity. measure does NOT correlate well with UNrelated measures
Known Groups validity
a type of construct validity. can the measure detect differences btwn groups?
Structural or Factorial validity
a type of construct validity. Degree to which scores accurately reflect the dimensionality of the construct. Useful for multi-item tests. Are we covering all aspects without spending hours or asking 100 questions?
Cross-Cultural validity
a type of construct validity. Does the translated or cross-culturally adapted tool reflect the performance of the original tool?
