2 - Measurement Properties

Question 1

Ratio

Answer 1

Equal intervals
True zero
Examples: age, time, distance, weight.

Question 2

Interval

Answer 2

Equal intervals
No true zero
Examples: temperature, calendar years

Question 3

What does true zero mean?

Answer 3

Absence of the thing being measured. For example, 0deg F doesn’t mean there’s no temperature, therefore not a true zero. But 0 sec means no time, therefore a true zero.

Question 4

Ordinal

Answer 4

Numbers indicate ranking. Examples: MMT grades, pain scale

Question 5

Nominal

Answer 5

Category labels. Examples: gender, blood type, diagnosis

Question 6

What measurement properties are related to context?

Answer 6

Appropriateness
Acceptability
Feasibility
Sample characteristics

Question 7

What measurement properties are related to tools/instruments?

Answer 7

Interpretability
Reliability
Validity
Responsiveness
Floor/Ceiling Effect
Dimensionality & Internal Consistency

Question 8

Appropriateness addresses what questions?

Answer 8

Is the measure appropriate for intended use?
Who/what is being measured?
Why is the measurement being taken?

Question 9

Acceptability addresses what question? Affected by what factors?

Answer 9

Is the measurement acceptable to pts?
Time, comfort (pain, exertion, emotion), language, cultural bias, cog

Question 10

What factors affect feasibility?

Answer 10

Equipment, space, cost, time, personnel availability and compitency, calculation, documentation

Question 11

Sample characteristics address what questions?

Answer 11

Has the measure been tested with the types of pts with whom it will be used? Is your pt similar to the study sample? Would the pt be eligible to participate in the study?

Question 12

Random error

Answer 12

Unpredictable trial to trial.
Average of trials is closer to the true score.

Question 13

Systemic error (bias)

Answer 13

Always off by the same amount.
Does not affect reliability.

Question 14

Classical measurement theory

Answer 14

All error is due to random error

Question 15

Generalizability measurement theory

Answer 15

Some error can be attributed to specific sources.
If you can quantify some sources of error, a smaller portion of error will be random.

Question 16

Inter-rater reliability

Answer 16

Multiple raters, same trial

Question 17

Test-Retest reliability

Answer 17

Same rater, same pt, different days (more time btwn trials than intra-rater reliability)

Question 18

Common reliability statistic for nominal data

Answer 18

Kappa

Question 19

Common reliability statistic for ordinal data

Answer 19

Weighted Kappa

Question 20

Common reliability statistic for continuous (interval/ratio) data

Answer 20

ICC

Question 21

Association vs Agreement

Answer 21

Association: correlation, y=mx+b. Does not assess systemic error.
Agreement: every x = y

Question 22

What measurements assess association, but not agreement?

Answer 22

Spearman’s rho
Pearson’s correlation coefficient (r)
Percent agreement

Question 23

What measurement assesses both association & agreement?

Answer 23

ICC (intra-class correlation)

Question 24

What is a good ICC?

Answer 24

0.70 or higher (values range 0.00 to 1.00)

Question 25

Can ICC be used for ordinal data?

Answer 25

Yes, if intervals are equidistant

Question 26

What causes a low ICC?

Answer 26

Poor agreement in ratings and/or btwn raters.
Systemic error.
Sample too homogenous (e.g. basketball plays, all on upper range height, but varying weights)

Question 27

What measurement measures agreement for categorical data?

Answer 27

Kappa

Question 28

Kappa takes into account ____ and ____

Answer 28

Po: percent agreement.
Pc: chance agreements.

Question 29

What does weighted kappa take into account?

Answer 29

Severity of disagreements

Question 30

What is an acceptable kappa value?

Answer 30

0.70 or higher

Question 31

Kappa > 80%

Answer 31

excellent agreement

Question 32

Kappa > 60%

Answer 32

substantial agreement

Question 33

Kappa = 40-60%

Answer 33

moderate agreement

Question 34

Kappa < 40%

Answer 34

poor agreement

Question 35

What causes low kappa values?

Answer 35

Low variability in responses.
Small sample size.
Greater number of choices on a nominal scale (more opportunity for disagreement)

Question 36

Content validity

Answer 36

items that make up the questionnaire accurately measure the construct

Question 37

Face validity

Answer 37

a type of content validity. surface level, appears to measure what it’s supposed to

Question 38

Criterion validity

Answer 38

how well a measure compares to a gold standard

Question 39

Reference standard

Answer 39

some acceptable level to be used as a criterion. May be used if no gold standard exists

Question 40

Concurrent validity

Answer 40

a type of criterion validity. how much the test correlates with a reference standard taken at the same time

Question 41

Predictive validity

Answer 41

a type of criterion validity. how much the test can predict a future reference standard. Example: practice exam predicts score on actual exam

Question 42

Construct validity

Answer 42

how well a measure captures the dimensions and theoretical foundations of an abstract construct

Question 43

Convergent validity

Answer 43

a type of construct validity. measure correlates well with related measures

Question 44

Divergent or Discriminant validity

Answer 44

a type of construct validity. measure does NOT correlate well with UNrelated measures

Question 45

Known Groups validity

Answer 45

a type of construct validity. can the measure detect differences btwn groups?

Question 46

Structural or Factorial validity

Answer 46

a type of construct validity. Degree to which scores accurately reflect the dimensionality of the construct. Useful for multi-item tests. Are we covering all aspects without spending hours or asking 100 questions?

Question 47

Cross-Cultural validity

Answer 47

a type of construct validity. Does the translated or cross-culturally adapted tool reflect the performance of the original tool?