Module 5: Validity and reliability; screening and diagnostic tests Flashcards
What are “constructs”?
Theoretical concepts are complex and cannot be directly observed (e.g., health status, quality of life, pain, depression, intelligence).
What is operationalization?
The process of measuring unobserved constructs using observable indicators.
Why are observable indicators helpful?
Indicators are our attempt to get at some truth. Even clinical conditions are based on observable indicators, e.g., high blood pressure
What is a nominal scale (categorical)? Provide two examples. What is the key property?
Uses numbers as labels for categories.
1 = Male; 2 = Female. Province of Residence
Key property is that A = B or A does not equal B.
What is an ordinal scale (ranks)? Provide two examples. What is the key property?
Uses numbers as labels for categories, but the numbers reflect ranked order of the characteristic being measured:
0 = no pain, 1 = a small amount of pain, …, 10 = the worst pain conceivable. 1 = lowest income group, 2 = middle income, 3 = highest income group.
Key property A < B < C. Higher numbers indicate more pain / income… distance from 1 to 2 not meaningful, does not relate to other distances (e.g., 2 to 3).
What is an interval scale (cardinal)? Provide an example. What is the key property?
Assigns numbers to response categories so that a unit change in scale values represents a constant change across the range of that scale.
Temperature in Celsius or Fahrenheit.
Key property is that A − B = (A + 10) − (B + 10). The distance from 20 to 30 is the same as the distance from 0 to 10. Ratios, however, don’t make sense: 40°C is not twice as hot as 20°C.
What is a ratio scale? Provide two examples. What is the key property?
A meaningful zero point exists.
Weight. Number of physician visits.
Key property is that A × B = C and C ÷ B = A. 10 visits are five times as many visits as 2 visits. A weight of 100 points is half the weight of 200 pounds.
What is random measurement error? What’s a term to summarize its extent?
The measure has random noise.
A term to summarize the extent the noise affects our ability to measure is “reliability” of an indicator.
What is systematic measurement error? What can these errors create?
An error that is rooted in study design / the measurement tool / the data-to-estimation process such that the estimate will always deviate from the true value, aka “bias”.
Systematic errors can create invalid results (“validity”), which means a study result is incorrect.
How is reliability calculated?
Variability of the true construct / Variability of the true construct + Measurement error
When we express reliability via the base calculation it is between 0 and 1. If the ratio = 1, then measurement error equals what? What does this imply?
Zero. We are perfectly measuring our construct. If this ratio = 0, then our measure is entirely measurement error.
A more reliable measure is less influenced by _____.
Error.
When we describe something as “reliable” we mean that σ2e is low or non-existent compared to σ2T, which is a theoretical concept. What do we need to do in practice?
Express/quantify reliability, indicated by reproducibility: does the same measurement produce the same result when repeated?
Test-retest reliability asks what?
Does the measurement produce the same result when the same rater makes a second assessment? (The same subject and the same measure by the same rater at different times.)
What are the consequences if a test-retest interval is too short? Too long? When is it appropriate?
Too short: the rater remembers first response.
Too long: the construct may change.
Appropriate interval: it depends (2-14 days?).
