Chapter 11 - Measurement and Data Quality Flashcards
What is a measurement?
involves rules for assigning numbers to objects or people to designate the QUANTITY of an attribute
-numbers assigned according to rules
Advantages of Measurement
- removes guesswork/ambiguity in gathering and communicating info (ex. temp is measured in Fahrenheit)
- minimize subjectivity, precise information
- less vague than words (ex. 6ft 3in is more descriptive than “tall”)
Nominal Measurement
LOWEST LEVEL
- ->involves using numbers to simply categorize attributes
ex. gender, blood type - don’t have to have quantitative meaning
Ordinal Measurement
- ->ranks people based on relative standing on an attribute
ex. ADL ranking (1=completely dependent, 2=needs assist x 1, etc) - ordinal ranking cannot show that one ranking is twice as great as another
Interval Measurement
–>ranking people on an attribute and specify the distance between them
Ex. IQ testing (distance between 100-110 is the same at 120-130)
-no meaningful zero
Ratio Measurement
HIGHEST LEVEL
–>have a meaningful zero and thus provide info about the absolute magnitude of the attribute
Ex. weight (100 lbs is twice as much at 50 lbs and 0 lbs = no weight)
What is the purpose of different levels of measurement?
A variable’s level of measurement determines the mathematic operations that may be performed in a statistical analysis.
Calculation for Errors of Measurement
Obtained Score = True Score +/- Error
Obtained (observed) score vs. true score
Obtained: the measurement we take from the patient
True: score we would get if it was an infallible measure (hypothetical)
What contributes to measurement error?
- Situational contaminants - environmental factors (temp, lighting, time of day)
- Response-Set Bias - enduring characteristics of respondents
- Transitory Personal Factors - temporary states (fatigue, stress, hunger)
- Item Sampling - bias by sampling items chosen (score on 100 pt test depends on which 100 questions were asked)
Reliablity
CONSISTENCY with which an instrument measures the attribute
-also includes ACCURACY, statement is reliable to the extent that it captures true scores
ex. scale measuring person at 100 lbs one minute and 5 minutes later measuring them at 150 lbs = UNRELIABLE
Reliability Assessments
Reliability assessments involve computing a reliability coefficient
– Reliability coefficients ranges from .00-1.00.
– Coefficients below .70 are considered unsatisfactory.
– Coefficients of .80 or higher are desirable.
Stability
*Aspect of reliability to assess
the degree to which similar results are obtained on separate occasions
- affected by time related influences (fatigue)
- assessed through TEST-RETEST RELIABILITY
Reliability Coefficient: quantifies an instruments reliability to assess objectively how small the differences are
-range from .00 to 1.00 (higher = more stable)
Internal Consistency
*Aspect of reliability to assess
the extent to which an instrument’s items measure the same trait
Coefficient (Cronbach’s Alpha) Alpha: how internal consistency is evaluated
-range from .00-1.00 (higher = more internally consistent)
Equivalence
*Aspect of reliability to assess
concerns the degree to which two or more independent observers/coders agree about scoring on an instrument
-high agreement = minimized measurement errors
Interrater (Interobserver) Reliability: two or more observers/coders make independent observations
–>evaluate congruency between ratings (more congruency = greater accuracy/reliability)
What is the purpose of reliability coefficients?
Indicators of data accuracy and quality - critical in interpreting research results (especially if hypotheses are not supported)
Effected by:
- Sample variability (more homogeneous = lower coefficient)
- -> scales are designed to measure differences, less differentiation = difficult to discriminate reliably
- ->want a heterogeneous and long-term/multi-item scale - Type of procedure (results may be different if procedures are not identical)
What is validity?
The degree to which an instrument measures what it is supposed to measure
*To be valid, a measuring device MUST be reliable (but a measuring device can be reliable WITHOUT being valid)
-easier to document than reliability
Face validity
whether an instrument looks as though it is measuring the appropriate construct
Content Validity
The degree to which an instrument has an appropriate sample of items for the contract being measured - want to capture the FULL content domain
-based on judgment (subjective)
Content Validity Index (CVI)
*assessment of content validity
a calculation that indicates the extent of expert agreement
-CVI of .90 or higher are standard for excellence
Criterion Related Validity
researchers examine the relationship between scores on an instrument and an external criterion
–>an instrument is valid if it’s scores correspond strongly with scores on the criterion
-assures decision-makers that their decisions will be fair, appropriate, and valid
Validity Coefficient
Computed with a mathematical formula to correlate the two scores
- magnitude of scores = estimate of instruments validity
- ->.70 or greater is desirable
Predictive validity
an instrument’s ability to differentiate between people’s performances on FUTURE criterion
–>ex. high school grades compared to college GPA
Concurrent validity
an instrument’s ability to distinguish among people who differ presently on criterion
–>ex. whether a pt could be released from a mental health hospital correlated with nurses’ behaviors at that time
Construct Validity
KEY for assessing research quality
- ->what is the instrument really measuring? does it validly measure the abstract concept of interest?
- hypothesis-testing
- explained in terms of other constructs - how it will function in relation to another construct
- OBJECTIVE criterion to compare a measure to
Known-groups technique
- approach to construct validation
- ->groups that are EXPECTED to differ on the target attribute are administered the instrument and group scores are compared
- ->ex. first time mothers are more likely to be anxious about birth than mothers who aren’t (compare both groups with “known” outcomes)
Factor analysis
- approach to construct validity
- ->method for identifying clusters of related items for a scale
- groups together different measures into a unitary scale based on how participants reacted to items (not researcher’s preconceptions)
Sensitivity
ability of a measure to correctly identify a “case”/to correctly screen in or diagnosis a condition
- rate of yielding TRUE POSITIVES
- ->ex. teens who said they did smoke and a positive UA DIVIDED by all real smokers detected by UA
Specificity
ability of a measure to correctly identify non cases, to screen out those without the condition
- rate of yielding TRUE NEGATIVES
- ->ex. teens who said they did not smoke and had a negative UA DIVIDED by all the nonsmokers detected by UA
