Chapter 11 - Measurement and Data Quality

Question 1

What is a measurement?

Answer 1

involves rules for assigning numbers to objects or people to designate the QUANTITY of an attribute
-numbers assigned according to rules

Question 2

Advantages of Measurement

Answer 2

• removes guesswork/ambiguity in gathering and communicating info (ex. temp is measured in Fahrenheit)
• minimize subjectivity, precise information
• less vague than words (ex. 6ft 3in is more descriptive than “tall”)

Question 3

Nominal Measurement

Answer 3

LOWEST LEVEL

• ->involves using numbers to simply categorize attributes
  ex. gender, blood type
• don’t have to have quantitative meaning

Question 4

Ordinal Measurement

Answer 4

• ->ranks people based on relative standing on an attribute
  ex. ADL ranking (1=completely dependent, 2=needs assist x 1, etc)
• ordinal ranking cannot show that one ranking is twice as great as another

Question 5

Interval Measurement

Answer 5

–>ranking people on an attribute and specify the distance between them
Ex. IQ testing (distance between 100-110 is the same at 120-130)
-no meaningful zero

Question 6

Ratio Measurement

Answer 6

HIGHEST LEVEL
–>have a meaningful zero and thus provide info about the absolute magnitude of the attribute
Ex. weight (100 lbs is twice as much at 50 lbs and 0 lbs = no weight)

Question 7

What is the purpose of different levels of measurement?

Answer 7

A variable’s level of measurement determines the mathematic operations that may be performed in a statistical analysis.

Question 8

Calculation for Errors of Measurement

Answer 8

Obtained Score = True Score +/- Error

Question 9

Obtained (observed) score vs. true score

Answer 9

Obtained: the measurement we take from the patient

True: score we would get if it was an infallible measure (hypothetical)

Question 10

What contributes to measurement error?

Answer 10

1. Situational contaminants - environmental factors (temp, lighting, time of day)
2. Response-Set Bias - enduring characteristics of respondents
3. Transitory Personal Factors - temporary states (fatigue, stress, hunger)
4. Item Sampling - bias by sampling items chosen (score on 100 pt test depends on which 100 questions were asked)

Question 11

Reliablity

Answer 11

CONSISTENCY with which an instrument measures the attribute
-also includes ACCURACY, statement is reliable to the extent that it captures true scores

ex. scale measuring person at 100 lbs one minute and 5 minutes later measuring them at 150 lbs = UNRELIABLE

Question 12

Reliability Assessments

Answer 12

Reliability assessments involve computing a reliability coefficient
– Reliability coefficients ranges from .00-1.00.
– Coefficients below .70 are considered unsatisfactory.
– Coefficients of .80 or higher are desirable.

Question 13

Stability

Answer 13

*Aspect of reliability to assess

the degree to which similar results are obtained on separate occasions

• affected by time related influences (fatigue)
• assessed through TEST-RETEST RELIABILITY

Reliability Coefficient: quantifies an instruments reliability to assess objectively how small the differences are
-range from .00 to 1.00 (higher = more stable)

Question 14

Internal Consistency

Answer 14

*Aspect of reliability to assess

the extent to which an instrument’s items measure the same trait

Coefficient (Cronbach’s Alpha) Alpha: how internal consistency is evaluated
-range from .00-1.00 (higher = more internally consistent)

Question 15

Equivalence

Answer 15

*Aspect of reliability to assess

concerns the degree to which two or more independent observers/coders agree about scoring on an instrument
-high agreement = minimized measurement errors

Interrater (Interobserver) Reliability: two or more observers/coders make independent observations
–>evaluate congruency between ratings (more congruency = greater accuracy/reliability)

Question 16

What is the purpose of reliability coefficients?

Answer 16

Indicators of data accuracy and quality - critical in interpreting research results (especially if hypotheses are not supported)

Effected by:

1. Sample variability (more homogeneous = lower coefficient)
   - -> scales are designed to measure differences, less differentiation = difficult to discriminate reliably
   - ->want a heterogeneous and long-term/multi-item scale
2. Type of procedure (results may be different if procedures are not identical)

Question 17

What is validity?

Answer 17

The degree to which an instrument measures what it is supposed to measure
*To be valid, a measuring device MUST be reliable (but a measuring device can be reliable WITHOUT being valid)

-easier to document than reliability

Question 18

Face validity

Answer 18

whether an instrument looks as though it is measuring the appropriate construct

Question 19

Content Validity

Answer 19

The degree to which an instrument has an appropriate sample of items for the contract being measured - want to capture the FULL content domain
-based on judgment (subjective)

Question 20

Content Validity Index (CVI)

Answer 20

*assessment of content validity
a calculation that indicates the extent of expert agreement
-CVI of .90 or higher are standard for excellence

Question 21

Criterion Related Validity

Answer 21

researchers examine the relationship between scores on an instrument and an external criterion
–>an instrument is valid if it’s scores correspond strongly with scores on the criterion

-assures decision-makers that their decisions will be fair, appropriate, and valid

Question 22

Validity Coefficient

Answer 22

Computed with a mathematical formula to correlate the two scores

• magnitude of scores = estimate of instruments validity
• ->.70 or greater is desirable

Question 23

Predictive validity

Answer 23

an instrument’s ability to differentiate between people’s performances on FUTURE criterion
–>ex. high school grades compared to college GPA

Question 24

Concurrent validity

Answer 24

an instrument’s ability to distinguish among people who differ presently on criterion
–>ex. whether a pt could be released from a mental health hospital correlated with nurses’ behaviors at that time

Question 25

Construct Validity

Answer 25

KEY for assessing research quality

• ->what is the instrument really measuring? does it validly measure the abstract concept of interest?
• hypothesis-testing
• explained in terms of other constructs - how it will function in relation to another construct
• OBJECTIVE criterion to compare a measure to

Question 26

Known-groups technique

Answer 26

• approach to construct validation
• ->groups that are EXPECTED to differ on the target attribute are administered the instrument and group scores are compared
• ->ex. first time mothers are more likely to be anxious about birth than mothers who aren’t (compare both groups with “known” outcomes)

Question 27

Factor analysis

Answer 27

• approach to construct validity
• ->method for identifying clusters of related items for a scale
• groups together different measures into a unitary scale based on how participants reacted to items (not researcher’s preconceptions)

Question 28

Sensitivity

Answer 28

ability of a measure to correctly identify a “case”/to correctly screen in or diagnosis a condition

• rate of yielding TRUE POSITIVES
• ->ex. teens who said they did smoke and a positive UA DIVIDED by all real smokers detected by UA

Question 29

Specificity

Answer 29

ability of a measure to correctly identify non cases, to screen out those without the condition

• rate of yielding TRUE NEGATIVES
• ->ex. teens who said they did not smoke and had a negative UA DIVIDED by all the nonsmokers detected by UA