Outcome measures/reliability and validity (3) Flashcards
actual results of implementing the plan of care that indicate the impact on functioning?
measurement
Outcome measure include: (7)
- Health of functional status (activity limitations and participation restrictions)
- Impairments of body function and structures
- Adverse outcomes and complications
- Morbidity and mortality
- The individual’s self-reported outcomes
- The individual’s satisfaction with the care received
- Changes in health, wellness, and fitness
T/F most measurements are not directly observable
True, much harder for abstract terms (intelligence, health, strength, pain)
Biomarker as a surrogate endpoint -
ex:
- objective, quantifiable characteristics of biological process
- surrogate endpoint - a substitute for clinically meaningful endpoint
ex: easier to measure bp than use EKG but easier to measure EKG than morbidity/mortality in long term
T/F test for an outcome measurement should be checked for its reliability and validity.
True
Reliability: Consistency
Validity: Accuracy
Reliability -
- extent to which a measurement is consistent and free from error (with as little variation as possible
- reproducibility / dependability
What makes a reliable PT practitioner?
one who will be able to measure repeated outcomes with consistent scores
What makes a reliable test?
one that will perform with predictable consistency under set conditions
Why is reliability fundamental to all aspects of measurement?
- we can have confidence in the data we collect
2. we can draw rational conclusion from those data
Many reliability coefficients are based on measure of correlation, what does that mean?
correlation – the degree of association between two sets of data
How do you measure reliability?
- Draw scatter plot to visualize it
2. compute correlation coefficient (r) to quantify it (larger the r the more correlated)
test-retest reliability
how to check?
- consistency of repeated measurements that are separated in time stability of a measurement
- kappa or correlation coefficient to check
intrarater reliability
how to check?
- consistency of repeated measurements by the same PT practitioner at different times
- kappa or correlation coefficient to check
interrater reliability
how to check?
- consistency of repeated measurements by the multiple PT practitioners important to be good unless the usefulness of that test is limited
- kappa or correlation coefficient to check
internal consistency
how to check?
- consistency across parts of measurements
2. Cronbach’s alpha to check
Validity -
degree to which a test is measuring what it is intended to measure
face validity
Who inspects item?
- Does the content of the test appear to be suitable to its aims?
- the weakest form of the validity
- inspection of items for proper selections made by non-experts
content validity
Who inspects item?
- Is the test fully representative of what it aims to measure?
- inspection of items for proper selections made by experts
construct validity
How do you check this?
- Does the test measure the concept that it is intended to measure?
- correlation, factor analysis, or cluster to check
- Strongest
Confirmatory factor analysis -
- Commonly used method to investigate construct validity
2. How well measurement represent the number of constructs
Cluster analysis -
- Commonly used method to investigate construct validity
2. Classify a group of subjects on the basis of a set of measured variables into a number of different groups
What is an ordered system for a series of questions that provide an overall rating?
Scale
Continuous scales
- Ratio - rep units with equal intervals measure from true zero (distance, age, time)
- Interval - rep units with equal intervals but no true 0 (calendar years, temp)
- used with continuous variables such as age; blood pressure; years of experience
Categorical scales
- ordinal - numbers indicate rank order (MMT, pain)
- nominal - numerals are category labels (gender, blood type)
- used with categorical variables such as gender; race; diagnosis
Ordinal scales
used with ordinal variables such as:
- pain measured by minimal; moderate; severe
- function by independent; minimal assist; moderate assist; maximal assist; dependent
Dichotomous Scale
ex:
- categorical
- a two-point scale;
ex: Yes/No; True/False
5-point Likert Scale
ex:
- ordinal
- a five-point scale;
ex: Frequency: Always/Often/Sometimes/Rarely/Never
Quality: Excellent/Good/Fair/Poor/Very Poor
Visual Analogue Scale (VAS)
- continuous
- a line with verbal anchors at either end and subject places a mark at a point on the line corresponding to the subject’s rating
what is a Performance Based Outcome Measurement?
ex:
when the patient is required to perform a set of functional tasks
ex: Functional Independence Measure (FIM) - which assesses the patient’s ability function
what is a Self-Reported Outcome Measurement?
ex:
when the patient is required to complete a questionnaire
ex: Oswestry Low Back Pain Disability Index (ODI) - which measures the patient’s permanent functional disability including
Error parameters -
- quantify errors in measurements
- not as widely reported as the reliability or validity are
- important to determine the amount of error associated with a patient’s outcome measure because it always comes with some imprecision
Standard Error of Measurement (SEM) -
the amount of variability that can be attributed to measurement error
Minimal Detectable Change (MDC) -
minimal change that a patient must show on the scale to ensure that the observed change is real and not just measurement error
Clinical utility considerations -
- appropriateness of the test for application at the level of pathology or health condition, body function or structure, activity, or participation
- precision of the test to accurately measure change
- interpretability of the test to the individual’s situation
- acceptability of the test to the individual
- time and cost of administering the test
Timed Up and Go test (TUG)
- a measure of function with correlates to balance and fall risk
- outcome: the time taken to stand up from a standard arm chair, walk a distance of 3 meters
- Scale of measurement - continuous scale as it is recorded with the time
- performance-based
Tinetti Balance Test (Tinetti test)
- a measure of balance and gait
- outcome: based on the number of chronic disabilities
- Scale of measurement - continuous scale as it is recorded with the number of chronic disabilities
- performance-based
Berg Balance Scale (BBS)
- a measure of balance
2. outcome: total score computed out of maximum = 56
T/F Reliability implies validity
False, Reliability does not imply validity
T/F Reliability is a prerequisite for validity
True
How to interpret correlation coefficient r -
Correlation – the degree of association between two sets of data
- a measure of linear association between two variables
- a decimal number somewhere between -1 and +1
- sign of r indicates the direction of the correlation
- absolute value of r indicates the strength of the correlation
Cohen’s kappa (k) -
Pr(o)-Pr(e)/ 1 - pr(e)
- a measure of agreement between two categorical variables
where
Pr(o): probability of the relative observed agreement among two examiners
Pr(e): probability of the expected agreement just by chance among two examiners
How to interpret Cohen’s kappa (k)?
- a decimal number somewhere between 0 and +1
- 1 = perfect agreement
- 0 = no agreement
