Outcome Measures- Reliability and Validity Flashcards
OUTCOME MEASURES
OUTCOME MEASURES
Outcome measures can be both ________ and _________. List examples of each.
Objective
-ROM and length, Temp, Weight, Height, BP, HR, Strength
Subjective
-Pain, Intelligence, Health, Burnout
Measurement is the process of assigning _________ to variables to represent quantities of characteristics by rules. They are the actual results of implementing the ____ that indicate the impact on functioning.
- numerals
- POC
Very few variables are measured _________, most are not directly observable.
directly
________ as a _________ _________ are often cheaper and easier to measure than ‘true’ endpoints (ie. it is easier to measure a patient’s blood pressure than to use EKG).
biomarkers as a surrogate endpoint
What is a biomarker?
Objective, quantifiable characteristics of biological process
What is a surrogate endpoint?
A substitute for clinically meaningful endpoint
Psychometric properties to consider to make selection of right measures?
- Reliability
- Validity
- Scale of measurement
- Self-report vs. Performance-based measure
- Error parameters (minimal detectable change)
- Clinical utility of the measures
What are the 2 most important properties to consider before choosing measurement?
reliability and validity
Reliability = \_\_\_\_\_\_\_\_\_\_ Validity = \_\_\_\_\_\_\_\_\_\_
- consistency
- accuracy
_________ is an extent to which a measurement is consistent and free from error. It has consistency time after time with as little variation as possible.
Reliability
Many reliability coefficients are based on measures of correlation. What is correlation?
The degree of association between two sets of data
Draw a _______ plot to visualize correlation, and compute _______ _________ (r) to quantify it.
- scatter
- correlation coefficient
What are the 4 major types of reliability?
- ) Test-retest reliability
- ) Intrarater reliability
- ) Interrater reliability
- ) Internal consistency
What is test-retest reliability?
consistency of repeated measurements that are separated in time
What is intrarater reliability?
consistency of repeated measurements by the same PT practitioner at different times
What is interrater reliability?
- consistency of repeated measurements by multiple PT practitioners
- important to be good unless the usefulness of that test is limited
What is internal consistency?
consistency over multiple items or parts in a test, where each part is supposed to measure one, and only one, concept
_______ is the degree to which a test is measuring what it is intended to measure.
Validity
What are the 3 major types of validity?
- Face validity
- Content validity
- Construct validity
What is face validity?
- Does the content of the test appear to be suitable to its aims?
- the weakest form of the validity
- inspection of items for proper selections made by non-experts
What is content validity?
- Is the test fully representative of what it aims to measure?
- inspection of items for proper selections made by experts
What is construct validity?
- Does the test measure the concept that it is intended to measure?
- correlation and factor analysis to check
- What is the weakest form of validity?
- What is the strongest?
- face
- construct
What are the 4 scales of measurement in order based on their relative precision of assigned values?
Ratio -distance, age, time, decibals, weight Interval -calender years, C or F Ordinal -MMT, function, pain Nominal -gender, blood type, diagnosis
- The categorical scale combines _______ and _________.
- The continuous scale combines _______ and _________.
- nominal and ordinal
- interval and ratio
________ is an ordered system for a series of questions that provide an overall rating to distinguish among people who demonstrate different intensities measured and is developed to measure attitudes.
Scale
What are the 3 types of scales and what are they used for?
- ) Continuous
- used with continuous variables such as age; blood pressure; years of experience - ) Categorical (nominal)
- used with categorical variables such as gender; race; diagnosis - ) Ordinal
- used with ordinal variables such as pain measured by minimal; moderate; severe or function by independent; minimal assist; moderate assist; maximal assist; dependent
List 3 common scales used and what type of scale they are.
- Dichotomous Scale = categorical (nominal) two point scale
- 5-Point Likert Scale = categorical (ordinal) 5 point scale
- Visual Analog Scale (VAS) = continuous scale with verbal anchors at either end of the scale
___________ _______ Outcome Measurement is when the patient is required to perform a set of functional tasks.
Performance Based
______-___________ Outcome Measurement when the patient is required to complete a questionnaire.
Self-Reported
Is the Functional Independence Measure (FEM) a performance based or self-reported outcome measure?
Performance Based
Is the Oswestry Low Back Pain Disability Index (ODI) a performance based or self-reported outcome measure?
-Self-Reported
______ ________quantify errors in measurements and are not as widely reported as the reliability or validity are.It is important to determine the amount of error associated with a patient’s outcome measure
because it always comes with some imprecision.
Error Parameters
What is Standard Error of Measurement (SEM)?
The amount of variability that can be attributed to measurement error.
-SEM = SD * sqrt(1 - r)
What is Minimal Detectable Change (MDC)?
Minimal change that a patient must show on the scale to ensure that the observed change is real and not just measurement error.
-MDC = 1.96 * SEM * sqrt(2)
Clinical Utility Considerations:
- __________ of the test for application at the level of pathology or health condition, body function or structure, activity, or participation
- precision of the test to accurately measure change
- _________ of the test to the individual’s situation
- acceptability of the test to the individual
- ____ and _____ of administering the test
- appropriateness
- interpretability
- time and cost
What are 7 common outcome measures used in PT?
- Timed Up and Go test (TUG)
- Tinetti Balance Test (Tinetti test)
- Berg Balance Scale (BBS)
- Six Minute Walk Test (SMWT)
- Functional Reach Test (FRT)
- Oswestry Low Back Pain Disability Index (ODI)
- Functional Independence Measure (FIM)
- The TUG test is a measure of function with correlates to ________ and _____ risk.
- What is the outcome measure of this test?
- balance and fall
- the time it takes to stand up from chair and walk 3m
- The Tinetti Balance Test is a measure of ________ and ______.
- What is the outcome measure of this test?
- balance and gait
- number of chronic disabilities
- The Berg Balance Scale (BBS) is a measure of ________.
- What is the outcome measure of this test?
- balance
- total score computed out of maximum 56
- The Six Minute Walk Test (6MWT) is a measure of ________ capacity and ____________.
- What is the outcome measure of this test?
- aerobic capacity and endurance
- distance covered over a time of 6 minutes
- The Functional Reach Test (FRT) is a measure of __________.
- What is the outcome measure of this test?
- balance
- measures the distance between the length of an outstretched arm in maximal forward reach from a standing position, while maintaining a fixed BOS
- The Oswestry Low Back Pain Disability Index (ODI) is a measure of permanent ____________ disability.
- What is the outcome measure of this?
- functional
- total score
- Functional Independence Measure (FIM) is a measure of individual’s physical, psychological, and ______ function.
- What is the outcome measure of this?
- social
- total score
Summary for Outcome Measures:
- ____________ in a clinical research study is the process of assigning numerals to variables to represent quantities of characteristics by rules is the actual results of implementing the plan of care that indicate the impact on functioning.
- Most measurements are not directly observable, and we can measure only a correlate of the actual property, that we name the _________________________.
- What are 7 common outcome measures in PT?
- What are the 6 psychometric properties for a research study?
- Measurement
- indirect nature of measurement
- Timed Up and Go test (TUG), Tinetti Balance Test (Tinetti test), Berg Balance Scale (BBS), Six Minute Walk Test (SMWT) , Functional Reach Test (FRT), Oswestry Low Back Pain Disability Index (ODI), and Functional Independence Measure (FIM)
- reliability, validity, scale of measurement, self-report vs. performance-based measure, minimal detectable change, clinical utility of the measures
RELIABILITY AND VALIDITY CONT.
RELIABILITY AND VALIDITY CONT.
Reliability does not imply validity but is a _____________ for validity.
prerequisite
Reliability is fundamental to all aspects of measurement because with it we can have ___________ in the data we collect and we can draw rational __________ from the data.
- confidence
- conclusions
Many reliability coefficients are based on measures of correlation. What is correlation?
The degree of association between two sets of data
- _____________ visualized correlation.
- _________________ quantifies correlation.
- Scatter Plot
- Correlation Coefficient (r)
Correlation Coefficient (r) is a measure of linear association between two variables. It is a decimal number between __ and __.
-1 and +1
A Correlation Coefficient (r) of 0 means what? -1 means what? 1 means what?
0 = no correlation -1 = perfect negative correlation \+1 = perfect positive correlation
The ______ of (r) indicates the direction of correlation and the ___________ of (r) indicates the strength of the correlation.
- sign
- absolute value
What is Cohen’s kappa (k)?
A measure of agreement between two categorical values
What is the formula for Cohen’s kappa (k)?
k = (Pr(o) – Pr(e)) / (1 – Pr(e))
Where
Pr(o): probability of the relative observed agreement among two examiners
Pr(e): probability of the expected agreement just by chance among two examiners
Pr(o) = probability of the relative ___________ agreement among two examiners
Pr(e): probability of the ___________ agreement just by chance among two examiners
- oberved
- expected
Cohen’s kappa (k) is a decimal number between 0 and +1 with 0 being ______ agreement and 1 being _______ agreement.
- no
- perfect
Cohen’s kappa (k):
- 0-.4 = _______
- .4-.75 = __________
- .75-1.0 = __________
- poor
- fair to good
- excellent
What are the 4 major types of reliability?
- ) Test-retest reliability
- ) Intrarater reliability
- ) Interrater reliability
- ) Internal consistency
What are the 3 major types of validity?
- ) Face validity
- ) Content validity
- ) Construct validity
_________________ Analysis is a commonly used method to investigate construct validity. It is a statistical procedure used to test how well the measurement represent the number of constructs.
Confirmatory Factor
__________ Analysis s a commonly used method to investigate construct validity. It is a statistical procedure used to classify a group of subjects on the basis of a set of measured variables into a number of different groups
Cluster
Reliability and Validity Summary:
- Reliability is ___________ time after time, with as little variation as possible.
- Validity is __________ that a test is measuring what it is intended to measure.
- Types of reliability includes test-retest, _________, _________, and internal consistency
- Types of validity includes face, _________, and __________
- Reliability can be estimated using correlation coefficient (r) for two ___________ measurements and Cohen’s kappa (k) for two _____________ measurements
- Internal consistency reliability can be estimated using __________ alpha
- Construct validity can be estimated using correlation, confirmatory factor analysis, and cluster analysis
- consistency
- accuracy
- intrarater, interrater
- content, construct
- continuous, categorical
- Cronbach’s
