Exam 1 Flashcards
Internal Consitency
*Tells you about a test
Consistency of a construct across individual items of an outcome measure.
Which psychometric property conducts the outcome measure on a group of people and analyze the intra-subject correlation between items?
Internal consistency
Which psychometric property uses statistical analysis of Chronbach alpha? What is the ideal?
Internal consistency
0.7-0.9
If it’s too low it’s not consistent enough to cover what you’re trying to test
If it’s too high the items are too similar.
Test-retest
*Tells you about the test
Consistency of the test when given to a person, unchanged in outcome, on two different occasions
(Taking a test on Mon then on Wed)
Which psychometric property gives the test to the same people on different days?
Test-retest
Which psychometric property want only one person (tester) b/c they want to keep the conditions between two testing situations to be similar as possible?
Test-retest
Which psychometric properties uses ICC (continuous) or Kappa (nominal)?
What number is preferred?
Test-retest
Intra-rater
Inter-rater
Close to 1 as possible!
> 0.80 excellent agreement
0.60 substantial agreement
0.40 moderate agreement
<0.39 poor agreement
What does a state of close to 1 indicate with test-retest?
Person had a very similar score as compared to when they took it originally.
Far away from 1 indicates the person had a very different score when they originally took it.
Intra-rater
*Tells you about the tester
Consistency of raters compared to themselves, on two different occasions.
How consistent is the tester to themselves?
Which psychometric property is it if a PT admins a test and you want to make sure that they would be able to determine consistent scores every time they administer the test.
Intra-rater
Which psychometric property is it if several PTs give the test to the same people at different time and the scores compared to their own scores to look for consistency?
Intra-rater
Inter-rater
*Tells you about the tester
Consistency of raters compared to each other
What psychometric property is it if multiple PTs were to admin a test and their scores are consistent between each other?
Inter-rater
Which psychometric property is it when PTs measure the same participants and their scores are compared?
Inter-rater
Reliability? And their tests?
Tells you about the test & testers. Can the test produce reliable results that you can trust?
Internal consistency
Test-retest
Intra-rater
Inter-rater
Validity? Tests?
Tells you if the test is testing what it is supposed to be testing.
Content Face Criterion Concurrent Predictive Construct Convergent Discriminative Known groups
Content validity?
Experts are actually looking at it. Includes all of the content required to cover the topic of interest your objective measure is covering.
Which psychometric property measures considers if everything that should be included in the outcome measure?
Content validity
Which psychometric property focus group of clinical experts are brought together and asked to determine everything that contributes to what your outcome measure is going to include?
Content validity
Face validity?
Is your own opinion. On the surface, did the test cover what it said it would? Does it look good to me.
*Less rigorous than true content validity
Which psychometric property is it when on the surface, did you feel that it covered the material that was overhead in class or was there something left out?
Face validity
Which validity is it when you determine it on your own experience and gut feeling?
Face validity
Criterion validity?
Is when you’re comparing your outcome measure to an established measure. Is your objective measure is just as good as the gold standard?
Does the Content and Face validity have statistical analyses?
No
Which validity tests uses the Spearman rho or Pearson correlation?
Criterion Concurrent Predictive Construct Convergent Discriminative
Which psychometric property measures the outcome measure of interest and established outcome measure on the same person?
Criterion validity
What stats do you want on the Spearman rho or Pearson correlation? What does it mean?
From -1 to 1
Ideally close to +1 as possible! Means the old test is close to the gold standard meaning you can use one or the other…
+1 Positive correlation
-1 Negative correlation
0 No correlation
> 0.85 strong correlation
0.60 moderate correlation
<0.39 poor correlation
If the Lachman’s has a Spearman rho of 1 for criterion validity what does it mean?
That the Lachman can be used instead of the MRI.
Concurrent validity?
Two measures correlate at the same time point.
If you are looking at ACL tests, you would want to compare your new outcome measure to the MRI (GS) at the same time to ensure that they are both testing the same thing at the same time. If we don’t do it at the same time we won’t know if it’s a true change…
Predictive validity
The outcome measure of interest correlated w/ another outcome measure at a later point.
The outcome measure measuring return to sports or fall risk.
Used for things that you’d have to wait and see if they came true…
Which psychometric property compares the outcome measure of the interest to a later outcome measure?
Predictive validity
Construct validity?
Does it measure the theoretical construct it is supposed to? Does the outcome measure what it is intended to measure?
(Intelligence, athletic performance capacity/potential)
Which psychometric property measures the outcome of interest and established gold standard or reference criterion?
Construct validity
Convergent validity
Does the outcome measure of interest correlate with another measure known to measure the same construct?
Which psychometric property is it if you want to measure strength w/ a new outcome measure, the results of a new outcome measure should correlate to other strength tests. If they score high on one they should score high on the others…If your new one is a true measure of strength.
Convergent validity
Discriminative validity
Does the outcome measure of interest NOT correlate w/ a measure known to measure a different construct
If you want to measure power, the results of your measure should be different than a measure measuring flexibility since they are different constructs…
Comparing performance on OCS exam vs EBP exam… Yes they take both, but measuring overall PT ability in treating ortho is OCS only…
Known groups?
Does the outcome measure of interest produce different results for groups of people known to be different on the outcome measure is supposed to be test?
Looking at scores of 1st year, 2nd year, 10 year PT…
Which validity measures distinctly different groups and look for differences in their scores?
Known groups
What type of statistically test is used for known groups validity?
Analysis of variance for linear trends (p-value)
Want <0.05 for significant difference between groups
Ceiling/Floor
Is it common for individuals to get to highest scores (ceiling effect) or lowest score (floor effect)?
If all people are scoring 100%, you won’t detect a change because they’ve maxed out…
IF all people got 0%, you might not have a sensitive enough test to detect a change even if there is one.
What psychometric property looks at measuring a diverse sample and look at the proportion that get highest/lowest possible scores?
Ceiling/Floor Effect
Skewness of ceiling/floor? (Numbers)
> 1 floor
Minimal Detectible Change (MDC)
Minimal amount of change required on an outcome measure to exceed anticipated measurement error and variability.
What score would the pt need to improve that isn’t just random day to day variability in testing and actually represents a true change?
What reliability test would occur before the MDC?
Test-retest reliability
Minimal Clinically Important Difference (MCID)
Minimum amount of change on an outcome measure patients re likely to perceive as beneficial.
Responsiveness
Outcome measures ability to detect (show) change over time.
Measure a pop at two time points, in between which you expect them to change.
What statistical analysis is done for responsiveness?
Effect size (d), did t have a significant on what you were looking at?
> 0.8 large
0.5 - 0.8 moderate
0.2 - 0.5 small
<0.2 trivial
