Session 8 - Validity

Question 1

Internal Validity

Answer 1

The study measures what it’s supposed to measure and its conclusions accurately reflect the studied relationship.

Question 2

A study that is internally valid is one that has

Answer 2

minimized error
minimized alternative explanations for the findings (chance, bias, confounding)
drawn appropriate conclusions from the data obtained.

Question 3

Is a study that includes a diagnostic test valid if you have not confirmed that the test you are using in valid?

Answer 3

NO!

Question 4

Content Validity

Answer 4

the extent to which measure is judged to reflext the underlying concept (construct) of the measure, without also including irrelevant or extraneous elements.

Question 5

Criterion-Related Validity

Answer 5

Compared to criterion or some “gold standard”
Concurrent (done simultaneously)
Predictive (big in PT) the extent to which the measure being taken is associated with some future measure or event.

Question 6

problem with validation of new tests on gold standard

Answer 6

new tests can appear to perform poorly compared to standard (but weaker) traditional tests.

Question 7

Construct Validity

Answer 7

has to do with identifying the construct or theory being tested. this is largely used because there is no standard available for criterion validity

Question 8

Face Validity

Answer 8

BECAUSE!

Question 9

What does a validity statistic compare?

Answer 9

Two DISSIMILAR ways of measuring an outcome rather than comparing the same measurement between two testers or over two different times.

Question 10

allows assessment of association between two dissimilar measures with or without similar units of measure

Answer 10

Pearson Product Moment Correlation Coefficient.

Question 11

what does Cronbach’s Alpha asses?

Answer 11

Internal Consistency.
the degree to which items in a scale correlate to each other or the total score. therefore it appears more related to the construct of the scape than to it’s reliability.
how well items are correlated to each other or to the total score.

Question 12

What could a very strong Cronbach’s alpha indicate?

Answer 12

an item may actually be redundant and the scale can be streamlined.

Question 13

Nonparametric correlation coefficients

Answer 13

Spearman rank order correlation coefficient (ordinal data)- most common of non parametrics
contingency coefficient ( dichotomous data)
Odds Ratio (dichotomous data)
Kendall coefficient of concordance (ordinal data)
Sensitivity, specificity , predicitve value and likelihood ratios

Question 14

Pitfalls in ascertaining test performance characteristics

Answer 14

we don’t have a gold standard for most PT_related medical diagnoses.
tests are frequently tested on a different population than they will later be used on.
Statistically significant tests
given testing conditions, the prevalence is already atypical of what exists in the real population .

Question 15

Responsiveness

Answer 15

the degree to which the test score can be expected to change when people actually change (sensitivity to change)

Question 16

Responsiveness is generally considered to be “true change” and is therefore tied to the concept of what?

Answer 16

SEM and MDC
it cannot be true change unless the score changes by more than random error.
Responsiveness Indices

Question 17

(values that “quantify” a test’s ability to measure change over time

Answer 17

Effect size index
standard response mean
interpretation
(required parametric data since based on mean and SD)

Question 18

Ceiling effect

Answer 18

if a lot of patients on whom you wish to measure change score high on a test, then there is no room to increase the score with change in status

Question 19

Basement Effect

Answer 19

if a lot of patients on whom you wish to measure change score low on a test, then there is no room to decrease the score with change in status.