Chapter 10: Validity

Question 1

What is validity?

Answer 1

Validity relates to the confidence we have that our measurement tools are giving us accurate information about a relevant construct so that we can apply results in a meaningful way.

Question 2

What questions are addressed by validity?

Answer 2

Is a test capable of discriminating among individuals with and without certain traits, diagnoses, or conditions?
Can the test evaluate the magnitude or quality of a variable or the degree of change from one time to another?
Can we make useful and accurate predictions about a patient’s future function or status based on the outcome of a test?

Question 3

How are validity and reliability different?

Answer 3

Reliability relates to consistency of a measurement
Validity relates to alignment of the measurement with a targeted construct; i.e., can inferences be made?

Question 4

How are validity and reliability similar?

Answer 4

Do not consider as all-or-none
Not an immutable characteristic of the instrument itself

Question 5

What are the three types of evidence for validity?

Answer 5

Content validity
Criterion-related validity
Construct validity

Question 6

What is content validity?

Answer 6

Establishes that the multiple items that make up a questionnaire defines the construct being measured.
The items must adequately represent the full scope of the construct being studied.
The test should not contain irrelevant items.

Question 7

What is criterion validity?

Answer 7

Establishes the correspondence between a target test and a reference or “gold” standard measure of the same construct.
Comparison of the results of a test to an external criterion
- Index test
- Gold or reference standard as the criterion
Two types
- Concurrent validity
- Predictive validity

Question 8

What is construct validity?

Answer 8

Establishes the ability of an instrument to measure the dimensions and theoretical foundation of an abstract construct.
- Assessing presence of a latent trait
Methods of construct validation
- Known-groups method
- Convergence and divergence
- Factor analysis

Question 9

What is face validity?

Answer 9

The implication that an instrument appears to test what is intended to test
Not the same as content validity
Face validity is a judgment by the users of a test after the test is developed
Content validity evolves out of the process of planning and constructing a test

Question 10

What’s a norm-referenced test?

Answer 10

a standardized assessment designed to compare and rank individuals within a defined population

Question 11

What is a criterion-referenced test?

Answer 11

test that is interpreted according to a fixed standard that represents an acceptable level of performance

Question 12

Change scores are use to…

Answer 12

Demonstrate effectiveness of an intervention
Track the course of a disorder over time
Provide a context for clinical decision making

Question 13

What are important strategies to use as one approaches validity studies?

Answer 13

Fully understand the construct
Consider the clinical context
Consider several approaches to validation
Consider validity issues if adapting existing tools
Cross-validate outcomes

Question 14

What is concurrent validity?

Answer 14

the extent to which the target test correlates with a reference standard taken at relatively the same time

Question 15

What is predictive validity?

Answer 15

the extent to which the target test can predict a future reference standard

Question 16

What is convergent validity?

Answer 16

the extent to which a test correlates with other tests of closely related constructs

Question 17

What is discriminant validity?

Answer 17

the extent to which a test is uncorrelated with tests of distinct or contrasting constructs

Question 18

What is confirmatory factor analysis?

Answer 18

CFA is used to support the structure of an instrument to determine whether it fits with empirical understanding of a construct.

Question 19

What is minimal detectable change?

Answer 19

Minimal detectable change (MDC), which is the minimum amount of measured change needed before we can rule out measurement error as being solely responsible.

Question 20

What is the minimal clinically important difference?

Answer 20

The smallest important difference measured in the patient’s condition.
It acts as an indicator of the instrument’s ability to reflect impactful changes.
The MCID reflects the test’s validity.

Question 21

What is a floor effect?

Answer 21

Not being able to show small differences when patients are near the bottom of the scale.
For instance, a developmental assessment with a floor effect might not be able to distinguish small improvements in children who are low functioning.

Question 22

What is a ceiling effect?

Answer 22

A patient who is functioning at a high level may show no improvement if a scale is not able to distinguish small changes at the top