Exam 2 Chapter 5 Flashcards
Outcome
Characteristic of target population or social condition that a program is expected to change
Outcome Level
Status of an outcome at a certain point in time:
Resting heart rate
Blood pressure
Vocabulary
Knowledge of the alphabet
Outcome Change
Difference between outcome levels at different points in time:
Resting heart rate pre program vs. immediately post program
Number of vocabulary words before starting Pre-K vs. number of vocabulary words after Pre-K
Program Effect
Portion of an outcome change that can attributed uniquely to a program as opposed to the influence of some other factor
Reliability
consistency or dependability (across time, items, or judges)
Three Types of Reliability
Test-Retest
Inter-Item
Inter-Rater
Test-Retest Reliability
consistency of participants’ responses on a measure over time
Useful only if the attribute being measured should not change over time
Inter-item Reliability
assesses the degree of consistency among the items on a scale
Use only if you have multiple items in a scale
Inter-rater Reliability
the consistency among two or more researchers who observe or judge behavior
Use only if you have multiple observers
Validity
accuracy (does the measurement procedure actually measure what it is supposed to measure?)
A measure cannot be valid if it is not reliable but a reliable measure is not necessarily valid
Types of Validity
Face
Construct
Criterion
Face Validity
the extent to which a measure appears to measure what it’s supposed to measure
Construct Validity
Construct validity – the extent to which a measure of a hypothetical construct relates as it should to other measures
Convergent Validity – a measure correlates with other measures that it should correlate with
Discriminant Validity – a measure does not correlate with other measures that it should not correlate with
Criterion Validity
Criterion-related validity – assess the relationship between the measure and a behavioral criterion
Concurrent validity – scores on a measure are related as expected to a criterion that is assessed at the time the measure is administered
Predictive Validity – scores on a measure are related as expected to a criterion that is assessed in the future
Sensitivity
The extent to which the values on the measure change when there is a difference in the thing being measured.
Must be particularly careful when evaluating measures designed to measure psychopathology for use in wellness promotion programs
How do we choose which outcomes to measure?
Program theory:
Logic model
Goals and objectives
Outcome indicators
continual measurement of easy to collect indicators of targeted change
Advantages to monitoring outcomes:
Provide timely information for program management
Much less expensive than outcome evaluation
Disadvantage to monitoring outcomes:
Cannot be used to infer causality
Can’t determine if program is effective
Moderator variables
Variables that characterize subgroups in the client population
EG: Race, or risk factor level for program
Ethics of Program Evaluation
High stakes outcome monitoring will always change staff performance
Staff will work to achieve measured outcomes that are used to judge their performance
Be aware of issues related to using means to judge performance
Be aware of staff with varying job responsibilities
