Exam 2 Chapter 4 Flashcards
Program Process Theory (process evaluation builds off of this)
identifies the critical components, functions, and
relationships assumed necessary for the program to be effective
Criteria for assessing program process performance may include:
stipulations from the program theory, administrative standards, applicable legal, ethical, or professional standards, and after-the-fact judgment calls
Process evaluations can be stand alone or ongoing, which is called:
Program Process Monitoring
Process evaluation is usually carried out along with this other evaluation:
Impact Evaluation
Process Evaluation Slide show definition
Information about what the program or program staff have to do to create change
Activities
Attendance
Satisfaction
Fidelity
Outcome Evaluation Slides definition:
Information about changes program seeks to create in clients or for clients
Knowledge
Attitudes
Behaviors
Changes in health status
What are Process and Outcome measuring?
Change = Outcome
Activities that Cause Change = Process
Types of Process Evaluation
Snapshot Approach
Program Process Monitoring
Type of Process Eval: Snapshot approach
Time limited
Done primarily for evaluation purposes
Type of Process Eval: Program process monitoring
Ongoing
Done for both management and evaluation purposes
Why monitor program process?
Confirm that the program is reaching its intended targets
Understand outcome/impact data
Improve the program even without outcome data
Facilitate program management – are administrative standards being met?
Components of Comprehensive Process Evaluation
Fidelity
Dose Delivered
Dose Received
Satisfaction
Reach
Recruitment
Components of Comprehensive Process Evaluation: Fidelity
Fidelity
Delivered as designed
Components of Comprehensive Process Evaluation: Dose Delivered
Dose Delivered
Amount of program delivered
Components of Comprehensive Process Evaluation: Dose Received
Dose Received
Actual use of program
Components of Comprehensive Process Evaluation: Reach
Reach
Proportion of intended targets served
Components of Comprehensive Process Evaluation: Satisfaction
Satisfaction
Program and staff
Components of Comprehensive Process Evaluation: Recruitment
Recruitment
Procedures
Main components of process evaluation
Service Utilization
Organizational Function
Main components of process evaluation:
Service Utilization
Service Utilization
Who is the program serving?
Other Aspects of Service Utilization
Coverage
Is program participation reaching intended levels?
How many people are we serving?
Bias
Is the program serving the types of people it was designed to serve?
Skimming (Related to service Utilization)
Systematically eliminating clients who are difficult to serve
Must be distinguished from legitimate efforts to refer clients who cannot be successfully served by program
Creaming
Systematically Including clients who would be easy to serve.
Monitoring Organizational Functioning
Is the program actually delivering the intended services? Are the services being delivered according to plan?
Implementation Failure (related to monitoring organizational functioning)
Intervention not delivered
Wrong intervention is delivered
Intervention is completely unstandardized
How Do We Collect Process Data?
Use existing program records
Develop simple record keeping instruments or databases
MIS systems
Client or Community Surveys
Satisfaction
Awareness
Utilization
Process Evaluation or Monitoring Plans must be designed to:
Avoid being overly punitive to staff
Avoid encouraging dishonestly or unwanted changes in staff behavior
