Evaluation 1 Flashcards
Health Program Evaluation – Summary
What is Evaluation?
Systematic assessment of the design, implementation for learning/decision-making
Goals-Based Evaluation:
Measures progress toward goals/objectives
Varies by program stage
Planning & evaluation must align
10-Step Process for Goals-Based Evaluation
Planning Phase
Describe the program
Identify & engage partners
Determine timelines & resources
Develop evaluation questions
Select measurement methods
Create an evaluation plan
Implementation Phase
Collect data
Process & analyze data
Utilization Phase
Interpret & disseminate results
Apply evaluation findings
Other Evaluation Tools & Frameworks (for health programs)
CAMH Mental Health Guide
Helps evaluate mental health promotion programs.
Based on best practices.
Indigenous Evaluation
Culturally grounded.
Focuses on community values, relationships, and storytelling.
RE-AIM Framework (more commonly used)
Evaluates 5 key parts of a program:
Reach, Effectiveness, Adoption, Implementation, Maintenance
PRISM (more specific)
Focuses on real-world application.
Looks at how well a program is implemented and sustained.
Step 1:
Purpose: To gather information for the program evaluation
- Summarize key program info
- Use a logic model (see logic model) to organize and guide evaluation
- Goal = general direction
- Target audience = data source
- Objectives, strategies, activities, resources = specific guidance for evaluation
Step 1:
Are You Ready to Evaluate?
Evaluability Assessment
Checks readiness for evaluation (done before evaluation)
Builds evaluation capacity
Ensures evaluation is useful, relevant, and efficient
Key Conditions
Program is clearly defined and logical
Purpose of evaluation is agreed upon
Data is accessible
Results will be used by intended users
Evaluation Plan Should Consider:
Purpose/Use – Why is the evaluation being done?
Audiences/Users – Who will use the results?
Step 2:
Purpose: To identify evaluation partners and determine how best to engage them
Find those involved/impacted by evaluation
Consider roles:
Core – plan
Involved – give input
Supportive – assist
Peripheral – stay informed
Review logic model with partners
Choose agencies aligned with evaluation goals.
Step 2:
Evaluation Committee
Type
Internal or external
Participatory Evaluation
Involves target audience
Shared decisions, equity, local relevance
Public Engagement Spectrum
Inform → Consult → Involve → Collaborate → Empower
Step 3:
Purpose: To identify when the evaluation will occur what resources are available to complete it.
Consider internal/external factors impacting the timeline
Identify available resources:
- Personnel
- Data collection
- Supplies & equipment
- Communications
- Time
Ensure the program is ready for evaluation before moving forward
Step 4:
Purpose: To select and prioritize the evaluation questions.
Base questions on:
Logic model
Program stage (planning, implementation, winding-down)
Past evaluations
Partners, timelines, resources
Types of Questions:
Process Evaluation – Looks at implementation & quality
Outcome Evaluation – Assesses effectiveness
All step 4
Process Questions
Reached target audience?
Participation numbers?
Followed the plan (fidelity)?
Quality of content?
Barriers/facilitators?
Outcome Questions
Impact on quality of life?
Behaviour changes?
Change direction?
Linked to program?
Unintended outcomes?
See diagrams
RE-AIM Framework
Supports planning & evaluation
Focus: adoption, sustainability, real-world use
RE-AIM Components (with Pragmatic Focus)
Reach – Who is intended to benefit vs. who actually participates
Effectiveness – What is the key benefit? Any negative effects?
Adoption – Where is it applied and who applies it?
Implementation – How consistently is it delivered, adapted, and at what cost?
Maintenance – When is it sustained over time at individual/org levels?
