Program evaluation Flashcards
Steps of Program Evaluation
Step 1: Identify the goals of the program
Step 2: Describe the characteristics of the organization
Step 3: Engage in process evaluation
Step 4: Define outcomes to be measured
Step 5: Measure the outcomes of the program
Step 6: Report and disseminate findings
Step 1:
The social worker may speak with the agency Executive Director and/or program staff to
identify the goals of the program. Note: no goals or objectives are created during this
time, but we do need to know what the goals of the program were in order to evaluate
whether they were met
Step 2
Describe the characteristics of the organization:
● Describe the types of clients served by the program, the interventions and practice
techniques used by staff, and the organizational structure of the agency and program.
● These things will help with step 3 as we seek to understand how the program services
are delivered as well as possible hindrances to goal achievement.
Step 3
Engage in process evaluation
● Process evaluation examines how a program was carried out and whether program
activities were implemented as intended.
● This could include things like the management of the program, if the agency supported
the program well, and which interventions were implemented for the program. All of
these factors impact the process of implementation and will in turn end up impacting the
overall program outcomes. So process evaluation has to do with implementation
whereas ‘outcome’ or ‘summative’ evaluation has to do with overall outcomes (whether
the goals were met). Here are some examples of process evaluation:
○ If clients were supposed to attend X number of therapy sessions, were all
sessions offered and attended?
○ Did all clients receive the same quality of services from their providers?
○ If a goal was to respond to client inquiries within 12 hours, you would need to
determine ways to find out if this occurred.
Step 4
Define outcomes to be measured
● Involve key program staff and others who were involved in setting the project’s goals.
● Outcome goals are comprised of data that the agency already keeps track of:
○ Example: Number of clients served, satisfaction with services, change in
symptoms/diagnoses, number of service hours utilized.
● Measurable outcomes must be determined to identify whether the projects’ goals are
being met.
● A baseline of what clients were experiencing before the program was enacted must be
established.
Step 5
Measure the outcomes of the program
● Outcomes can be measured with two types of data: qualitative and quantitative.
○ Qualitative measurements: Information that is not in numerical form.
Ex. Open-ended surveys, unstructured interviews, observations.
○ Quantitative measurements: collecting data involving numbers that can be
statistically manipulated. Ex. Number of clients, Survey with a Likert scale
gauging client satisfaction, scores on tests (such as the Beck Depression
Inventory).
Step 6
Report and disseminate findings
● Findings should be clear and concise and may include suggestions for
changes/improvements.
● Positive and negative feedback should both help organizations improve their programs.
Steps of Program Development
- Conduct a Needs Assessment: Explore the problem and gather information
- Establish the planning team/Create a task force
- Define the goals and objectives of the program based on results of the needs
assessment - Develop an action plan and timeline for implementation
- Implement the action plan/logic model
Step 1 for program development
Conduct a Needs Assessment: Explore the problem and gather information
● A community needs assessment is done to develop an informed understanding of the
gaps or needs that exist within a community and their impacts upon the community’s
members.
● A community needs assessment identifies the strengths and resources available in the
community to meet the needs of children, youth, and families.
● 5 Steps of Conducting a Community Needs Assessment:
Step 1: Define the Scope: What issue are you looking at? Housing? Mental
Health?
Step 2: Collaborate: Partner with existing groups/organizations.
Step 3: Collect Data: Use things such as surveys, focus groups, demographic
information, and/or interviews.
Step 4: Determine Key Findings.
Step 5: Share your Findings.
● Note: If we are working with a new population, there may be a step that comes before
conducting the needs assessment. This step will occur in order to better understand the
population to ensure the needs assessment is done with respect to the needs of that
specific community. This can be done in one of two ways:
○ Identifying and meeting with experts around this population or issue.
OR
○ Completing a literature review to understand what has been done (and what has
worked/not worked) with similar presenting issues or populations.
Step 2 for program development
Establish the planning team/Create a task force
● Identify and build a team of key staff to be included in the planning, development, and
implementation process for the new program.
● Task forces are usually for larger programs.
● Note: this is the team that will be developing the program, not the people that will staff
the program itself.
Step 3 for program development
Define the goals and objectives of the program based on results of the needs
assessment
● The goal(s) should define what the program is specifically looking to achieve in order to
address the identified problem(s) from the Community Needs Assessment.
● The goal(s) describe the overall, measurable outcomes or changes that you are
expecting through the implementation of the program.
● The objectives specify the measurable tasks that will occur in order to achieve the
broader programmatic goals
Step 4 for program development
Develop an action plan and timeline for implementation
● Programs should be research-supported and theoretically sound
○ Gather and organize data on programs that have addressed similar issues.
○ If you haven’t already done so at the beginning, conduct a literature review. A
literature review is a process of gathering relevant information from multiple
sources. It involves a critical, in-depth evaluation of previous research on a
specified topic.
● Clarify the components of the program.
○ Identify program activities, output, and outcomes of the program.
● Identify people and resources needed for successful implementation of the program.
○ Outline teaching and learning methods for staff.
● Determine benchmarks and tools/measures for assessment and evaluation.
○ How we will measure success should be determined during the program
development process. This is critical in evaluating whether or not goals are met.
○ This will ensure data is used throughout program implementation, influencing the
development of the program and can be used for continuous improvement
throughout.
● Create a timeline with a schedule for the implementation of the program.
Step 5 for program development
Implement the action plan/logic model
● Evaluate progress throughout and make adjustments as needed.
