Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Health and Healing 3005 > Week 10- program evaluation > Flashcards

Week 10- program evaluation Flashcards

1
Q

Evaluation….. is the KEY!

A

The ‘right’ evaluation depends on what you and the program stakeholders want to know.Evaluation could focus on the activities, outputs, reaching the population, the outcomes, the overall goal of the program … or a combination of these. Choose evaluation questions that will answer questions stakeholders actually care about and that will be used to make decisions.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Evaluation Should be able to Answer these

A

What really matters? Who is most at risk? Why are we creating the program? Does the evidence justify the need?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Planning Process- Identify and formulate objectives

A

Objectives: Specific measurable statements that identify the steps planned to reach the overall program goalSeveral objectives (short term, intermediate, and long term) are stated to meet each program goal

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Strategies

A

The means which changes will be made.

Identify the “vehicles” for how you will provide the program.

Public policy

Create supportive environments

Develop personal skillsReorient health services

Strengthen Community Action

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Activities

A

The specific ways that you will carry out for the strategy.WorkshopsProduct developmentCounselingCurriculum designTraining

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

PATCH:

A

The goal of this model is to give clients control of their health based on health education, and to enhance the capacity of communities to plan, implement, and evaluate all-inclusive, community-based health promotion programs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

PLM:

A

This model is used in health care as a planning and evaluation framework, and depicts the components in diagrammatic form. It describes and defines the parts of a program, but is not an evaluation model, therefore other tools need to be used for evaluation of the parts

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Questions to ask in Program Planning 
Stage

A

Analyze the community: What is the situation?Identifying issues: what is important? Whose views have been heard? Who’s have not? What is the most important issue right now? Why does the issue exist? What are the parts of the issue?Clarify goals: What do we want to happen?Designing Actions: what needs to be done, what resources are needed, who can do what?**Evaluation although the final step is linked to assessment, the first step.***

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Questions to ask during implementation

A

How can we ensure we do “good” and “no harm”How can we keep things on track?How will we deal with the unexpected?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Benefits of Program Evaluation

A

The major benefit of program evaluation is that it shows whether the program is fulfilling its purposeProgram records are the major source of information

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

The two levels of evaluation are:

A

Formative evaluationSummative evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

The evaluation process involves:

A

Goal settingDetermining outcome goal measurementIdentifying goal-attaining activitiesMaking the activities operationalMeasuring the goal effectDetermining whether the program goal was achieved

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

** Assigning value to evaluation

A

What will be evaluated? What aspects of the program will be considered when judging program performance?What standards What evidence will be used?What conclusions regarding program performance are justified?How will the lessons learned from the inquiry be used?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Aspects of Program Evaluation

A

Relevance: The need for the program

Adequacy: The extent to which the program addresses the entire problem defined in the needs assessment

Progress: The tracking of program activities to meet objectives

Efficiency: The relationship between program outcomes and costs

Effectiveness: The ability to meet program objectives and the results of program efforts

Impact: Long-term changes in the client population

Sustainability: Enough resources to continue the program

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What will I measure?

A

Sometimes you can measure objectives directly…”improved knowledge”
Indicators: Increase in self-reported knowledge, improved scores on post-test, ability to show new skill
Think:
Outcome evaluations: self-rate scores, prevalence of a disease, decreased rates of smoking, participation levels
Process evaluations: attendance, # of sessions held, # of people reached, % satisfied with workshop

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Evaluation:

A

The method used to determine whether a program or service is required and likely to be used, whether it is conducted as planned, and whether it actually works.How to improve the program and how to best use resources.It is a systematic, step by step processIt is always tied with decision-makingDifferent kinds of decision-making require different kinds of evaluationShould involve the community as partner

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Sources of Program Evaluation

A
  1. Program clients

Written surveysI, nterviews, Observations

  1. Program records

Clinical records, Community indicators (e.g., mortality and morbidity data)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Principles of Program Evaluation

A

Strengthens programsUse multiple approachesDesign evaluation to address real issuesCreate a participatory processAllow for flexibilityBuild capacity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Community as Partner & evaluation

A

Purpose of evaluation is to determine if threats or challenges to health (stressors) have been minimized, if the health status of the population or community (normal lines of defence) are improved, if the community’s capacity (lines of resistance) or resilience (flexible lines of defence) is strengthened as a result of an intervention.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Questions to ask during evaluation

A

What has changed as a result of the program?Was the program carried out as planned?Were the results worth the effort?What is the situation now?Evaluation should involve the community: YOU want a participatory, empowering, equitable process.It is not RESEARCH!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

**10 Steps to Program Evaluation

A

1: Clarify your Program
2: engaging stakeholders
3: assess your resources
4: organize and select evaluation questions
5: determine methods
6: develop work plan, budget, timeline
7: collect data
8: process data and analyze results
9: interpret and disseminate results
10: take action

22
Q

Step 1

A

Clarify your Program:**One of the goals in program planning is to know what your overall change is that you expect!! That way you can evaluate it.Ensure that your program has clearly defined goal(s), populations of interest, outcomes, strategies, activities, outputs, and indicators Often accomplished using a logic model A logic model is a diagrammatic representation of a program It shows the relationships among: WHAT: what does the program do? WHO: who are the recipients of the program? WHY: what outcomes are to be achieved?

23
Q

Step 2

A

Engaging StakeholdersStakeholders must be engaged in the inquiry to ensure that their perspectives are understood. When stakeholders are not engaged, an evaluation might not address important elements of a program’s objectives, operations, and outcomes.Evaluation findings might be ignored, criticized, or resisted because the evaluation did not address the stakeholders’ concerns or values**The plan must have clearly defined & assessed the health concern in the program plan.

24
Q

4 levels of stakeholders

A

1, core: key to program & initiative, involved in everything2, involved: consulted parties: clients, management3, supportive: provide some form of support, private sector partners4, peripheral: need to be kept informed, Board of health, funders

25
Q

Step 3

A

Assess your resourcesClarify staff time, money and other resources available for evaluation This will inform your evaluation design decisions Consider these:Funds Staff and volunteer time and interests Timeline Equipment and tools Support of partners

26
Q

Step 4

A

Organize and Select Evaluation QuestionsOrganize your evaluation questions by type: Formative, Process , Outcome Select key evaluation questions: *What type of decisions need to be made with the evaluation data *Stakeholder interests *Resources

27
Q

Process (Formative) Evaluation

A

Intended to improve the operation in an existing program.Sometimes known as program tracking or monitoring Answers: Are we doing what we said we would do? Concerned with improving program operations.On an ongoing basis such evaluations are used for monitoring and revising program activities when necessary.Accomplished through activities such as continuous feedback from program participants.Frequently conducted by internal evaluation groups.May also provide documented evidence of the historical development of various aspects of a program Example: Quality assurance audits

28
Q

Outcome (Summative) Evaluation

A

Concerned with the immediate impact of a program on a target group.Measures change produced by program activities (i.e., the effect) Assess the effectiveness of the program.Completed at the end of program activities, therefore the focus is on the outcome. - Is the program worth continuing or expanding?Examines whether a program has met its stated goals and objectives and whether the outcomes of delivery the program have achieved the intended effect.Often mandated by policy or conducted as part of policy reviews and frequently carried out by outside evaluators or by senior managers.Incorporates impact evaluation and use tools such as population surveys, measures of behavioural change, and tests to show biological changes.Answers: Is our program effective?

29
Q

Mapping Evaluation Questions and Indicators to a Logic Model

A
30
Q

Participatory/collaborative/
empowerment evaluations

A

It is formative and constructive process that uses evaluation concepts, techniques, and findings. The participants in the program including staff, volunteers, and other stakeholders exercise control over the entire evaluation process. Role of manager or others may be as facilitator or mentor. The participants determine whether the program is responsive to their needs and determine future program directions

31
Q

Balanced Scorecard

A

Focuses on measuring and improving organizational performance through examining different components of the organization and its programs. Focuses on four indicators – customer perspective, internal-business processes, learning and growth, and financials – to monitor progress toward an organization’s strategic goals. It provides feedback around both internal business processes and external outcomes in order to continuously improve strategic performance and results

32
Q

Step 5

A

determine methods What will you measure? When will you collect data? How will you collect data? Who will you collect data from? Who will use the data?

33
Q

Consider these..

A

Informed consent Purpose/goal/objectives of the evaluation Evaluation method Potential risks Potential benefits Anonymity, confidentiality and security of data How the participants will be informed of the results

34
Q

Indicators

A

Specific measures indicating the point at which goals &/or objectives have been achieved. Objective: By the year 2018, all Ottawa hospitals will have policies, procedures and practices to promote and support breast feeding. Indicator: % of hospitals with “baby friendly” designation Short, long term and Process indicators Each objective should have one indicator to prove success Can relate them to standards

35
Q

Long term indicator:

A

Can be linked to changes in knowledge, behaviour, or policy change.

36
Q

Short term indicator:

A

usually only affect the participants. -how many were registered in program, how well they rated program, etc.

37
Q

Process indicator:

A

tells us how well the program was delivered. -# of people who attended, participant views, how many condoms handed out, # of classroom visits, etc.

38
Q

Step 6:

A

Develop work plan, budget, timeline Identify specific tasks, roles, resource allocations and deadlines for the evaluation In-house versus out-source?

39
Q

Step 7:

A

Collect data Pilot test tools and procedures Revise Train Collect data

40
Q

Step 8:

A

Process data and analyze results Double-check at least 10% of data Put data in a format that can be summarized and interpreted Identify themes in qualitative data Enlist the support of a data analysis expert whenever possible

41
Q

Step 9:

A

interpret and disseminate results Work with stakeholders to make sense of findings Draw conclusions based on the findings. What are the answers to your evaluation questions? Make recommendations on actions that should result from the evaluation Share results in formats that are tailored to specific audience needs/preferences

42
Q

Step 10:

A

Take action Involve your stakeholders Use the answers to your evaluation questions: Try something different? Keep doing something that worked well? Improve program? Replace program?

43
Q

Parts of a Formal Written Evaluation Report

A

A title page Table of contents An executive summary providing a concise overview of the findings and recommendations A background explanation about the program A statement regarding the purpose (s) of the evaluation An explanation of the methods used in conducting the evaluation An account of the findings with interpretations and expressions of limitations about the findings, and Any conclusions and recommendations arising from the appraisal.

44
Q

Most Importantly….

A

Ensure that your intervention successes are shared back with the community! They remain part of the plan…..

45
Q

What is a Grant?

A

A sum of money given to an agency or individual to address a problem or need in the community. The written document that one prepares as a means of requesting or applying for this money (funding) is a grant proposal.

46
Q

Why Would I write a grant?

A

Want to start a new program/project in your community. Want to expand an existing project and costs cannot be covered in your current budget. Grant proposals are labour intensive…..you need to be committed! Grants can take 60-120 hours to complete!

47
Q

Budget Form

A

Comes from the agency Lists year by year and line by line items. Common budget line items include 1. Staff salaries 
2. Taxes
 3. Fringe benefits 
4. Indirect costs
 5. In kind items 6. Rent and utilities
 7. Equipment and supplies 
8. Postage 
9. Travel

48
Q

Budget Narrative

A

Excel sheet makes this easy. Detailed description of budget items and formulas used to determine various costs. Indirect costs: Overhead

49
Q

SMART goals and objectives

A

Specific. That is, they tell how much (e.g., 40 %) of what is to be achieved (e.g., what behavior of whom or what outcome) by when (e.g., by 2010)? Measurable. Information concerning the objective can be collected, detected, or obtained from records (at least potentially). Achievable. Not only are the objectives themselves possible, it is likely that your organization will be able to pull them off. Relevant to the mission. Your organization has a clear understanding of how these objectives fit in with the overall vision and mission of the group. Timed. Your organization has developed a timeline (a portion of which is made clear in the objectives) by which they will be achieved

50
Q

Appendix

A

Required documents that support & confirm the grant & budget narratives. May include: signed letters of commitment, signed memoranda’s of agreement, agreements for funds/resources, resumes of participants and partners.

51
Q

GANTT Chart

A

This chart is used to present a detailed list of all activities and their projected date of completion. Activities are usually listed in sequential order. Will keep you on track. Adjust as you go

52
Q

Tips for filling in a time line chart

A

Try to anticipate every activity an objective might entail and estimate at which point in the program’s time frame the activity will be completed. Understand that the time line is meant to be used for planning purposes and may be revised over time. For example, some activities will be dependent upon the completion of prior activities. It is fine to show multiple items with the same completion date. Remember that all activities in the time line will shape your budget request

Health and Healing 3005 (5 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions