chapter 8; implementation and evaluation Flashcards
a soft launch is when the program planners do not call a lot of attention to initiating the intervention. one or several sites may be used to test out procedures and ensure that all processes are running smoothly before expanding the program to the intended scale
yes
testing out the intervention in this manner is called pilot testing
regardless of the decision of whether to run a pilot test, most programs set a hard launch data (after the pilot, if one will be used) and hold a partner kickoff meeting, news event, town hall, or other activity designed to generate interest and enthusiasm in the program
shortly after the launch, program managers will monitor all aspects of the intervention to ensure they are functioning as planned
these monitoring activities constitute process evaluation. in operational settings, corrections - or continuous quality improvement - can still take place
yes
in experiments (e.g. the effectiveness trials of interventions), protocols would need to be resubmitted to institutional review boards before making major changes.
yes
throughout the programs implementation, program managers will want to collect and review some key metrics with stake holders to see whether objectives are being reached
after the intervention has run for long enough to have reached a sufficiently large audience and potentially made an impact, an evaluation of outcomes can then be conducted
the evaluation design depends heavily on the logic model for the intervention, which must take opportunities and threats into consideration.
two of the key tools used to build a program, are the logic model and SWOTE analysis, are also used in its evaluation
SWOTE stands for “strengths, weaknesses, opportunities, threats, and ethics”.
to see if there has been a change, it is essential to collect what are called baseline data before launching the program
therefore, even though evaluation often comes at the end of an intervention it must be included in the design from the beginning
logic model, “a picture of how you organization does its work- the theory and assumptions underlying the program”
or
a logic model displays the connections between resources, activities and outcomes. as such it is the basis for developing a more detailed management plan. during the course of implementation, a logic model is used to explain, track and monitor operations, processes and functions. it serves as a management tool as well as a framework to monitor fidelity to the plan
preparing logic model; resources/inputs range from tangible items, such as money, paid staff or volunteer hours, facilities, and shipment, to more intangible items, or the involvement of collaborators at the state, national , or global level.
preparing logic model; activities is a description of what the program will do . a communication program often includes the following activities; [..] activities, outputs, outcomes, impact
- mass media such as paid advertising, public service announcements, entertainment education events (plays, television program scripts, radio soap operas), social media, or student classroom materials
- patient decision aids such as computer animations, decision software, pencil and paper flipbooks, kits, or worksheets
- training workshops for healthcare providers, teachers, coaches, or clergy to use media effectively
- outreach and education activities such as health fairs, speaking engagements, trade shows, or grocery store or drugstore promotional activities
numbers are associated with activities in terms of what is anticipated and budgeted; for example,
conduct 10 teacher training workshops” or “distribute 10,000 patient decision aids”
logic model; outputs
outputs describe whether the activities were delivered as planned; for example ; for mass media
- mass media ; if you planned to deliver 10k student workbooks, how many students actually received workbooks. if you bought radio and television air time, when and where did you spend air? what was the measured audience. if you ran a website, how many pages, downloads, or forms were completed?
- for patients; how many health facilities, practices, or individual healthcare providers have agreed to use the patient decision aids? how many patients have received them ?
- for intermediaries (people using the health communication materials to work with others): how many workshops were implemented? how many participants completed pre and posttests? how many materials were ordered post training?
- for outreach: how many public appearances, speaking engagements or health fairs were held? where and when ? Who came?
assessing whether your outputs were generated according to plan is an essential prelude to outcomes
logic model; outcomes
are normally divided into immediate, intermediate, and long term ranges.
- immediate outcomes for mass media might include the follow (next day recall of message, awareness of issue, change in attitude or motivation to try something). responses would be assesses through survey research.
- intermediate outcomes; generally include changes in individual behaviors, enactment of policies, or uptake of technology or strategies by organizations.
- Long term outcomes may be identical to the impact, although the former refers more to individuals 9e.g. an individual quits smoking and their life is extended) and the latter to population (e.g. the death rate from tobacco goes down)
logic model; impact is generally measured in terms of population level health or socioeconomic improvements. this would be reflected in reductions in age specific mortality rates, prevalence of disease, and disease specific mortality rates. population measures of quality of life would improve
logic model; putting the model together. should fit on a one printed page (it can be a big page)
logical model; backwards planning with logic models
taking your logic model in hand, works backwards and ask;
1. what needs to happen in the medium term for these objectives to be met?
2. before that, what needs to happen in the near term?
3. step back again, and ask: what activities must be conducted, and which what audiences, to achieve these short term outcomes
4. what resources are needed to conduct these activities
a SWOT analysis is the standard business tool for assessment of strengths, weaknesses, opportunities, and threats.
new addition oof an E to SWOT for ethical assessment which we feel is an indispensable piece for health communication programs
SWOT;
strength and weaknesses refer to conditions that are within your control, or at least inherent to your organization’s ability to implement the program .
SWOT;
opportunities and threats are outside of your control. you may plan for or around them, but they will happen on their own.
SWOT;
ethical considerations may reside inside your organization, but it is important to assess whether intentionally, or inadvertently, your program could harm someone, limit someones rights, or promise something that cannot be delivered
- strengths are the attributes within the organization. these include personnel capabilities, experiences, material resources, organizational commitment, time allowed, and budget
- strengths are the attributes of the intervention/communciation campaign. these include the positioning of a product or service, its cost, its attractiveness or reputation, and so forth. if you have a great product to service to work with (e.g. a free pizza nigh on a college campus), that is a program strength. if you have na unpopular idea to sell (e.g. an increase in tuition), this can be a weakness