Evaluation Flashcards
What is process evaluation?
Ongoing assessment of intervention implementation, targeting how it is carried out, its reach, and contribution to desired outcomes.
Process evaluation involves a mix of ____ and ____ research methods.
Qualitative; quantitative.
In a nutrition education campaign, exposure is measured by the number of people who received pamphlets. What type of evaluation measure is this?
Process
How is reach calculated in a school-based intervention?
As the percentage of students attending sessions compared to the total number of students.
Fidelity focuses on maintaining the ____ and ____ of the intervention.
Quality; consistency.
A health program measures participant satisfaction using surveys and focus groups. What process evaluation component is being assessed?
Satisfaction.
What does context evaluation examine in process evaluation?
Social, cultural, economic, and environmental factors affecting intervention success.
What is impact evaluation?
Assessment of immediate effects of an intervention, such as behavior, attitudes, and awareness changes.
A nutrition program measures changes in knowledge about healthy eating. Which type of evaluation is this?
Impact evaluation.
Outcome evaluation focuses on ____ effects, such as long-term changes in health behaviours or conditions.
Longer-term.
How might a community-based nutrition intervention evaluate its long-term success?
By measuring actual increases in fruit and vegetable consumption over time.
List the five timing effects in evaluation and explain their significance.
- Ideal effects: Immediate and sustained improvements.
- Sleeper effects: Delayed impacts.
- Backsliding effects: Short-term change followed by regression.
- Trigger effects: Accelerated behavior change.
- Backlash effects: Negative consequences after the intervention.
A researcher conducts repeated measurements over time on a single group to track changes. What evaluation design is this?
Single group time-series.
A randomised control trial (RCT) involves ____ allocation of participants to intervention and control groups.
Random.
What is a key strength of non-equivalent time-series design?
It strengthens evidence by comparing repeated measurements between intervention and control groups.
What are the challenges in evaluating capacity gains?
Dynamic contexts, multiple definitions, invisibility of capacity-building, and attribution of changes.
A health initiative uses participatory processes to engage stakeholders in evaluation. What strategy is being applied?
Incorporating participatory processes.
Mixed methods in evaluation involve combining ____ and ____ techniques.
Qualitative; quantitative.
How can trustworthiness in capacity evaluation be enhanced?
Through transparency, triangulation of data, and ethical considerations.
What is economic evaluation?
A method to compare interventions by analyzing their costs and consequences to determine cost-effectiveness.
For economic evaluation to be meaningful, an intervention must first prove its ____.
Effectiveness.
What does cost-utility analysis measure?
Outcomes in terms of quality-adjusted life years (QALYs) or disability-adjusted life years (DALYs).
A public health program is compared to another to find the lowest cost option with identical outcomes. What type of economic evaluation is this?
Cost-minimization analysis.
What is reflective practice?
A structured process of analyzing one’s experiences to gain insights and improve future performance.
Types of reflective practice include ___ and ___.
On action; in action.
A practitioner keeps a journal to critically evaluate the effectiveness of their interventions. What reflective practice tool is being used?
Reflective journaling.
What is valorisation in health promotion?
Sharing knowledge and lessons learned from interventions with a broader audience to ensure sustainability and impact.
Valorisation can involve sharing findings through ____, ____, or creative formats.
Written reports; presentations.
A project shares its results with policymakers and stakeholders to encourage program continuation. What strategy is this?
Valorisation.
Process Evaluation:
Focuses on assessing intervention implementation, operation, and its contribution to outcomes.
Uses qualitative and quantitative methods.
Key components:
- Exposure: Measures how many people engage with materials (e.g., pamphlets distributed).
- Reach: Percentage of target audience participating (e.g., students attending sessions).
- Satisfaction: Captures participant feedback via surveys, interviews, and focus groups.
- Delivery: Checks whether the intervention is implemented as planned.
- Fidelity: Ensures consistency and quality (e.g., correct use of materials).
- Context: Examines external factors like social, cultural, and economic influences.
Impact evaluation:
Assesses immediate effects of the intervention, such as changes in behavior, attitudes, or awareness.
Outcome evaluation:
Focuses on longer-term effects aligned with intervention goals.
Example: A nutrition program measuring knowledge changes (impact) and increased fruit/vegetable consumption (outcome).
Timing of Evaluation - Different effects occur over time:
Ideal effects: Immediate, sustained improvements.
Sleeper effects: Delayed impact.
Backsliding effects: Temporary improvement followed by regression.
Trigger effects: Accelerated behaviour change.
Backlash effects: Negative consequences post-intervention.
Single Group Post-Test Only:
Collects data after intervention; lacks comparison.
Single Group Pre & Post Test:
Measures changes before/after intervention; influenced by external factors.
Non-equivalent Control Group Pre & Post Test:
Compares intervention and control groups; lacks randomisation.
Single Group Time-Series:
Repeated measures over time; subject to external influences.
Non-equivalent Time-Series:
Measures intervention/control repeatedly to strengthen evidence.
Randomised Control Trial (RCT):
Gold-standard; random allocation to intervention/control groups, robust for causation evidence.
Challenges in measuring capacity:
Ambiguity in terms and definitions.
Invisibility of capacity-building.
Dynamic contexts and sensitivity of issues.
Attribution for change.
Strategies to address challenges in evaluating capacity gains:
Use participatory processes.
Combine qualitative and quantitative methods.
Build trust and establish validity/reliability.
Be flexible and acknowledge the dynamic nature of health systems.
Purpose of economic evaluation:
Compare interventions based on cost and outcomes to inform resource allocation
Types of economic evaluation:
- Cost-minimisation analysis: compares identical outcomes for lowest cost
- Cost-effectiveness analysis: quantifies effects like behaviour change eg. BP, F+V intake
- Cost-utility analysis: uses QALYs/DALYs to assess QoL
- Cost-benefit analysis: measures all outcomes in monetary terms to assess cost-benefit balance
Reflective practice in evaluation:
Analyses experiences to gain insights for improvement.
Types:
“On action” (post-event reflection).
“In action” (real-time reflection).
Tools: Journals, structured questioning, discussions with peers.
Valorisation in evaluation:
Sharing intervention outcomes to ensure sustainability and wider impact.
Formats: Reports, presentations, creative methods.
Emphasises transparency and learning from failure.
