Videos

Question 1

What is an impact evaluation?

Answer 1

• Measuring impact of program on outcomes to see if program is effective in achieving its ultimate goals (eg. how much has program X reduced STD rates)
• May be long-term follow-up!

Question 2

What is the best way to answer causal questions in an impact evaluation? What can you find out?

Answer 2

• Randomized Controlled Trials (RCTs)
• Program can even be complex (multiple components that produce change)
• RCTs can tell you whether intervention does more good than harm in specific context but cannot explain why they work and under which cases

Question 3

What is outcome measurement?

Answer 3

Focused on the target changes that occur within participants in programs (eg. knowledge, attitude, behaviour, status)

Question 4

What is a logic model?

Answer 4

• Part of outcome measurement

- Depiction of what program’s theory is, designed collaboratively with stakeholders of the program

Question 5

What are the components of a logic model?

Answer 5

1. Inputs (resources used by program)
2. Activities (what program is)
3. Outputs (countable units of service; how many classes, participants, sessions)
4. Outcomes of interest (MUST be represented by indicators)
   - Proximal outcomes ⟶ now
   - Distal outcomes ⟶ in the future

Question 6

What does outcome measurement tell you?

Answer 6

• All we are measuring is whether an outcome changed while a participant was in a program
• We haven’t proved that the change is due to the program itself
• We need more rigorous evaluation design to make claim about causality (control groups, matched cohort studies)

Question 7

What is process evaluation?

Answer 7

• Nested within RCTs (experimental design); can be used to assess quality of implementation (how the program was set up), clarify causal mechanisms, and identify contextual factors
• Explore qualitatively to see how intervention works in a RCT rather than assuming it works through anticipated mechanisms

Question 8

What do you do if randomization is not possible in a trial?

Answer 8

• Quasi-Experimental Design

- Match participants based on similar characteristics to compare instead of control