Monitoring and Evaluation

Question 1

What is monitoring?

Answer 1

the routine reporting of data on program implementation and performance
- are we doing the work we planned?

Question 2

What is evaluation?

Answer 2

the periodic assessment of program impact at the population level and value
- how effective were our activities?

Question 3

Compare monitoring and evaluation?

Answer 3

1. monitoring is continuous e.g. day to day and evaluation is periodic e.g. mid term (important milestones)
2. monitoring documents progress using selected indicators whereas evaluation is a comprehensive investigation of program achievements and other determinants
3. monitoring focuses on inputs, activities and outputs and evaluation focuses on outcomes and impacts
4. monitoring provides warning signs to managers and evaluation provides managers with strategy and policy options if corrective action is needed
5. monitoring is self assessment and evaluation is external analysis

Question 4

Why monitor and evaluate?

Answer 4

1. Collect accurate information about the project
2. Use that information to improve the project

Question 5

Why is monitoring and evaluation important?

Answer 5

1. Tracking resources
2. Feedback on progress
3. Improving project effectiveness
4. Informing decisions
5. Promoting accountability
6. Demonstrating impact
7. Identifying lessons learned

Question 6

What are the purposes of monitoring?

Answer 6

1. Management
2. Reporting
3. Accountability
4. Advocacy
5. Evaluation

Question 7

What are the purposes of evaluation?

Answer 7

1. Episodic assessment of specific indicators
   - determine effectiveness or impact of services or activities
   - during a given interval
2. Determine whether goals are being met
3. Assess impact of a specific service or intervention
   - HIV testing among TB patients
4. Advocacy

Question 8

What is the definition of a mission?

Answer 8

• Deals with present and leads to the future
• It explains what an institution is currently doing.

Question 9

What is the definition of a vision?

Answer 9

• Project’s dream
• Big picture of what you want to achieve
• It explains what an institution wants to be in future

Question 10

What is a goal?

Answer 10

• The ultimate result of efforts at a broad, population level.
• Achieved over the long term (years) and through combined efforts of multiple programs
  e.g. Decrease morbidity and mortality due to TB in Country X.

Question 11

What is an objective?

Answer 11

• Specific milestones with defined timelines
• How the results of your short-term program activities contribute to the big goal.
• Several objectives can relate to the same goal.
• Link between activities and the goals.

Question 12

Objectives must be?

Answer 12

SMART
1. Specific
2. Measurable
3. Achievable
4. Realistic
5. Time constrained
e.g. Increase NTP budget by 8% each year for the next four years

Question 13

What are inputs?

Answer 13

• Resources needed to plan and implement ACSM
• “Raw materials” of an ACSM project
  e.g. money, staff, policies, guidelines, equipment, partners

Question 14

What are activities?

Answer 14

• The work that we do, what we implement
• Also called “processes”
  e.g. training events, meetings, events, outreach, home visits

Question 15

What are outputs?

Answer 15

• Immediate results of activities
• What we can measure/count right after the activity
• It does not address the value or impact of service
  e.g. number of people trained, number of brochures produced

Question 16

What are outcomes?

Answer 16

• Level of performance or achievements
• “Ripple effects” of activities
• What changes after outputs are produced
  e.g.
  Increased funding for TB after lobbying meeting
  Short: Improved attitudes toward TB patients among DOTS nurses after a training
  Medium: Increased satisfaction of TB clients
  Long: TB clients stay in treatment longer

Question 17

What are indicators?

Answer 17

• A variable that measures one aspect of a program/project or health outcome
• How we define our activities, outputs, or outcomes
• Signs or evidence we watch for to see if we have reached them

Question 18

What makes up a good indicator?

Answer 18

1. Provide information useful for program decision-making
2. Are consistent with international standards and other reporting requirements, as appropriate
3. Are defined in clear and unambiguous terms
4. Non-directional and “independent”
5. Have values that are:
   - Easy to interpret and explain
   - Precise, valid and reliable measures
   - Comparable across relevant population groups, geography, other program factors, as needed

Question 19

Characteristics of good indicators?

Answer 19

1. Valid - accurate measure of a behavior, practice or task
2. Reliable - consistently measurable in the same way by different observers
3. Operational - defined in unambiguous, clear terms
4. Measurable - quantifiable using available tools and methods
5. Non-directional - allowed to vary in either direction
6. Timely - provides a measurement at time intervals relevant and appropriate in terms of program goals and activities
7. Programmatically important - linked to a public health impact or to achieving the objectives that are needed for impact

Question 20

How do we plan for monitoring and evaluation?

Answer 20

1. Identify who will be involved in the design, implementation, and reporting
2. Engaging stakeholders.
3. Clarify scope, purpose, intended use, audience, and budget for evaluation.
4. Develop the questions to answer what you want to learn as a result of your work.
5. Select indicators.
6. Determine the data collection methods.
7. Analyze and synthesize the information you obtain
8. Interpret these findings, provide feedback, and make recommendations

Question 21

What is impact?

Answer 21

More related to goal
The result of achieving specific outcomes
e.g.
- Higher rate of treatment success
- Reduction in deaths among MDR-TB patients
- Improved standard of living