changing evaluation landscape

Question 1

evaluation basics

Answer 1

systemic and objective assessment of an on-going or completed policy, program or project focusing on its design, implementation and results.

Question 2

aim and purpose of evaluation

Answer 2

determine the relevance and results of policies, programmes and projects:

Results:

• effectiveness
• efficiency
• impact
• sustainability

provide a basis for accountability

Question 3

evaluation basics

Answer 3

evaluators have to simply live with tensions

a strong backbone is needed: do not give in to pressure and remain independent and credible. stakeholders may try to influence:

• evaluation programming
• content of the terms of reference
• review of evaluation findings
• attack independence of the evaluators & methods

credible evaluation will ultimately provide a ‘body of knowledge’ resulting in chance and adaptation of policy and practice

Question 4

Randomised control trials (RCTs)

Answer 4

• comes from the health sector
• the randomistas generate data on development effects
• focus on identification of causal effects
• key advantage: clear identification ./ any difference
  between treatment and control must be due to
  treatment
• development intervention as experiment
• external validity
• often short term studies

Question 5

attribution (prefferably ex durant)

Answer 5

• comparison helps ex post understanding of effects of an intervention>
• does it claim causal effects that aren’t there?
• how measurable is development?

Question 6

problems in evaluation

Answer 6

• simple models risk overstating causal contribution
• more likely to represent a single theory of change
• anxiety provoked by uncertainty and ambiguity can lead managers and evaluators to seek the reassurance of a simple logic model

Question 7

Complex systems

Answer 7

difference between complex and complicated:

Complex is used to refer to the level of components in a system. If a problem is complex, it means that it has many components. Complexity does not evoke difficulty.

On the other hand, complicated refers to a high level of difficulty. If a problem is complicated, there might be or might not be many parts but it will certainly take a lot of hard work to solve.

Rogers (2008) says: complex adaptive systems are composed of a diversity of agents that interact with each other, mutually affect each other, and in so doing generate novel behavior for the system as a whole.

Question 8

complexity science

Answer 8

• theory transfer
• structural representation / surrogate reasoning
• holistic / systems theory
• two approaches for social scientist
• main themes: non- linearity / emergence / far-from-equilibrium