Chapter 8 Flashcards
Logic model -
Graphic representation of a program showing the intended relationships between investments and results.
Ways to help you depict what the program they are going to do.
Inputs -
Resources that go into a program including staff time, materials, money, equipment, facilities, volunteer time
Outputs
The activities, products, and participation generated through the investment of resources. Goods and services delivered.
Outcomes -
Results or changes from the program such as changes in knowledge, awareness, skills, attitudes, opinions, aspirations, motivation, behavior, practice, decision-making, policies, social action, condition, or status.
Outcomes may be intended and/or unintended: positive and negative.
Outcomes fall along a continuum from immediate (initial; short-term) to intermediate (medium-term) to final outcomes (long-term), often synonymous with impact
Distal and Proximal
Distal (long-term); Proximal (short term)
program theory
composed of the impact theory and the process theory.
Impact theory
This is the theory showing the results or outcomes that are supposed to be produced by the program activities.
The action theory (or action hypothesis) specifies the expected immediate or proximal outcomes of the program actions; the conceptual theory (or conceptual hypothesis) specifies the expected distal or longer term outcomes.
_________(or action hypothesis) specifies the expected immediate or proximal outcomes of the program actions; the conceptual theory (or conceptual hypothesis) specifies the expected distal or longer term outcomes.
Action Theory
Note the possibility of ______ (the program is implemented, but the theory is faulty; hence, the expected results do not occur) an___________(the planned program services are not delivered; hence, the expected results do not occur).
Theory failure; implementation failure
The next two components of the program theory (service utilization and organizational plan) are known as the _____________ and they require process evaluation.
Program process theory
You need to work with ____________ is to develop a program theory.
stakeholders
Program evaluators make implicit theory explicit
Work with stakeholders to adjust the theory to make it a good theory.
- is defined as the “approximate validity with which we infer that a relationship between two variables is causal” (Cook and Campbell, 1979. P.37).
Internal validity
- is defined as the “approximate validity with which we infer that a relationship between two variables is causal” (Cook and Campbell, 1979. P.37).
Internal validity
A good synonym for the term internal validity is ___________ because that is what internal validity is all about.
causal validity
What are the 3 required conditions for causation?
Condition 1- A is related to B. (the relationship condition)
Condition 2-proper order must be established (the temporal ancedence condition )
Condition 3-The relationship b/w A and B must not be due to some confounding or extraneous variable or “third variable”(the lack of alternative explanation)
____________- has to do with the degree to which the results of a study can be generalized to and across populations of persons, settings, times, outcomes, and treatment variations.
External Validity
A good synonym for external validity is________ because it always has to do with how well you can generalize research results.
generalizing validity
What are the major types of external validity?
population validity, ecological validity, temporal validity, treatment variation validity, and outcome validity.
ISD and evaluators prefer which kind of assignment for research subjects?
Random Assignment over random selection
The ________ variable is the variable that is assumed to be the cause of the effect. It is the variable that the researcher varies or manipulates in a specific way in order to learn its impact on the outcome variable.
Independent
The_____________ is a situation in which a researcher objectively observes phenomena which are made to occur in a strictly controlled situation where one or more variables are varied and the others are kept constant.
experiment
When an experimental research study is conducted, the researcher’s interest in always in determining___.
cause and effect
_____________ is the most important technique used to control for confounding variables because it has the ability to control for both known and unknown confounding extraneous variables.
Random Assignment
STRONG experimental research design
Random selection helps ensure ______ validity.
external
Random assignment helps ensure_____validity.
internal
Altschuld and Kumar present a model that involved three phases of a needs assessment. What are these three phases?
A. Planning, Conducting, and Assessing.
B. Preassessment, Assessment, and Postassessment
C. Attitudes, Learning, and Behavior
D. Input, Output, and Moderation
B
When is there a lack of treatment fidelity?
A. When the evaluator conducts more than one study at a time.
B. When the evaluator does not implement the program/study as intended
C. When the evaluator relies on an external auditor.
D. When the evaluator is part of the organization that is conducting the evaluation.
B
Cost Analysis is defined as an evaluation’s determination of whether a program’s effect was worth its cost.
A. True
B. False
A
The majority of evaluations are conducted for more than one purpose.
A. True
B. False
A
Which evaluation type or types are associated with the transformative paradigm? A. Outcome Evaluation B. Country-Led Evaluation C. Deliberative Democratic Evaluation D. B & C
D
In international development evaluation, how is the “gaining insight” purpose of evaluation typically labeled? A. Needs assessment B. Impact C. Relevance D. Responsiveness
C
The second major category of evaluation purposes is to figure out areas in need of improvement. What types of evaluations fall in this category?
A. Implementation, responsive, process, and developmental evaluation
B. Context, capacity, needs and assets assessment, and relevance evaluation
C. Impact, policy exportability, and sustainability evaluation
D. Gender analysis, deliberative democratic, and country-led evaluation
A
Which of the following is NOT a purpose for an evaluation?
A. To identify needed inputs, barriers, and facilitators to program development or implementation.
B. To determine inequities on the basis of gender, race, ethnicity, disability, and other relevant dimensions of diversity.
C. To determine how a program can be made to look positive despite its lack of impact/effect.
D. To demonstrate that accountability requirements are fulfilled.
C
When the researcher can draw clear causal conclusions from what went on in a study, this is called: A. Internal validity B. Internal reliability C. Internal consistency reliability D. External Validity
A
The Solomon four-group design is commonly used in evaluation.
A. True
B. False
False
What are the purposes of Evaluation?
- Gain insights or to determine necessary inputs
- Find areas in need of improvement or to change practices
- To assess program effectiveness
- To address issues of human rights and social justice
Which types of evaluation gain insight or determine necessary inputs?
Context evaluation Capacity Buidling Needs and assets assessment Organizational Assessment Relevance evaluation
Which types of evaluation find areas in need of improvement or to change practices/
Implementation evaluation responsvie evaluation participatory evaluation process evaluation monitoring formative evaluation developmental evaluation
What evaluations are used to assess program effectiveness?
outcome/impact summative policy replicability/exportability/transferability sustainability cost analysis
Which evaluations address issues of human rights and social justice?
DDE CLE CRT Indigenous Culturally responsive Dsiability and deaf rights based feminist gender analysis transformative participatory
