Lessons 5-7 (Quiz 2) Flashcards
What is formative evaluation?
Collecting data in pilot situations while developing an intervention
What is the main purpose of formative evaluations?
Obtain feedback
What is formative evaluation concerned with?
Concerned with feasibility
- Are proposed activities appropriate?
- Do they FIT with setting & recipients?
Are there specific methodology or procedures?
NO
What are the 3 Fs of formative evaluation?
Feasibility
Fit
Fine Tuning
Is “internal” agency business reported to stakeholders?
NO
What are the 3 approaches to conduct formative evaluations?
Approach 1: Locate model standards
- Look @ what has already been successful
Approach 2: Get expert consultation
- Who knows what has been successful?
- Pay them to find out what they think
Approach 3: Form an evaluation committee
- Interview staff & visit similar programs
What are process evaluations primarily concerned with?
Delivery of services and dosage.
What is dosage?
Requirement of time.
Formative evaluation vs. Process evaluation?
Formative evaluation - used to FORM program in early stages - program development Process evaluation - Occurs once the program has been established - Concerned with program fidelity
What is program fidelity?
Program loyalty (how faithfully a program is carried out) - means not only doing it, but doing it well
Why should programs be concerned with fidelity?
High fidelity = greater chance of significant behavioral change
- programs with highest fidelity = great outcomes
Why should evaluators be concerned with fidelity?
lack of fidelity could lead an evaluator to mistake the effectiveness or non-effectiveness of a program
What are the main goals of process evaluation?
Program monitoring and quality assurance.
What can process evaluation identify?
variations in program delivery called “program drift”
Without process evaluation what may occur?
Type 3 error
What is a type 1 error?
Incorrectly inferring program effectiveness
- REJECTING a null hypothesis that is truly accurate
What is a null hypothesis?
null hypothesis = no difference in effectiveness or significant change
What is a type 2 error?
Failing to reject the null hypothesis
What is type 3 error?
related to fidelity (cannot give outcome because plan was not followed)
What is the evaluation big picture?
- Need assessment
- Formative evaluation
- Process evaluation
- Outcome evaluation
What does process evaluation provide feedback on?
On the quality of ongoing intervention delivery
What can process evaluation determine?
Determine if failure = poor program or poor implementation
How can you tell if the right program elements were delivered to the right people?
Collect data on program components
Data from pilot situations, feasibility, and fit are elements of which type of evaluation?
Formative evaluation