Chapter 13: Program Evaluation Flashcards
What occurs in a program evaluation?
scientific method is used to assess whether an organization activity is achieving its intended objectives
A program evaluation is similar to a _______
progress check-list
What are some pros to a program evaluation?
- Forces members of an organization to clarify the program’s purpose and intended outcomes
- Helps identify the organization’s “mission”
- Provides the organization with evidence of the program’s benefits or knowledge that the program is using its resources effectively
- Allows the organization to evaluate a new approach or resource in order to determine if it should continue or not
What are some cons to a program evaluation?
- Less control of extraneous variables that can bias the findings
- Program’s outcomes are not always immediately obvious
- Can introduce ethical issues
What are the 3 steps to a program evaluation process? Explain each step.
- Planning
- identify stakeholders
- desscribe program
- clarify evaluation’s goals
- create evaluation plan - Execuation
- gather data
- analyze data - Communicate Results
- interpret data analysis
- form conclusions
- make recommendations
- report the results
What is a needs evaluation? (type of program evaluation)
- assess which features of a program are most valuable and who they benefit most
- identifies if the program features should continue or not
- helps determine new components to add
- helpful for program planning
Which type of program evaluation assesses the general program operation, including whom the program serves, and how the program delivers services to that population?
process evaluation
In a ______ evaluation, methods are looked at for how to improve program implementation and delivery
process
A(n) _________ evaluation determines whether a program effectively produces outcomes that are consistent with stated objectives or goals
outcomes
Which type of program evaluation identifies unmet goals and outcomes the program can improve on?
outcomes evaluation
What is a focus group?
several individuals that come together to discuss a specific topic
How many people are generally in a focus group?
6-12
What does a focus group require?
moderator
A person who asks questions and facilitates the discussion in a focus group is called a _______
moderator
T or F: a modertator should encourage overly talkative or verbose members from dominating the conversation
FALSE; the moderator should prevent these type of people from dominating the conversation
What is a single-iten indicator?
when only 1 item or question is used to measure a variable
What are 2 concerns of the single-item indicator?
- reliability
- the item only measures effectiveness in the general sense (can’t tap into specific outcomes)
What is a word cloud?
a way to visually represent the frequency of specific words used by participants in a qualitative study
What is locus of control?
degree to which people feel they have control over the outcomes/events in their lives
What is grant writing?
developing an application in which an organization provides specific information regarding a program’s goals along with evidence of the program’s effectiveness in order to receive monetary gifts
A(n) ______ is used to assess whether an organized activity is achieving its intended objectives
a) program evaluation
b) functional assessment
c) organizational analysis
d) systems approach
a) program evaluation
_____ evaluations help organizations determine which new programs should be developed
a) needs
b) outcome
c) process
d) formative
a) needs
“Made for TV” focus groups are often criticized for all of there reasons EXCEPT
a) the size of the focus group
b) the representativeness of the focus group
c) moderator bias
d) problematic group dynamics
a) the size of the focus group
Haley is conducting a focus group with staff at a business. There are 50 staff people in the company. Haley should
a) conduct two focus groups of 25 staff members
b) conduct a focus group of 9-12 volunteers
c) randomly sample 9-12 staff members to be included in the focus group
d) attempt to include all 50 staff members in a single focus group
c) randomly sample 9-12 staff members to be included in the focus group
Monetary gifts given to help organizations meet a specific goal or objective are termed
a) credits
b) start-ups
c) loans
d) grants
d) grants
Mixed methods research
a) necessitates a factorial experimental design
b) has a between-and a within-subjects factor
c) allows the participant to interact with the researcher in developing measurement tools
d) use both quantitative and qualitative methods
d) use both quantitative and qualitative methods
Which is NOT part of the planning phase of a program evaluation?
a) identify key stakeholders
b) make recommendations for change
c) clarify evaluation’s goals
d) describe the program
b) make recommendations for change
Why are focus groups usually kept relatively small?
a) to minimize the effects of attrition
b) to keep hidden the purpose of the program evaluation
c) to maximize individual participation
d) to minimize social desirability bias
c) to maximize individual participation
A ______ t-test is a statistic used to evaluate whether a sample mean statistically differs from a specific value
a) paired-samples
b) single-sample
c) independent-samples
d) related-samples
b) single-sample
Using a single-item indicator to measure a variable is
a) desired because of its reliability
b) desired because it minimizes demand characteristics
c) undesired because it is more sensitive to random error
d) undesired it often lacks face validity
c) undesired because it is more sensitive to random error
A copy of the actual survey used in a program evaluation would be expected to appear in which section of a program evaluation report?
a) procedure and methods
b) summary of key findings
c) conclusions and program recommendations
d) appendices
d) appendices
A program evaluation can determine program effectiveness, but it cannot pinpoint the cause of effectiveness:
a) without follow-up longitudinal data collection
b) unless reliable and valid survey instruments were used
c) because it is not an experimental design
d) without risking participant confidentiality
c) because it is not an experimental design
