PSYC*2360 Chapter 13: Program Evaluation Flashcards

1
Q

What does a program evaluation involve?

A

Involves using the scientific method to assess whether an organized activity is achieving its intended objectives

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are grants?

A

Monetary gifts to help organizations meet a specific goal or objective

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

T or F: Program evaluations are helpful, but not often essential to the grant writing process.

A

False. They are often essential.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are three types of program evaluations?

A
  • A needs evaluation
  • A process evaluation
  • An outcomes evaluation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Which type of program evaluation involves an assessment of which features of a program are most valuable and who they benefit?

A

A needs evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are the two main goals of a needs evaluation?

A
  • Identify features to continue or discontinue
  • Determine potential components to add
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Which type of program evaluation is particularly helpful for program planning?

A

A needs evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Which type of program evaluation involves an assessment of a general program operation, including whom the program serves and how it delivers services to that population?

A

A process evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is the main goal of a process evaluation?

A

Determine ways to improve program implementation and delivery

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Which type of program evaluation seeks to ensure a match between program goals and those serviced?

A

A process evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Which type of program evaluation involves an assessment of whether a program effectively produces outcomes that are consistent with stated objectives or goals?

A

An outcome evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the two main goals of an outcome evaluation?

A
  • Identify unmet goals and outcomes that can be improved
  • Establish evidence of program effectiveness
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are four potential benefits of program evaluations?

A
  • Forces members to clarify the program’s purpose and intended outcomes
  • Provides the organization with evidence of the program’s effectiveness and that the program is using resources efficiently
  • Allows the organization to evaluate a new approach or resource
  • Encourages organizations to identify areas for improvement
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is the main goal of a program evaluation?

A

To provide actionable information that can have immediate implications to benefit a specific program

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

T or F: In a program evaluation, there is often less control over extraneous variables that may bias findings.

A

True

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What is one way to avoid potential bias during a program evaluation?

A

Use an outside consultant to conduct the evaluation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

T or F: A program’s outcomes are often immediately obvious.

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

T or F: Program evaluations seek to help improve a program, therefore no additional attention must be paid to potential consequences.

A

False

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

T or F: If a program evaluation is too intrusive, researchers might undermine client’s trust or limit the program’s effectiveness.

A

True

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are the three phases of a program evaluation?

A
  • The planning phase
  • The execution phase
  • Communication of results
21
Q

What are the four steps involved in the planning phase of a program evaluation?

A
  • Identify key stakeholders
  • Describe the program
  • Clarify the evaluation’s goals
  • Create an evaluation plan
22
Q

What are the two steps involved in the execution phase of a program evaluation?

A
  • Gather data
  • Analyze data
23
Q

What are the three steps involved in the communication of results phase of a program evaluation?

A
  • Form conclusions
  • Make recommendations
  • Report findings
24
Q

Which method of data collection involves assembling several individuals to discuss a specific topic?

A

Focus groups

25
Q

Ideally, how many participants should be included in a focus group?

A

6-12

26
Q

Ideally, should participants in focus groups know each other beforehand?

A

No

27
Q

Is the fact that in a focus group, participants have the opportunity to react to each other’s contributions most often considered an advantage or a disadvantage?

A

An advantage because it provides richer information

28
Q

T or F: Focus groups provide a high volume of insightful content using participants’ own words in a quick and efficient manner.

A

True

29
Q

In focus groups, are facilitators typically allowed to ask follow-up questions to clarify meaning?

A

Yes

30
Q

What is the most notable disadvantage of focus groups?

A

The inability to generalize to other populations

31
Q

What is the role of a moderator in a focus group?

A

To ask the questions and facilitate discussion

32
Q

What are the four characteristics of an ideal moderator?

A
  • Has a thorough understanding of the focus group’s topic
  • Has a strong foundation in research methods
  • Understands group dynamics (the behaviours and psychological processes of social groups)
  • Remains as neutral and bias-free as possible during discussions
33
Q

In a program evaluation, information from focus groups can be used to develop what additional method of data collection/measurement?

A

Surveys

34
Q

T or F: Although focus groups provide quantitative information about the program’s effectiveness, surveys enable qualitative information to be gathered.

A

False. Focus groups provide QUALITATIVE data and surveys provide QUANTITATIVE data.

35
Q

What is a single-item indicator?

A

When only one item or question is being used to measure a variable

36
Q

What are two concerns when using single-item indicators?

A
  • Poor reliability
  • The variable is only being measured in a general sense
37
Q

What is a single sample t-test used to evaluate?

A

Whether a sample mean statistically differs from a specific value

38
Q

T or F: Even though program evaluations aren’t intended to expand generalizable knowledge, IRB approval is still always required.

A

False. IRB approval is often not required.

39
Q

Rather than having participants sign an informed consent form, it can be noted at the beginning of a survey that returning it implies consent to be part of a study. This is known as what?

A

Implied consent

40
Q

T or F: A program evaluation report should present findings in a clear and succinct way, avoid jargon, and be tailored to the audience.

A

True

41
Q

What are word clouds?

A

Visual representations of the frequency that certain words are used in qualitative assessments

42
Q

In a word cloud, what do larger words indicate?

A

Higher frequency of use

43
Q

What can be used to visually represent a study’s findings that quickly conveys a lot of information in an engaging and easy-to-understand format?

A

Infographics

44
Q

The “conclusions and program recommendations” section of a program evaluation report is the equivalent to which section of an APA style report?

A

The “general discussion” section

45
Q

What are the three things that a “conclusion and program recommendations” section of a program evaluation report focuses on?

A
  • How the outcome findings match with the program goals
  • What the findings indicate about the program’s strengths and weaknesses
  • What steps the program should take for improvement
46
Q

When discussing findings that support a program, stakeholders often exhibit hindsight bias, therefore it is important to remind them of what?

A

The benefits of having data to support assumption

47
Q

When discussing findings of a program evaluation that disagree with prevailing opinions, what it is important to remind stakeholders of?

A

The natural flaws in thinking and the importance of data to support claims

48
Q

What type of research involves a systematic and planned sequence of related studies, where subsequent studies build directly from a previous study’s findings?

A

Programmatic research