Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

*BrainBuds -Core Study Time > Program Planning and Evaluation (PowerPlay, Eh?) > Flashcards

Program Planning and Evaluation (PowerPlay, Eh?) Flashcards

1
Q

3 ways PH programs can intervene

Provide examples.

A

Policy/Structural
E.g., Fluridate water, treat sewage, outlaw transfat

Incentives & Disincentives
E.g., fine for not wearing seatbelts, tax cigs, require immunization

Volitional Behavior
E.g., Excercise, sleep under bednets, use contaceptives

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Questions we want to answer with/from our program

A

Did it work?
Achieve intended outcomes?

  • *What else do we want to know?**
  • reach target group?
  • adopted by program/staff?
  • fidelity (delivered as intended)?
  • quality?
  • *Cost effective?**
  • did targer groups continue over time?
  • continue working of time?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Types of Eval: Efficacy & Effectiveness

A

**Efficacy — ideal conditions **
*RCT, well-resourced program, ‘pilot’ projects. E.g., plenty of staff, time and money to train and pay for people’s time, very tight controls, high standardization assured.

**Effectiveness — real world conditions **
*more variation – in who’s delivering, who’s getting, in what is delivered; less control and standardization; fewer resources.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Difference between 2 types of evalution: Outcome v. Process Evals

A

**Outcome Eval **
Focuses on efficacy or effectiveness

Process Eval
Focuses on what elese we want to know

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What evaluation does PH prioritize?

A

OUTCOME

Outcome over process eval

Within outcome evaluaitons – efficacy over effectivess

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Bottom Line: If we focus on efficacy…

A

Narrow focus on efficacy can obscure serious issues related to reach and adoption

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Bottom Line: If we focus on outcomes…

A

Narrowly defined outcomes can mask key issues about implementation and quality of services

Note: Implementation and quality issues are usually defined as ‘process’ and therefore tend to receive less attention. But outcomes can (and should?) be defined to incorporate these ‘additional’ program elements for full public health achievement.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the steps of the RE-AIM framework?

A

REACH
-reach intended recipients?

EFFICACY & EFFECTIVENESS

  • work in ideal circumstances (efficacy)
  • work in real-world (effectivess)

ADOPTION
-adoped by target delievers (both program and staff)

IMPLEMENTATION
-implemented as intended

MAINTENANCE

  • effects last on individuals
  • sustained by programs/providers
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is a Logic Model?

Other names.

A
  • *Visual depiction of program** showing:
  • what will occur
  • what will be accomplished
  • if-then ‘causal’ relationships among elements
  • underlying theories

AKA:

  • Theory of change
  • Causal Pathway
  • Logical Frame (Logframe)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is the order, direction, of a common logic model?

A

Inputs > Activities > Outputs > Effect/Outcomes > Impact

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Describe the causal pathway framework.

A

<!--StartFragment-->

Impact

  • Change in health status of the population of interest
  • May also have socioeconomic benefits
  • *Effect**
  • Effect –> Impact*
  • Change in knowledge, attitudes, skills, behavior of population of interest

Outputs
Outputs –> Effect –> Impact
-Good quality products and services that must be in place for effects to occur
-Product must be available for use

  • *Activities**
  • Activities –> outputs –> Effects –> Impact*
  • Technical and support tasks required to produce the outputs

Inputs
Inputs –> Activities –> Outputs –> Effect –> Impact
Resources needed to support the activity

<!--EndFragment-->

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Input

A

<!--StartFragment-->

Resources needed (does not specify quality)

Inputs –> Activities –> Outputs –> Effect –> Impact

Resources needed to support the activity

<!--EndFragment-->

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Activity

A

Technical and support (active voice)

Activities –> outputs –> Effects –> Impact

Technical and support tasks required to produce the outputs

<!--EndFragment-->

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Output

A

Quality products in place; influenced by activity (passive voice)

Outputs –> Effect –> Impact

  • Good quality products and services that must be in place for effects to occur
  • Product must be available for use

<!--EndFragment-->

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Effect

Are we BASK?

A

Knowledge, attitudes, skills, behavior (BASK in effects)

Effect –> Impact

Change in knowledge, attitudes, skills, behavior of population of interest

<!--EndFragment-->

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Input

A

Change in health status

Change in health status of the population of interest

May also have socioeconomic benefits

<!--EndFragment-->

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Trick for catergorizing certain causal pathway elements

A

<!--StartFragment-->

If in program – eliminates effect and impact

If in population – effect and impact

<!--EndFragment-->

18
Q

From Re-Aim to Framework

  1. Efficacy
  2. Adoption
  3. Reach
  4. Implementation
  5. Effectiveness
  6. Maintenance
A
  1. Moderates:
    Output –> Effect
    Effect –> Impact
  2. Outputs
  3. Outputs
  4. Activities and Outputs
  5. Entire pathway
  6. Outputs (for orgs), Effect (individuals), and Impact (individuals)
19
Q

Causal Hypothesis verbage (keywords/how write)

AKA: Theory of change

A

This intervention, a result of this set of inputs, activities and outputs, will facilitate these changes in the population, which will contribute to the desired impact.

20
Q

Why use causal frameworks?

A
  • Specify intended results
  • Determine actions needed for results
  • Determine where causal chain broke-down
  • Use eval questions to help planning
21
Q

Why develop evaluate programs?

PH perspective (4).

A
  • improve health
  • confidently establish program changes in a positive way
  • establish best practices
  • premium on efficacy over effectiveness
22
Q

Gold standard for efficacy? Why?

A

RCT — maximizes internal validitiy

Efficacy — ideal circumstances

23
Q

Efficacy evals and internal validity

A

Internal validity refers to our degree of certainty that:

a) change in outcome variables due to intervention

or

b) lack of change due to failed intervention

24
Q

What must an evaluation design include?

A

Counter-factual

A compariosn point indicating what would the outcome have been w/o the intervention

25
Q

What could cause change aside from intervention?

What about lack of change?

A
  • selection bias
  • attrition bias
  • diffusion/contamination
  • Implementation problems
26
Q

When is attrition harmful to internal validity?

Possible solution?

A

Depends on evaluation design

Pre-Post —> always harmful (no comparison)

Intervention/Ctrl –> only harmful if differential attrition

Possible soultion: Intent to treat — hard to track down

27
Q

What are some internal validity threats?

A
  • history
  • testing
  • insturmention

Fix: Control group

  • selection bias
  • attrition
  • contamination

Fix: Randomization, intent to threat, non-interacting arms

28
Q

Differences between internal and external validity
in program evaluations

A

<!--StartFragment-->

**Internal --\> is observed change due to intervention**
Prioritizes efficacy (in ideal circumstances)

External –> how generalizable
-Prioritizes effectiveness (in real-world settings)
***PH requires this
-RCTs ≠ external validity

  • You really can’t have both be strong
  • TENSION between internal validity and external validity
  • Inverse realtionship

<!--EndFragment-->

29
Q

Why don’t RCT’s have strong external validity?

A

Can’t randomize people in everyday life

Can’t make choices for individuals – can not put them into an ‘arm’

Want inclusion, do not want to exclude groups of people

Want diffusion/contamination

Want to move beyond ‘ideal conditions’

30
Q

What reduces external validity?

A

**Attrition **
<!--StartFragment-->

  • **Problem with attrition is differential attrition ***
  • Internal validity
  • Differential matters for internal
  • ALSO reduces external validity*
  • Not just differential, but also drop-out
  • Drop-out –> not something want/be able to do
  • Chances are the you’re not paying in effectiveness studies*
  • You want to know who’s dropping out and why
  • All about making it work for real-world circumstances
  • So you are more likely to change intervention –> willing to revise the model for the context, group, etc

<!--EndFragment-->

31
Q

External Validity

How matters in programs.

A

<!--StartFragment-->

Degree to which program efficacy findings can be generalized beyond the specific situation studied to other:

  • Other target groups and sampling criteria
  • Other types of delivery sites
  • Adaptions in the intervention that reduce the resources needed for program delivery
    • Staff, training and supervision, time, space, materials, supplies, and the duration of the intervention
    • May not have had training for years –> so retrain (activity) for skilled care (output)
  • Participation Rates: % invited in who participated:
    • by setting, delivery staff, subgroups of consumers/patients
    • comparison of decliners/targets
  • Reach within Target Group–
    • Representativeness of Target Group as it was defined (selection or inclusion/exclusion criteria)
    • % in target group among: people recruited; people who received the program

<!--EndFragment-->

32
Q

Factors influencing validity

A
  1. Eligibility criteria
  2. Intensive intervention
  3. Level of standardization in intervention
  4. Level of control by reasearcher
33
Q

What are the two most important frameworks in Re-Aim?

State definition and differences.

A
  • <!--StartFragment-->Linking external to Re-AIM
  • Reach and adoption are most relevant to external validity
    ​
  • Reach –> ability of program to reach target program
    • Participation rates
    • And how acceptable
  • Adoption –> participation and feasibility of sites
    • Participation rates among sites​
    • Participation among staff
    • Resources/costs
  • Be clear on this difference
    • Reach refers to population
    • Adoption is about organizations and staff w/in orgs

<!--EndFragment-->

34
Q

Reach

A

<!--StartFragment-->

Reach: participation and acceptability rates in the target (participating) population

  • Participant rates
    • % recruited who agree to participate
  • Target group
    • % of study participants who are from target groups or sub-groups
  • Attrition rate
    • % of barriers who dropped out of the program

<!--EndFragment-->

35
Q

<!--StartFragment-->

Data on Reach and acceptability may raise what questions?

RE-AIM

A

<!--StartFragment-->

GOAL: How might reach and acceptability be improved?
​

  • Why reach limited to certain sub-group segments
  • Major barriers to acceptability
  • Which components acceptable and not? Why?

<!--EndFragment-->

​

36
Q

Name data sources for Rearch and Acceptability

RE-AIM

A
  • Clinical or medical
  • Program service stats
  • Attendance records
  • Survey interviews (also want to talk to refuses and attriters)
  • In-depth interview (also want to talk to refuses and attriters)
  • Focus (also want to talk to refuses and attriters)

<!--EndFragment-->

37
Q

Adaption

A

<!--StartFragment-->

Adoption: participation of, accessibility by, feasibility for the delivery sites

  • Site participation (and type)
  • Staff participation
  • Attrition rate (by organization and staff)
  • Resources/Cost –> staff, space, time demands

KEY: Both organization and staff

<!--EndFragment-->

38
Q

Data on Adoption and acceptability may raise what questions?

RE-AIM

A

<!--StartFragment-->

  • Why did certain sites decline?
  • Why did staff decline?
  • Would sites who agreed to participate do so again?
  • Would staff who agreed, do so again?

<!--EndFragment-->

39
Q

Main challenge in adoption/feasibility.

RE-AIM

Name other challenges

A

Really want to look at fidelity!!!!!

  • Data logs on site recruitment
  • Close-ended survey questionnaires with delivery staff and admin
  • In-Depth interview/focus group with staff and administrators
  • Personal logs charting the # of staff delivering program
  • Personal log charting # of hours staff engaged in work
    • Include training on this one
  • Cost data on site-level expenses for deliver
  • Asses of staff buy-in
  • Assess adequacy of staffing and level of staff turn-over a
  • Assess adequacy and stability of funding at delivery site
  • Asses adequacy of the management structure and communication

<!--EndFragment-->

40
Q
A

*BrainBuds -Core Study Time (3 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions