Health-Related Evaluation Methods Flashcards

Module 12

1
Q

What is the formative realist case evaluation example

A

Involved an evaluation of a difficult to describe large scale “transformation” of health services and its related report.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the overall questions considered in all realist evaluations?

A

What works, for whom, under what circumstances?

What are transferable lessons about effective change?O

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Theory or logic theory whose aim is to explain how an intervention is understood to work. It used mixed data sources and collection methods of ethnographic observation, semi-structured interviews and document analysis

A

Realist context-intervention-mechanism theory

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Summarize the “lessons” that were reported related to Mechanism 1 from the formative realist case evaluation example

A
  1. Relationships between organizations are trusting
  2. Approaches to integration are imaginative, locally responsive, negotiable and supported by technology rather than rigid and driven by tech
  3. External incentives are designed to reward collaborative performance and do not make organizations compete
  4. The strategy for integration includes both “soft” and “hard” approaches
  5. Solutions are participatory rather than developed by one party and imposed on others
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

The generalizability of the findings in this type of formative realist case evaluation example can be increased by ___________________.

A

Providing details of the change actions and of the context

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

The description of the mechanisms or change principles and how they were triggered by the intervention in the setting enables others to try to ________ the mechanism in their different settings, possibly using different ___________.

A

Reproduce

Interventions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How does this type of formative realist case differ from many other forms of evaluation, particularly experimental evaluations?

A

Gives a narrative understanding of the process of change in an environment rather than showing a few measured outcomes attributed to a change

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What difficulties are involved with formative realist case evaluations?

A
  1. Distinguishing between context and mechanism
  2. Is there an end point?
  3. When to “close”/
  4. Time-consuming
  5. Can realist evaluation studies be replicated?
  6. How does realist evaluation differ from traditional mediator/moderator analysis?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Describe the reasons to evaluate a reform before it starts, during its implementation and after its implementation

A

Assess whether it is feasible, to predict problems and to help plan how to implement it

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

During a reform, an evaluation is to help make corrections _________ implementation

A

During

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

After the reform, an evaluation helps to explain the lessons for _______ changes or reforms and ________ to scientific knowledge that could help the reformers explain, understand or predict the effects of _____________ changes.

A

Future
Contributes
Large-scale

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

List and explain the usual steps for evaluation a reform

A
  1. Describing the reform “instrument”
  2. Gathering data about changes in health service performance and possibly also in health
  3. Assessing whether these data really are outcomes of reform
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

The aim of the reform is often to do the following

A
  1. Present evidence of the results of the reform
  2. Describe the process of implementing the reform
  3. Assess the strengths and weaknesses of the reform as implemented and judge the value of the results
  4. Recommend improvements
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

In most reforms, the intention is to change the organization, procedures, financing methods and other “things”.

A

Immediate subject or target

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Reform that is often aimed to improve patients health or well-being

A

Long term aim

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Intervention that does not work directly on the _________ _________ (patients or the population) but works through changing the immediate subject (health organization) in order to achieve the ultimate benefits or better health.

A

Ultimate beneficiaries

17
Q

4 designs typically utilized to evaluate health reforms

A
  1. Implementation designs
  2. Comparing achievements with reform objectives
  3. Before/after design
  4. Comparing one site with one of more others implementing the reform
18
Q

Designs that report on strengths and weaknesses of the process and make explicit the criteria used to assess the process. For example, one criterion might be whether all stakeholder groups were involved.

A

Implementation designs

19
Q

Compares the intended goals of the reform with the extent to which they are achieved

A

Comparing achievements with reform objectives

20
Q

This design describes the reform implementation process but also gathers data before the reform and then after the reform has been proceeding for enough time to have an effect.

Common criteria are costs, efficiency, access, equity, quality and service comprehensiveness.

A

Before/after design

21
Q

This comparative design gathers data about different sites or areas implementing the same reform. It can be used cross-nationally to compare similar types of reform.

A

Comparing one site with one or more others implementing the reform