Health-Related Evaluation Methods Pt 2 Flashcards
Module 12
What are the 3 designs for performance evaluations?
- Audit
- Single care time trend performance evaluation
- Comparative performance evaluation
What are the three key questions for all performance evaluations?
- Who is the evaluation for?
- What do they need to know to act differently and better (their criteria)?
- Which data could provide this actionable knowledge?
This design compares the health system with a statement of what is expected of the system. The employer might use an evaluator to write a specification of which services are needed and compare this specification with what the local system provides.
Audit performance evaluations
These designs track the performance of a health system over time. Example is the rate of patients who were hospitalized and discharged alive after a heart attack
Single-case time trend performance evaluations
These designs compare the performance of a set of services at one time, in relation to one or more dimensions of service performance.
Service-comparison performance evaluations
Detail the steps in conducting a return on investment evaluation
- Define the change that the investments will be used to make
- List the time and other resources needed(a) to make the change and (b) to sustain the change once it is made
- Define the investment and the investor
- Define the returns to be valued
- Decide the ROI study design and data
- Collect and analyze the data
- Specify assumptions, limits and degree of certainty of the evaluation
What is a business case assessment
Where the justification, plan and expected results are laid out in one document
What might a business case statement include
- Current problems and opportunities
- How and why a change will solve the problem facing the organization
- The time, money and other resources needed from the organization and from others to make the change successfully
- Expected benefits, especially in relation to the current problem or strategy, and expected ROI at one, three and five years
- Risks and risk management
- The expected situation if the change is not undertaken, and the possible alternative use of the resources proposed for the change
- An outline draft plan for next steps
Changes that both improve quality and reduce waste and that usually reduce the costs of producing a service.
Value improvements
How are value improvements evaluated
Assess whether promised changes do in fact improve quality and reduce waste.
Also needed to assess whether the cost of the change is worth the improved quality and reduced waste that the change may produce.
Type of calculation that considers both the quality returns and the resource returns
Cost-quality return on investment (CQROI)
A way of assessing from previous research or information whether a change will be a value improvement
Value improvement estimates
Questions to ask local service providers
- Does previous research or info give “strong enough” evidence or quality improvement? If yes, then continue with points 2-7
- How much does intervention cost
- Is there an evidence of reductions in waste and more efficient use of resources
- What is the money value of these reductions and efficiencies
- If there is no evidence from research for points 2 or 3, then use theory or estimations for deciding a cost range
- Overall, what is the return on investment
- Time to pay off: How are the spending and savings distributed between different parties at year one, year to, year three, year five or year ten?
What constitutes a “good” ROI evaluation?
- Specifies the user of the evaluation and the decisions the evaluation will inform
- Identifies all the costs and benefits that are important to the users decisions, and measures in financial terms those that are feasible to measure
- Distinguishes in the findings the theoretical ROI and also the actual or likely monetized ROI
- Specifies the limitations, assumptions and range of certainty of the results
