Evaluation of health services Flashcards
What is the definition of health service evaluation?
the assessment of whether a service achieves its objectives
What framework do we use for health service evaluation?
Structure
Process
Outcome
Give examples of structure
structure is what there is (buildings, staff, equipment)
- number of ICU beds per 1000 population
- number of vascular surgeons per 1000 population
- ratio of patients with diabetes to diabetic nurses
Give examples of process
process is what is done
- number of patients seen in A+E
- number of operations performed
- waiting time in A+E
- how is priority assessed in A+E (in general, the process through which patients go through in A+E)
Give examples of outcome
- mortality eg 30 day mortality rate
- morbidity eg complication rates
- quality of life/PROMs eg symptom scores, ADLs
- patient satisfaction
In relation to CBT and mental health, give quantitative examples of structure, process and outcomes
structure: the ratio of patients with depression to available CBT appointments
Process: proportion of patients who complete the CBT course
outcome: percentage change in PHQ-9 questionnaire post CBT
In relation to breast screening, give quantitative examples of structure, process and outcomes
Structure: the number centres offering breast cancer screening in South Yorkshire
Process: the proportion of patients accepting a screening offer who were invited to take part
Outcome: The number of breast cancer cases detected as a result of the screening programme
In relation to hip replacement for OA, give quantitative examples of structure, process and outcomes
Structure: the ratio of orthopaedic surgeons to patients with OA needing hip replacement
Process: The number of hip replacements carried out per year in Sheffield
Outcome: complication rates of hip replacements in patients with BMI over 30
In relation to management of diabetes in GP, give quantitative examples of structure, process and outcomes
Structure: the proportion of practices with a diabetic nurse specialist in Sheffield
Process: the proportion of patients who attend their diabetic reviews in one year
Outcome: the proportion of diabetic patients with foot ulcers in primary care in Sheffield
What are the disadvantages of ‘outcomes’ (eg morbidity, mortality etc) in health service evaluation?
- link between cause and effect difficult to establish due to confounding factors
- lack of data
- poor data quality - completeness, relevance
- time lag between service provided and outcome may be long eg diabetic review and amputation, so difficult to measure
Separate from structure, process and outcome in health services evaluation, we can also think of assessing the quality of care. What are Maxwell’s Dimensions of quality?
Effectiveness Efficiency Equity Acceptability Accessibility Appropriateness (Relevance)
(3Es and 3As)
How would you explain effectiveness?
Does the intervention / service produce the desired effect?
How would you explain efficiency?
Is the output maximised for a given input (or is the input minimised for a given level of output)?
How would you explain equity?
Are patients being treated fairly?
How would you explain acceptibility?
How acceptable is the service offered to the people needing it?
How would you explain accessibility?
Is the service provided? Geographical access; Costs for patients; Information available; Waiting times
How would you explain appropriateness?
Is the right treatment being given to the right people at the right time?
Outline some qualitative methodologies used to evaluate a service
Focus groups
Interviews
Outline some sources of quantitative data for evaluating health services
Medical records
Routinely collected data - mortality, admissions
Surveys
Overall, what is the general framework used for evaluating a health service, including structure, process and outcomes?
- define what the service includes
- aims/objectives of the service
- structure, process and outcomes and Maxwell’s Dimensions of quality
- methodology to be used (qualitative, quantitative)
- results, conclusions and recommendations
How would you evaluate one aspect of mental health services in Sheffield?
- define - effectiveness of CBT
- aims - return to work, reduction in antidepressant use, improve mood, save resources eg GP appointments, improve functioning in ADLs
- structure - ratio of people with depression to CBT appointments; process - proportion of patients who complete the course of CBT; outcomes - percentage change in PHQ-9 post course
- methodology: quantitative - compare percentage change in PHQ-9 to those who have CBT and those who don’t; qualitative - people’s experiences of CBT and effectiveness focus group
- Maxwell’s dimensions - effectiveness - %change in PHQ9, efficiency - waiting times for CBT, equity - CBT access to deprived populations, acceptability - is CBT acceptable to pts; accessibility - where is it provided; appropriateness - are the right people receiving CBT or do they have complex mental health problems
Other examples of structure, process and outcome from my lecture notes…
Quantitiave- needs a denominator in all cases
structure
- ratio of diabetic nurses to patients
Process
- Proportion of patients with type 2 diabetes that attend their diabetic review
- proportion of patients with type 2 diabetes going to DESMOND
- proportion of patients who are offered breast screening that turn up to their appointment
Outcome
- proportion of patients with type 2 diabetes with foot ulcers
Qualitative
- patient satisfaction with diabetic review
- interviewing pts on the helpfulness of district nurse support
In terms of qualitative research, what is the difference between participant observation and non-participation observation?
Participant observation - people who provide the service are doing the evalulation
Non-participant observation - a researcher who does not provide the service is doing the evaluation
Think of it as - are the study participants also participants in the researchers service or not
