1c Health Care Evaluation Flashcards

Question

What should any objectives identified during a healthcare system evaluation be?

Answer 1

SMART Specific – effectiveness/efficiency/acceptability/equity Measurable Achievable – are objectives achievable Realistic (can objectives realistically be achieved within available resources?) Time- when do you want to achieve objectives by?

Answer 2

Generally, it is best to pick a framework, but the chosen framework will depend on the intervention or system you are evaluating, in general: Decide on the role of the evaluation Pick a study design Pick which measures you will use Decide how and when you will collect data

Answer 3

Donabedian E.g. evaluating a programme Structure/inputs > process > outputs > outcomes Black E.g. priority setting Effectiveness (efficacy), efficiency, equity, humanity Maxwell E.g. a screening programme Effectiveness (efficacy), efficiency, equity, access, acceptability, appropriateness

Answer 4

Randomised Trials * Randomised control trial Non-Randomised Trials: * Cohort studies * Case-control studies Ecological studies Descriptive studies Health technology assessment Qualitative studies

Answer 5

Advantages: Removes confounders Disadvantages: Expensive to run Not always practical

Answer 6

Advantages: Good for rare disease Disadvantages: Control group selection is a major form of bias

Answer 7

Advantages: Cheap Quick Useful for studying the impact of health policy Disadvantages: Crude Less sensitive

Answer 8

Examines what technology can best deliver benefits to a particular patient or population group. It assesses the cost-effectiveness of treatments against current or next-best treatments.

Answer 9

There are several problems which can arise when undertaking evaluation in practice, including: How the evaluation will sit within national and local policy context (it can be difficult to isolate an intervention or programme from the context within which it operates) Politics - Some stakeholders may have biases towards which outcomes should be reported Practicalities - finance, time, resources, level of detail required, being able to measure what you want to measure Research versus practice - ensuring that the evaluation is undertaken in a rigorous and robust way can sometimes be compromised by the practicalities of running a service The evaluation should go beyond being just an audit - ideally it should relate the separate parts of the evaluation framework to each other e.g. input to outcomes as opposed to just reviewing whether processes are occurring as they should be Assessment - it can be difficult to measure what you want to measure in practice The evaluation may be wider than existing performance management processes requiring the collection of additional data.

Answer 10

A state of complete physical, mental and social well-being, and not merely the absence of disease.

Answer 11

Health status - Health can be considered in terms of a person’s body structure and function and the presence or absence of disease or signs Quality of life - Their symptoms and what they can and cannot do i.e. the extent to which the condition affects the person’s normal life.

Answer 12

Health care is the prevention, treatment, and management of illness and the preservation of health through the services offered by health care organisations and professionals. It includes all the goods and services designed to promote health, including “preventive, curative and palliative interventions, whether directed to individuals or to populations”.

Answer 13

Health status can be measured using pathological and clinical measures and is usually observed by clinicians or measured using instruments. Types of disease measurement include: Signs e.g. Blood pressure, X-ray findings, Tumour size Symptoms Co-morbidity - Charlson Index, ICED- index of co-existing disease, adverse events (pain, readmission, complications).

Answer 14

Quality of life is a measure of the difference between the hopes and expectations of the individual and the individual’s present experience. Health-related quality of life is primarily concerned with those factors which fall within the spheres of influence of health care providers and health care systems.

Answer 15

Measures of health-related quality of life. They can be applicable across different types of diseases, medical treatments and demographic/cultural groups or they may relate only to specific diseases, interventions or population groups. Generic tools: Short form (SF)-36 WHOQOL EuroQoL (EQ5D) Nottingham health profile (NHP) Sickness Impact Profile (SIP) Disease-specific: Asthma Quality of Life Questionnaire PROMS - Measures for hip and knee replacement and varicose veins Warwick Edinburgh score (Mental health measures) Population-specific: The Child Health and Illness Profile/CHIP

Answer 16

They are measures the effect of health care on health status at a population level i.e. it is aggregated data. They may reflect a state of health at a point in time, a change in a health state over a period of time; or change in health status as a result of an intervention. In England, data on population health outcome indicators are published by the Health and Social Care Information Centre, Public Health England, Care Quality Commission and other agencies such as Dr Foster Health Intelligence Unit. At a national level, health outcomes achieved for the population of England are published through the NHS Outcomes Framework and the Public Health Outcomes Framework. The Adult Social Care Outcomes Framework also published social care outcomes.

Answer 17

Controlled high blood pressure in patients with hypertension – the intended outcome being fewer complications and sequelae of disease in patients with better-controlled blood pressure, i.e. a better health outcome Deaths within 30 days of emergency admission to hospital for stroke – the intended outcome is to avoid mortality in patients with stroke and a low mortality rate within 30 days of admission would suggest that the management and treatment of stroke is good.

Answer 18

Deprivation scores can be ascribed to an area or population to give a picture of its social and economic status.

Answer 19

No, it is ecological data and so ascribing an area’s characteristics to the individual can lead to ecological fallacy.

Answer 20

National statistics socio-economic Classification (NSSEC) Index of Multiple Deprivation Jarman score DETR 2000 Townsend Material Deprivation Score Carstairs

Answer 21

Income Occupation Years of education Housing Ownership of various goods such as car, washing machine etc.

Answer 22

Pros: Can be aggregated by population-weighted averages. A broad range of measures Cons: Cannot make direct comparisons between 2015 IMD and previous version because scores are calculated differently by year. Can only comment on relative rank differences between years.

Answer 23

Pros: Can be used for small areas. Diverse range of measures. Cons: Differences between wards are masked. Data is linked to last census (2001) so can be out of date. Does not indicate the proportion of people in an area who are deprived. Biased towards urban population.

Answer 24

Pros: Data is updateable Can distinguish between different aspects of deprivation. Gives the proportion of people that are deprived for the income and employment domains. Cons: Some domains (housing) derived from only a few data sources. Variables are weighted differently though justification is not clear. Access score is more reflective of rural rather than urban need. Data is not available below ward level. District averages are applied to ward level in some instances.

Answer 25

Pros: Can be used to look at small areas. Highly correlated with measures of ill-health. Easy to calculate because sum of standardised scores. Cons: Data can be out of date as linked to last census- particularly in terms of housing tenure. Does not indicate the proportion of people in an area who are deprived. Better indicator of deprivation in urban areas that rural areas.

Answer 26

Pros: Based on census so allow objective results for whole population. Can be used to look at small areas. For an outcome such as low birthweight, Carstairs scores have been shown to perform better than individual social class in describing the extent of inequalities in the population. Cons: Data can be out of date as linked to last census- particularly in terms of housing tenure. Better indicator of deprivation in urban areas that rural areas.

Answer 27

Quality assessment is the data collection and analysis through which the degree of conformity to predetermined standards and criteria are exemplified. Quality assurance is the process of guaranteeing quality. If quality indicators are being measured and quality standards are reached, through the quality management process, then quality assurance can be given.

Answer 28

A systme of policies and protocoles then

Answer 29

Indicators of success in the early, mid and end stages of an intervention or programme. Such indicators need to be reported and fed back into the loop so that quality improvements can be made continually.

Answer 30

Healthcare organisations’ performance against national targets and indicators GPs - performance as measured by quality outcomes framework

Answer 31

Some examples of quality assurance. National Cancer Screening programmes quality assurance guidelines. NICE guidance and Health Technology Assessments

Answer 32

Clinical governance is the quality management system for ensuring clinical quality within an organisation, for example, ensuring that processes/policies/protocols are in place to ensure that the correct kidney is removed and that these processes/policies/protocols are put in action to ensure good clinical outcomes and to maximise patient safety.

Answer 33

Whilst clinical guidelines set out appropriate steps in a patient pathway usually based on systematic reviews of evidence, actually assessing clinical outputs in terms of quality is difficult. It is difficult to know what to benchmark against – should it be average performance or should there be a threshold? Should there be a minimum safe standard that many will achieve or an aspiring standards of excellence?

Answer 34

It must be clear how they can be achieved by individuals It must be an achievable standard

Answer 35

Equality = sameness Inequality = unequal Equity = fairness Inequity = unfair or unjust

Answer 36

Effective care: clinical evidence supports different treatments Preference-sensitive care: where alternative valid treatment options are available and variations are caused by patients exercising choice Supply-sensitive care: variations reflect the different availability of alternative treatment approaches.

Answer 37

Measuring health need (measuring demand through health needs assessment - see section ‘Participatory needs assessment’) Measuring access to health care (measuring supply and demand) health care use e.g. avoidable admissions, barriers to admission e.g. waiting lists, patients’ perceptions of healthcare provision Measuring quality of health care (see section ‘Principles of evaluation’).

Answer 38

The Gini-coefficient of inequality is a commonly cited measure of inequality and is shown by a Lorenz curve of income distribution. The Lorenz curve (B) plots the share of cumulative income enjoyed by the relative share of the population – on the curve shown below, 40% of the population obtains 20% of the total income. Curve (A) is the line of total equality whereby 40% of the population obtain 40% of the total income. The difference between the 2 curves, the Gini-coefficient, indicates the degree of inequality of distribution. A co-efficient of 0 reflects complete equality whilst 1 indicates complete inequality.

Answer 39

Horizontal equity refers to equity between people with the same health care needs, whilst vertical equity refers to those with unequal needs who should receive different or unequal health care. Vertical inequity can be justified in healthcare if morally relevant factors are causing this.

Answer 40

Moral: Need Ability to benefit Autonomy Deservingness Immoral: Age/sex Ethnicity Income, class Disability, genetics

Answer 41

Clinical audit is the review of a process or practice or outcomes against a pre-agreed set of standards and forms part of the quality improvement process. It is central to improving quality standards within healthcare and is an important part of maintaining high standards and patient safety. Clinical audit should form an intrinsic part of clinical activity - it should not be a one-off random process and should be conducted regularly to monitor for improvements (or failures) in clinical practice. Clinical audit is a key pillar of clinical governance and should be carried out by all healthcare staff.

Answer 42

Stage 1: Preparing for audit: * Define the purpose of the audit * Involve service users in the audit process * Provide the necessary structures for the audit * Identify the resources required for undertaking the audit (people, skills, training, funding, equipment) * Include service user involvement where appropriate Stage 2: Selecting audit criteria Criteria must explicitly define what is being measured and must represent elements of care which can be measured objectively. Criteria must be valid and appropriate. Criteria can be selected from: * Guidelines (e.g. NICE guidelines) * Standards based on professional or consensus opinion * Service user-defined measures (e.g. outcome measures) * Set performance levels * Care pathways Stage 3: Measuring level of performance Planning data collection: * Inclusion criteria – i.e. who should be included in the audit (with any exceptions to be noted) * Which healthcare professionals should be included in the audit i.e. who provides the patient’s care for the audit * Sample size * The time period over which the criteria apply * Data sources to use - consider retrospective versus prospective collection Plan data analysis * Consider and plan for any data protection issues Stage 4: Making improvements * Disseminating results * Changing behaviours Stage 5: Sustaining improvement * Regular audit * Clinical performance indicators

Answer 43

Sentinel Stroke National Audit Programme (SSNAP) National Emergency Laparotomy Audit (NELA)

Answer 44

Confidential enquiries are “confidential” in that details of the patients/cases remain anonymous, though reports of overall findings are published. The purpose of a confidential enquiry is to detect areas of deficiency in clinical practice and devise recommendations to resolve them. Enquiries can also make suggestions for future research programmes. Most confidential enquiries to date (at both national and local level) are related to investigating deaths, to establish whether anything could have been done to prevent the deaths through better clinical care. Confidential enquiries are “confidential” in that details of the patients/cases remain anonymous, though reports of overall findings are published.

Answer 45

The confidential enquiry process goes beyond that of audit, in that the details of each death or incident are critically reviewed by a team of experts to establish whether clinical standards met (similar to the audit process) and also that the right clinical decisions were made in the circumstances (audit is unlikely to include this aspect).

Answer 46

England's recent national-level confidential enquiries include: National Confidential Enquiry into Patient Outcome and Death (NCEPOD) - Sepsis (2015). Care for patients aged 16 years or older with sepsis was reviewed. Areas where the clinical and organisational care of these patients care might have been improved were identified. MBRRACE-UK Perinatal Confidential Enquiry Report – Term, Singleton, Normally-formed, Antepartum Stillbirth (2015). Cases were reviewed in detail against national care guidelines by a panel of clinicians, including midwives, obstetricians and pathologists who considered every aspect of the care. National Confidential Inquiry into Suicide and Homicide (NCISH) by people with mental illness (2015). The report presents findings from 2003 to 2013 and highlights areas of health care where safety should be strengthened across a range of organisations.

Answer 47

Confidential enquiries can also be conducted at a local level in a similar way whereby a panel of locally appointed experts review clinical practice to ascertain whether any deaths or adverse events were avoidable. Recommendations for action are then made and implemented. Examples of local confidential enquiries include drug related deaths.

Answer 48

Consensus methods provide a means of synthesising information. Their main purpose is to define levels of agreement on controversial subjects. The 3 best-known consensus techniques are: * Delphi process * Nominal group technique * Consensus development conference

Answer 49

The Delphi method is a systematic interactive way of gaining opinions/forecasts from a panel of independent experts over 2 or more rounds. It is a type of consensus method which does not require face-to-face meetings. The Delphi process aims to determine the extent to which experts or lay people agree about a given issue and with each other and in areas where they disagree, achieve a consensus opinion. Delphi technique is usually conducted through questionnaires. Where focus groups purposely use group dynamics to generate debate on a topic, Delphi methods maintain anonymity of the participants, even after the study.

Answer 50

The following steps outline how to undertake a Delphi study: 1. Design the questionnaire 2. Invite participants to take part 3. Send out first round of questionnaire (see example below) 4. Analyse responses from round 1 questionnaire 5. Prepare the second round questionnaire 6. Send out second round questionnaire (see example below) 7. Analyse responses from round 2 questionnaire 8. Design the third round questionnaire. For the third round, the second questionnaire is repeated but incorporates scores from the second questionnaire results. This gives participants a chance to see how the rest of the group prioritised the areas and if the participant then wants to change their opinion on the basis of the group consensus, has the opportunity to do so. 9. Analyse the results of the third round questionnaire for agreement and degree of consensus 10. Report findings

Answer 51

Strengths: A rapid consensus can be achieved Participants do not have to be in the same room together to reach agreement Individuals are able to express their own opinions as opposed to “Group think” Can include a wide range of expertise Relatively low cost to administer and analyse There is the potential to gain large quantities of data Offers a method which can be used where data are lacking Weaknesses: Does not cope well with widely differing opinions or large changes in opinions (paradigm shifts) The facilitator’s view may dominate in the analysis Differing opinions may not be sufficiently investigated Can be time-consuming Needs high participant motivation Success of the method depends on the quality of the participants The written response format may be less suitable for some potential respondents

Answer 52

Equity and excellence: Liberating the NHS, a white paper published in July 2010, which outlined plans to make patient experience a measurable outcome of care, along with introducing a ‘no decision about me without me’ ethos set a clear direction of a greater focus on acceptability of healthcare services. Including patient experience as a measurable outcome of care in the NHS Outcomes Framework (and other initiatives) has also drawn attention to this aspect of care at all levels within healthcare organisations.

Answer 53

Friends and Family Test ‘I’ statements GP Patient Survey Care Quality Commission patient experience surveys Patient complaints and compliments (Through the Patient Advice and Liaison Service)

Answer 54

A website where patients can look up their local hospitals and see how other patients rated their experience for given areas of care along with any Care Quality Commission assessments.

Answer 55

NHS Choice - Patient can look at results NHS e-Referral Service - Patients can select where they are referred to Payment by Results - Makes up some of the criteria involved in resource allocation.

Answer 56

Patient Reported Outcome Measures (PROMs) assess the quality of care delivered to NHS patients from the patient perspective.

Answer 57

Allow comparison of the relative clinical quality of providers of elective procedures, for clinicians, managers, commissioners, regulators, clinical audits, patients and GPs. Gives a better understanding of what works. The efficacy and cost-effectiveness of different technical approaches. Assess referral thresholds. PROMs data used to establish whether referrals for elective procedures are appropriate.

Answer 58

Primary Unilateral Hip Replacement Oxford Hip Score Primary Unilateral Knee Replacement Oxford Knee Score Groin Hernia Repair None Varicose Vein Procedures Aberdeen Varicose Vein Questionnaire Patients are asked to self-complete the questionnaires without assistance on the day of admission. Patients who are considered incapable of completing PROMs may be excluded from the data collection process (as judged by staff members) though any reasons for exclusion must be documented. Patients are subsequently asked to complete a post-operative PROM at an appropriate time after the intervention (sent via the post), which is likely to be 3 months for groin hernia repair and varicose vein surgery, and 6 months for hip and knee replacements.

Answer 59

Primary = Prevent Secondary = Screen Tertiary = Treat Primordial prevention - Seeks to prevent the risk factors for disease. For example, a child seeing their parents smoke cigarettes may wrongly consider this a good lifestyle choice for later in life: advising parents to quit smoking in such circumstances can be considered primordial prevention. Primary prevention - Prevention of disease through the control of exposure to risk factors. Strategies for primary prevention include population-wide strategies and targeted, high-risk strategies focusing on population sub-groups. For example, careful weight control prevents obesity which in itself is a risk factor for many conditions including heart disease and diabetes. Secondary prevention - The application of available measures to detect early departures from health and to introduce appropriate treatment and interventions. Screening is a major component of secondary prevention for example, cervical screening for women to detect early changes which may go on to lead to cancer of the cervix. Tertiary prevention - the application of measures to reduce or eliminate long-term impairments and disabilities, minimising suffering caused by existing departures from good health and to promote the patient’s adjustments to his/her condition. For example, a person identified as having type 2 diabetes will have regular blood glucose checks to monitor control of their diabetes and prevent complications of the disease.

Answer 60

There are two approaches to prevention - targeting a whole population whether they are exposed to risk factors or not, or tackling only those identified as being high risk.

Answer 61

Aka the Rose hypothesis (1992) Since diseases are rare, most individuals who adopt a behaviour designed to lower their risk of disease will not benefit directly, although a few individuals may benefit enormously. For example, any one person’s decision to lose weight may only have a small impact on that person’s risk of disease in the near future, but if many people each lose a little weight, this may have a substantial impact on the community’s obesity-related disorders.

Answer 62

Advantages: May be more cost-effective than population-wide approaches Those who are identified as being high risk may be more motivated to change their behaviour than the whole of society Easier for health professionals to promote change on an individual basis Individuals are usually aware of their exposure to adverse risk, whereas in society not everyone will have been exposed Society prefers focusing on individuals to change rather than a whole population Disadvantages: Can be expensive to identify and treat those at increased risk Fails to address public health problems arising from small but widespread risks that may be substantial Ignores the point that a large number of people exposed to a small risk may generate more cases than a small number of people exposed to a large risk Tends to medicalise prevention Strategies for the individual tend to be either palliative or temporary Does not focus on what influences behaviour Does not tend to predict an individual’s change in risk May have little overall impact on control of disease

Answer 63

Advantages: Recognises that society influences individual behaviour Risk reduction can be achieved at population rather than individual level In situations where there is a dose-response relationship in terms of risk and exposure, shifting the entire population distribution towards lower levels of exposure is effective. Disadvantages: Is less effective in situations where there is not a dose-response relationship in terms of risk and exposure.

Answer 64

Both have their own positives and negatives. Whilst the high-risk approach seemingly has many more disadvantages compared to the population approach, the prevention paradox reduces the effectiveness of the population approach, therefore a combination of population and high-risk approaches is usually most effective.

Answer 65

Screening Genetic testing Analytical studies linking risk factors and disease and ecological studies to identify groups

Answer 66

Health Impact Assessment (HIA) is a combination of procedures, methods and tools by which a policy, programme or plan may be judged as to its potential effects on the health of the population and the distribution of those effects within the population. HIA is a decision-making tool and is designed to take account of the wide range of potential effects that a proposal may have on a target population. It considers relevant evidence, takes into account opinions, analyses the potential health impacts of the proposal to enable informed decision making.

Answer 67

Prospectively, concurrently or retrospectively. Prospective HIA is usually the most useful as health impacts can be considered before commitments made.

Answer 68

The design and development of a policy or strategy The commissioning of services Resource allocation Community participation and service user involvement Community development and planning Preparing funding bids

Answer 69

1) Screening: a selection process which assesses policies, programmes and projects for their potential to affect the health of the population. It offers a systematic way of deciding whether a HIA is worth doing. Considerations during screening include: * Economic issues - size of the project and the population affected; and the costs of the project and their distribution * Outcome issues - nature of potential health impacts of the project; likely nature and extent of disruption caused to communities by the project; existence of potentially cumulative impacts * Epidemiological issues - degree of certainty (risk) of health impacts; likely frequency (incidence/prevalence) of potential health impacts; likely severity of potential health impacts; size of any probable health service impacts; likely consistency of “expert” and “community” perceptions of probability, frequency and severity of impacts. 2) Scoping: usually a steering group encompassing all the organisations involved will be formed and will set the boundaries for appraisal of health impacts. They will also agree the way in which the appraisal will be managed and allocate responsibility for decision-making. 3) Appraisal: this is the main part of the HIA and can be rapid, intermediate or comprehensive. To ensure that the views of local communities are heard a comprehensive HIA is the most effective. Appraisal includes analysing the policy, programme or project; profiling the affected population; identifying and characterising the potential health impacts, looking at the evidence base and making recommendations for the management of the impacts. 4) Presenting results: unless total consensus is reached, results should be presented as a range of options. 5) Decision-making: The ultimate result will be an agreed set of recommendations made by the steering group for modifying the project such that its health impacts are optimised. 6) Implementing, monitoring and evaluating: impacts of HIA processes are monitored to enhance the evidence base for future HIAs. Outcome evaluation is constrained by the fact that negative impacts which have been successfully avoided due to the modification of the project will not be clearly identifiable. Other beneficial outcomes include better partnership working.

Answer 70

A more formal, quantitative form of impact assessment is often referred to as quantitative risk assessment (QRA). By convention, this is described as having four stages: Hazard identification Based on review of scientific literature, including toxicological studies and epidemiology, as available. Exposure-response assessment Again based on toxicological and/ or epidemiological data, usually entailing extrapolation of exposure-response curve, and giving rise to uncertainties with respect to thresholds/ intercepts; shape of the exposure-response (extrapolation); and the relationship between risk, intensity and duration of exposure. Exposure assessment Direct measurement is often difficult or costly, so this is often based on indirect, semi-quantitative assessment using: type of exposure, duration, intensity etc. An exposure matrix may be produced that estimates exposure levels for various population groups defined by location, age etc. Risk characterisation This summarises patterns of risk based on information from Steps 2 and 3 above. QRA draws on all the available evidence, requires little or no new measurement, and generates results which are statistically stable. But estimates are indirect (both with respect to exposure & health impacts), and typically many assumptions are necessary (e.g. extrapolations across dose ranges, sometimes from one species to another etc).

Answer 71

An (EIA) is an assessment of the possible impact of a programme or project on the natural environment. The assessment ensures that decision makers consider any possible environmental impacts prior to deciding whether to proceed with a project. Environmental impact assessment enables environmental factors to be given due weight, along with economic or social factors. It helps to promote a sustainable pattern of physical development and land and property use in cities, towns and the countryside.

Answer 72

An environemtal impact assessment has the same steps as a health impact assessment. Screening Scoping Appraisal Presenting results Decision-making Implementing Monitoring and evaluating

Answer 73

Examples of EIAs include those undertaken in relation to proposals for building new dams, constructing new run ways or on a more local scale, a town by-pass.

Answer 74

European Union legislation on EIA of the effects of projects on the environment was introduced in 1985 and was amended in 1997, 2003 and in 2009. For some types of project EIA is a mandatory part of the planning process whereas for other projects, EIAs are only required if the particular project in question is judged likely to give rise to significant environmental effects (screening stage of the process).