Evaluating complex interventions

Question 1

What is a complex intervention and in what circumstances is one required?

Answer 1

• A SET of PLANNED ACTIONS with many COMPONENTS which are designed to BRING ABOUT a desired CHANGE which will address a health/social ISSUE
• They are used widely in the health service and in public health practice.
• They are required to address important factors of society which have health consequences, such as education, transport, and housing.
• Consists of many components which act on their own and in conjunction
• They often test ‘theories of change’ i.e. why some people continue to smoke even though they know it is bad for them.

Question 2

What are the 5 stages of developing and implementing a complex intervention?
(Give a study example)

Answer 2

Football Fans in Training (FFIT) programme by Wyke et al 2014. This was an RCT of healthy living programme involving group based weight management compared to not group training. People were invited along to their local football club with men of similar age and weight and received lifestyle advice and training from football coaches to help with weight loss.

IAMOO!

Input - Recreational (clubs, coaches, equipment) and Financial (paying coaches, club hire, time)

Activities - motivation, reassurance, training, food plans

Maintenance - value, support encouragement

Outputs - mutual learning, peer relationships, skills and competence, happy people

Outcomes - short term (better physical and mental wellbeing) and long term (learned health skills to share with others, better long term health outcomes)

Question 3

How are complex interventions evaluated?

Answer 3

Medical Research Council framework - this was developed with 4 stages and is used to evaluate a complex intervention.

1) Development
- identifying evidence base
- creating theory/plan
- make a model of the process and outcomes
- what questions need to be addressed by the intervention?
- choose study design

2) Piloting and feasibility
- determine sample size
- methodology e.g. methods of recruitment
- test procedures

3) Evaluation
- assess EFFECTIVENESS!
- understand change process
- 2 types; cost EFFECTIVE analysis (CEA) and cost UTILITY analysis (CUA)

4) Implementation
- dissemination
- surveillance and monitoring
- long term follow up

Question 4

How is cost effectiveness assessed?

Answer 4

1) CEA = cost effective analysis (measures cost per unit time)
2) CUA = cost utility analysis (measures cost per QALY) (quality adjusted life year)

You use both of these to create an ICER (incremental cost effectiveness ratio) to work out its overall cost effectiveness

Question 5

Give detail on the FFIT programme:

Answer 5

WYKE et al 2014!

A Scottish health survey 2017 showed that 78% of men aged 35-65yrs are overweight/obese.

Gough et al 2007: men think that dieting and slimming are “womens” activities and real men don’t diet

Aims:
- Assess the effectiveness and cost effectiveness of the training programme

What is it:
- A group based weight management and healthy living programme

Design:

• 2 arm pragmatic randomised control trial
• Individuals were randomised 1:1 ratio, stratified by club
• Intervention group started the trial within 3 weeks of randomisation, and the control group were put on a 12 month waiting list
• Involved 13 professional football clubs in Scotland
• key components: food, exercise, alcohol education, long term behaviour change strategies (changing habits, dealing with relapse)

Evaluation:
- CUA (cost-utility analysis) carried out by measuring ‘increment cost’ per ‘quality adjusted life year’ within the trial over individuals lifetimes

Participants:
- 747 men from 35-65yrs with BMI >28kg/m2

Result:

• The main difference in weight loss between the groups at 12 months was 4.94kg, with significant P-value therefore the result favoured the intervention
• It was cost effective and attracted men who were at risk of ill health in the future

Further research:

• To investigate if participants retained weight loss in the long term (LONG TERM FOLLOW UP)
• How the programme could be optimised in relation to effectiveness and intensity of delivery (in the trial there was weekly sessions)
• DISSEMINATION (conferences, publishing)

Question 6

What is PICO?

Answer 6

A template used to assess the effectiveness of a clinical intervention, used to plan out what you will do:

1) Population (who?) e.g. obese men who are football fans
2) Intervention (what?) e.g. FFIT programme
3) Comparison (control?) e.g. group of men not recruited in the programme (put on 12 months waiting list)
4) Outcome (result?) e.g. successful weight loss in the intervention group

Question 7

What is a ‘theory’?

Answer 7

• not a guess, but a basket/framework of facts that can be used to plan a new intervention
• based on long term knowledge that has been repeatedly confirmed through experiment

Question 8

What is a ‘theory of change’?

Answer 8

• a description of how and why a change is likely to occur in a specific context
• FIRST, identifies the long term goal, then works backwards to identify the conditions that need to be in place for the goal to occur

Question 9

Describe the direct non-medical costs that can arise from an intervention:

Answer 9

• patient transport

- home care

Question 10

Describe the direct medical costs that can arise from an intervention:

Answer 10

• drugs

- in patient procedures

Question 11

Describe the indirect tangible costs that can arise from an intervention:

Answer 11

• premature death

- time off work/sick leave

Question 12

Describe the indirect intangible costs that can arise from an intervention:

Answer 12

• cost of pain and suffering

- loss of leisure time