Clinical Audit and Service Evaluation

Question 1

Describe clinical audit and differentiate it from research.

Answer 1

Clinical Audit:
“a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change”

Research:
“the attempt to derive generalisable new knowledge by addressing clearly defined questions with systematic and rigorous methods”

Question 2

Describe audit questions. with examples

Answer 2

Does this service reach a predetermined standard?

• What proportion of pts presenting with an STI are tested for HIV: compliance with national guidance.
• What proportion of pts referred for urgent investigation of colorectal cancer are seen within 2 weeks?
• What proportion of pts with diabetes have a 6-month review by their GP or practice nurse?
• What is the effect of a pharmacist-led educational intervention on the administration of medication to acute surgical patients termed “nil by mouth”?

Question 3

Describe research questions with examples.

Answer 3

Qualitative Research:

• What is the patient experience of receiving this service?
• Example: What are the concerns of women being recalled for a repeat cervical smear?

Quantitative Research (hypothesis test):

• How do patient outcomes compare between this service (or treatment) and an alternative?
• Example: What is the impact of a nurse-led dermatology clinic in primary care on the quality of life of children with eczema?

*Parts to a (quantitative) research question: PICO (Patient / Intervention / Comparison / Outcome)

Question 4

Describe service evaluation questions with examples.

Answer 4

Does the service achieve its objectives?

• How ?
• What are the costs?

Does the service (still) meet pt’s needs? Is the service equitable?

Examples:
• What are the waiting times for pts attending the A&E department in XXX hospital?
• What are the patient experiences of attending a community pulmonary rehabilitation service?
• Pt satisfaction surveys (or surveys of satisfied pts?)

Question 5

Describe the purpose of service evaluation?

Answer 5

Define or judge current care

Question 6

Describe the question in research?

Answer 6

Test hypothesis (quant), explore themes (qual)

Question 7

Describe the questions in audit?

Answer 7

Does it reach a pre- determined standard?

Question 8

Describe the question in service evaluation?

Answer 8

What standard does this service achieve?

Question 9

Describe the objective in research?

Answer 9

Specific research objectives

Question 10

Describe the objective in audit?

Answer 10

To measure service against accepted / defined standard

Question 11

Describe the objective in service evaluation?

Answer 11

To measure current service without reference to a standard

Question 12

Describe the interventions and allocation to intervention in research.

Answer 12

Interventions: Novel use or application

Allocation to interventions: Yes, usually

Question 13

Describe the interventions and allocation to intervention in audit.

Answer 13

Interventions: Already in use

Allocation to interventions: No

Question 14

Describe the interventions and allocation to intervention in service evaluation.

Answer 14

Interventions: Already in use

Allocation to interventions: No

Question 15

Describe the data additional to usual care and randomization in research.

Answer 15

Additional date: Yes, including invasive tests

Randomization: Maybe

Question 16

Describe the data additional to usual care and randomization in audit.

Answer 16

Additional date: Can include questionnaire / interview

Randomization: No

Question 17

Describe the data additional to usual care and randomization in service evaluation.

Answer 17

Additional date: Can include questionnaire / interview

Randomization: No

Question 18

Describe the data additional to usual care and randomization in service evaluation.

Answer 18

Additional date: Can include questionnaire / interview

Randomization: No

Question 19

Describe the question-driven aspect of research and audit.

Answer 19

Research: what should we be doing ?
Audit: are we doing it right ?

Question 20

Describe ethics in research and audit.

Answer 20

Research – Yes

Audit – No, but may be needed if it is judged to put pts at risk If uncertain, seek advice from the ethics committee

Question 21

What are the key factors in distinguishing research and audit?

Answer 21

Intent: to generate new knowledge (RESEARCH)
to measure performance against a standard of care (AUDIT)

Allocation of treatment or service if by protocol (RESEARCH)

Question 22

Describe the main methods in audit.

Answer 22

Retrospective – notes review (beware missing data)

Prospective – ongoing data collection (beware the “Hawthorne effect”)

Question 23

Describe the processes (steps) in audit.

Answer 23

Steps:
1) Identify a topic / problem
2) Identify local resources (? local audit dept)
3) Choose the standard, create the audit proforma 
4) Define the sample
5) Collect data
6) Compare data with the
standard
7) Develop and implement
change 
8) Re-audit

Question 24

Describe the main barriers of audit.

Answer 24

1. Lack of resources
2. Lack of expertise or advice in project design & analysis
3. Problems between groups and group members
4. Lack of an overall plan for audit
5. Organisational impediments

Question 25

Describe the key facilitators of audit.

Answer 25

1. Modern medical records systems
2. Effective training
3. Dedicated staff
4. Protected time
5. Structured programmes
6. Shared dialogue between purchasers & providers

Question 26

Describe the criteria for choosing a topic.

Answer 26

• Is the topic of high cost, volume, or risk to staff or users?
• Is there evidence of a serious quality problem, for example patient complaints or high complication rates?
• Is evidence available to inform standards, for example systematic reviews or national clinical guidelines?
• Is the problem amenable to change?
  Is the topic a priority
  for the organisation?
  for a national policy initiative?
• Is there potential for involvement in a national audit project?

Question 27

Describe the criteria for choosing a standard.

Answer 27

Agree the standard (minimal, ideal or optimal):

• Minimal: lowest acceptable level of performance
• Ideal: the care possible under ideal conditions
  (e. g. 100% survival ???) – elusive!
• Optimal: realistic under normal conditions of practice (somewhere between minimal and ideal)

Is it evidence based and related to aspects of care?

‘Evidence-Based Practice’ versus ‘Practice-Based Evidence’

Question 28

Describe the advantages of audit.

Answer 28

1. Improved communication among colleagues
2. Improved pt care
3. Increased professional satisfaction
4. Better admin & data recording

Question 29

Describe the disadvantages of audit.

Answer 29

1. Diminished clinical ownership
2. Fear of litigation
3. Hierarchical and territorial suspicions
4. Professional isolation

Question 30

Compare the features of audit and research.

Answer 30

Audit:
• Never involves a new treatment
• Never involves allocation of pts to groups
• Should not involve anything being done to pts beyond their normal care
• Simple stats
• Results of local relevance
• Influence on clinical practice • locally
• Some aspects may require review by an ethics committee

Research:
• May involve pts getting a new treatment
• May involve pts being allocated to different groups
• May involve pts receiving experimental interventions (drugs, other treatments)
• Can involve complex stats • Results generalisable
• Influence on clinical practice everywhere
• Always requires ethics review