Session 7 - Quality and Lab Management Flashcards
What is the definition of “Audit”
A systematic, documented and independent process to obtain evidence and objectively evaluate the extent to which a set of criteria are fulfilled.
What are the 5 stages of the audit cycle?
Set standards Measure current practice Compare practice vs standards Reflect, plan, implement change Re-audit
What are the purposes of audit?
To demonstrate quality of service Identify areas for change Continued improvement Assist in implementation of guidelines and services Monitor consistency of performance Measure performance against benchmark
Name the 4 classes of Audit.
- Clinical audit
- Horizontal Audit
- Vertical Audit
- Examination audit
Describe the 4 classes of audit and give an example of each
Clinical audits - they form part of clinical governance, they are required to provide evidence of current practice against national guidelines and monitor patient outcomes. They can be prospective or retrospective. e.g. referral patterns or TATs
Vertical audits - allow audit of an entire workflow/process. Used in labs to follow one sample from receipt to report - record everything (temperature of fridges, lot numbers, staff training records etc.)
Horizontal audits - examine one part of the process in detail. Can be used to determine if all batch records are held in the right place.
Examination audits - Witness to see if a SOP is being correctly followed, or staff members are appropriately trained.
What should be contained within an audit record?
Name of staff member performing the audit Date performed Reference number The scope of the audit Any non-conformaties Any CAPAs Date and signature on completion
What is a Quality Management System?
A defined set of procedures, structures, regulations and responsibilities that ensure the organisation is capable of delivering required standard of services
What are the benefits of a QMS?
Improve and monitor lab performance
Achieve better control of processes
Safeguard the organisation, public and staff
Ensure minimum standard quality service for patients
Ensure the lab is managed efficiently and effectively to meet users’ needs
Provides a framework for staff to ensure good quality service
To ensure all labs have standard sets of practices with common goals
Achieve accreditation
What are the 9 components of the QMS?
Quality manual QMS documentation Quality Assurance - getting it right first time Internal QC Quality Improvement Continual Improvement cycle Quality Assessment Audit Accreditation
Give examples of QMS documentation
Quality Manual Policies Calibration certificates COSHH forms SOPs H&S records Meeting minutes Job descriptions and staff records Staff training records and CPD Appraisal records Maintenance records Batch records Incident forms Audit records Verification/Validation records Test results/reports
How does a QMS stop errors occurring?
Ensures all aspects of a process are considered
Documents procedures to ensure standards are met
Encourages review of techniques to ensure up-to-date methods are used
Controls SOP changes
Provides platform for continuous improvement
What does UKAS value, in terms of quality improvement?
PRIDE: Professionalism Responsibility Innovation Delivery Excellence
What are CAPA?
Corrective and Preventative actions.
Corrective actions eliminate the cause of the non-conformity
Preventative actions stop a potential non-conformity
Outline the Continuous Improvement Cycle
Plan (determine user requirements and plan how these will be met)
Do (acquire resources to meet requirements, document procedures, provide traiinng, implement working procedure)
Check (audit to ensure activities are carried out correctly)
Act (correct and prevent when things go wrong)
List standards applicable to a diagnostic laboratory
ISO - ISO15189:2012 (medical laboratories) CPA standards (superseded by ISO) HCPC standards for staff Professional BPGs UKNEQAS codes of practice, EMQN, WEQAS... NHS standards, NICE guidelines Human Tissue Act (2004) EuroMRD (new MRD test for haemonc)
What is the role of UKAS?
UKAS is the sole national body responsible for assessing labs against the internationally agreed standards.
What are ILAC and IAF? What do they do?
ILAC is the International Laboratory Accreditation Cooporation
IAF is the International Accreditation Forum.
They work together to ensure that accreditation body members (such as UKAS) only accredit bodies that are competent to do their work.
What do the UKAS standards 15189:2012 cover?
Equipment Reagents Staffing Documentation - SOPs and document control Premises Validation of test and services Quality management QA CAPAs and non-conformity records
What are the three stages of UKAS assessment?
Pre-assessment
On-site assessment
Post-assessment monitoring/surveillance/re-assessment
What are the potential outcomes of a UKAS inspection?
Immediate accreditation - no action needed
Accreditation offered, subject to satisfactory improvements by the lab
Accreditation for a reduced schedule - if lab fails to demonstrate competence in one or more areas
Accreditation refused - discuss with UKAS
List some EQA schemes currently available to UK genetics labs
NEQAS
CEQAS
EMQN
EuroMRD
How are NEQAS results scored?
Each criteria marked out of 2.0:
genotyping
interpretation
clerical accuracy
What is indicative of a poor performer in the NEQAS scheme?
Mean genotyping score <1.6
Mean interpretation score <0.7x the mean score of all labs
What is the process undertaken if lab are identified as a poor performer?
Genotyping: participant notified and given a defined timescale in which to respond. Extra round of EQA supplied - if performance satisfactory, poor performer status removed.
Interpretation: Possible extra round of EQA, most likely remains PP until next round of EQA
What happens if a lab is identified as a persistent poor performer?
Persistent PP classed as poor performance for a disease in 3 out of ANY consecutive 6 rounds OR poor performer in 2 consecutive rounds
Remedial action defined by NQAAP panel, possible contact of regulatory body - UKAS/CPA. Give advice to head of lab.
How is persistent poor performance defined in the CEQAS scheme?
2 poor performances over three distributions of material within a 36 month period
1 poor performance within 1 year of a previous persistent PP
Summarise the changes between ISO standard and previous CPA standards.
Updated clauses Measurements of uncertainty Staff competence Validation/verification Purchasing Traceability Lab director responsibilities EQA/IQC
New clauses: Evaluation and risk management Staff suggestions Information management Equipment records Service agreements
Define Validation
The process by which performance specification, accuracy and limitations are defined. Results compared against controls and known standards.
Define verification
The comparison of a modified performance specification with an existing specification.
What are the 15 considerations to be taken into account when deciding on whether to introduce a new test into the laboratory?
- Is there a clinical need for the test? - # of samples, disease incidence etc.
- Is there a clinical demand for the test? - who will order the test, where would referrals come from?
- Does the test have clinical utility? - would results affect treatment or health outcome, low penetrance, environmental effect?
- Is the test ethical/legal?
- Technical testing strategy - what type of test? what sample type? mutation types? Accuracy, reliability, sensitivity/specificity etc. Screen or diagnostic?
- Impact on current working practices - potential workload, staffing, set-up time
- Will the new test replace an existing test? - improve TAT? improve DxYx, how much DNA/cells needed? controls needed? externally or internally validated test? confirmation of results?
- Budget considerations - staffing, consumables, equipment, competitive pricing, need a business case? NCG funded test?
- Implications for staffing and training requirements
- Equipment/lab set-up - which techniques are best, new or existing equipment?
- Population factors
- Background work - disease information etc
- Validation work - including SOP writing, audits, enrol in EQA scheme
- Implement service - training other staff, ongoing improvements, authorise SOP, worksheets and information management, report templates, advertise service, gene dossier?
- Request user feedback once test established - user surveys
What is the role of UKGTN?
It is an advisory body which provides information about genetic testing services to support commissioning of services.
What functions does UKGTN fulfil?
Identify lab services Lab membership - need to meet criteria (accreditation etc) Evaluate new tests Oversee a transparent process Offer an online test directory Influence national policy
What is the UKGTN CSAG?
Clinical and Scientific advisory group.
Made up of 4 working groups: Genetic test evaluation group (receives gene dossiers) Lab membership and audit Development working group Rare disease service improvement group
What is the purpose of a UKGTN gene dossier?
Set of 44 questions to provide comprehensive information for the genetic test evaluation working group. Covers reasoning behind offering the test
What criteria are used in the UKGTN’s assessment of tests? Describe the framework
The ACCE framework. Analytical validity Clinical validity (PPV and sensitivity/specifcity) Clinical utility Ethical, legal and social
What is the ~ equivalent of the UKGTN in the USA?
The CDC established the EGAPP in 2004 to offer the same service as the UKGTN in the UK.
What is the role of NICE in genetic testing?
Producing evidence based guidelines
Develop quality standards of care and testing, and performance metrics
Provide information to commissioners and practitioners
Give some examples of NICE’s role in genetic testing
CG128:Autism - discusses use of aCGH
NG14: Melanoma - discusses use of genetic testing
Developed quality standards for Breast Cancer
Technology appraisal: TA34 for Trastuzumab for the treatment of Her2+ tumours
Diagnostics and Medical Technologies Evaluation Programme - evaluates new diagnostics and technologies for NHS use
Define clinical governance
Clinical governance is a system through which NHS organisations are responsible and accountable improving quality of their services and safeguarding high standards of care.
What are the three key attributes of clinical governance?
- recognisably high standards of care
- Accountability and responsibility for those standards of care
- Constant improvement
List some components of clinical governance
Clinical audit Clinical effectiveness Education and training Risk management Openness R&D Clinical information and IT Patient and public involvement Staffing
Which organisations produce Clinical Governance guidelines that should be followed by members of the UKGTN?
BSGM
Genetic Alliance Group
Define Risk
The potential that an action or inaction will result in an adverse outcome.
How can risk be managed?
Identify what can go wrong
Understand what causes things to go wrong
Learning lessons from adverse effects
Ensuring action is taken to precent recurrence
Put measures in place to reduce risk.
What should be considered the most effective way to reduce risk?
Staff training
Give some examples of risks in the lab
Sample mix-up Hazardous chemical usage Manual handling Incorrect information on report Streamlining services to cut budget / staff shortages Introduction of a new a technology Needlestick injury
What training should be offered to minimise risk to staff?
Manual handling
COSHH assessments
Prevention of infection
Clinical risk/risks to service provision
Which regulations mandate that all incidents must be reported?
RIDDOR (1995)
List health and safety regulations applicable in an diagnostic lab
RIDDOR
COSHH
Health and safety at work act
H&S display screen equipment
The management of H&S at work regulations
REACH (restriction, evaluation, autorisation and restriction of chemicals) - MSDSs
Advisory committee on dangerous pathogens - control measures, transport, handling etc.
Which ISO standard is dedicated to H&S in medical laboratories?
15190
What should every lab have to manage H&S?
Health and Safety officer
How can H&S risks be reduced in the lab?
Substitution (of chemical or method)
Containment
PPE
What are the minimum topics that should be covered in H&S training?
Fire prevention and preparedness
Chemical and radiation safety
Biological hazards and infection prevention
When should risk assessments be performed?
When introducing a new reagents or working practices into the lab.
What should a risk assessment cover?
Identify hazards Decide who might be harmed and how evaluate the risks Record and implement findings Review assessment and update as necessary
Define “Screen”
The process of identifying individuals at high risk of a disease/condition
Define “Diagnostic test”
In most circumstances a process which will provide a clinically useful result - establishes the presence of absence of disease, and a positive test allows treatment.
Summarise changes to the the genetics service likely to take place over the coming few years.
Move from multiple regional labs to fewer - possible single sequencing centre, regional hubs and local hubs
Increased reliance on bioinformatics and data sharing
Integrated reporting systems, and inclusion of EPR
Molecular-based testing to replace more cyto services
Integration between genetics and other disciplines to allow for greater triage and sharing of equipment to reduce costs
Increased MDT involvement
Move to WGS
Postnatal karyotyping replaced by aCGH, and switching to SNP arrays to detect LOH/UPD
Prenatal arrays for abscan referrals, possibly moving to NGS after completion of PAGE
NIPT for aneuploidy
NIPD for single gene disorders/sexing
G-banding like to continue in oncology, but increased demand for molecular testing:
Targeted drugs, MRD, PGx.
List some factors likely to have an impact on the future of genetic testing in the UK
100,000GP Big Data Lab reconfiguration Direct to consumer testing Point of Care Testing Personalised medicine Pathogen genomics 7 day working
What is the purpose of the human genomics strategy group?
To develop a strategy for mainstreaming genetic testing in the NHS.
What are the aims of the Barnes Quality Review?
To review current QA frameworks and governance mechanisms at a national level in pathology and to make suggestions for improvement
What are used by the Barnes Quality Review to deliver quality and cost improvements?
QIPP milestones
Give some examples of QIPP milestones
Reduce duplicate or inappropriate testing
Agree future network config and organisation of services
Release savings (financial)
Provide commissioners with benchmarking
Deliver 24/7 working across 70% of pathology
LEAN working
What are the 4 key principles to personalised medicine?
Prediction and prevention of disease
More precise diagnoses
Targeted and bespoke medicine packages
Patients more involved in their own care.
What are the advantages of personalised medicine?
Give an example
Prevent adverse drug reactions
Lower dosage of some treatments - reduce costs and side-effects
Safer to get the right dose first time - e.g. CYP2C9 polymorphisms in Warfarin metabolism - CYP2C92 and CYP2C93 reduce warfarin required dosage by 1/3 and 1/5, respectively
What information should be provided on a genetics laboratory’s website?
Contact details for key members of staff
Lab address and general enquiries number
Services offered
Test repertoire - including testing methods and disease information
Sample referral form
Sample referral information (how to refer)
Any supplementary form required for specific tests
Opening times of the lab
Details of any out of ours services
Availability of clinical advice and interpretation
Sample requirements
TAT
Any specific sample requirements - factors affecting testing
Accreditation stamp
What is the purpose of having a central lab database?
To monitor and record all incoming/reported samples, tracking testing and results.
How can a central lab database be used?
Audit purposes Prevent duplication of testing Review previous test results Control user access and permissions Trigger downstream testing from results of upstream testing TAT stats for National Quality Dashboard Monitor samples sent to other labs Track charges Link to lab equipment to transfer test info Barcode generation
How are lab databases linked to data protection?
Each user has own login credentials
Database maintained, backed-up and supported by local IT
ISO15189 states that data should be safe, secure, backed up and retrievable
What other databases are found in the lab?
Internal disease databases
Primer databases
QC databases - SOPs, staff training records, consumables, equipment, audit log
What data is recommended to be stored on the results of exome/genome sequencing?
The output files, only. It is acceptable to have to resequence in the future.
Non-NGS data should be stored for 25 years (RCPath recommendations, 2005)
What are the 4 options to reduce a lab budget?
- Service integration
- staff costs
- non-staff costs
- private work/generate additional income
What is the most important consideration when reducing lab budgets?
That quality standards are maintained.
How can service integration be used to reduce costs?
Optimise use of machinery
Shared staff
Shared resources
What proportion of of the lab budget is made up of staff costs?
70-80%
How can staff costs be reduced?
Redundancy Service integration and collaboration Don't replace staff who are leaving De-skill Replace leaving staff with lower-pay grade Don't provide maternity cover Automation
How can non-staff costs be reduced?
Look for cheaper suppliers for consumables
Sign long term contracts or large scale orders
Purchasing consortia between multiple labs
Stop offering a specific test that has expensive consumables, or replace with cheaper
Reduce sendaways
Restrict spending on research, developent
Optimise and eliminate waste
Replace kits with in-house methods (may not be acceptable for much longer)
Paperless reporting
Cut down on electricity, rent, facilities costs.
How can labs generate extra income?
Increase prices - still need to be competitive
Develop new test
Collaboration with clinical and academic colleagues to apply for grants
Offer private services - analysis of cells lines for research, clinical interpretation for bioinformatics groups etc.
What other considerations should be taken into account when trying to reduce costs?
High cost area living allowances
NI contributions (increase staff costs by a further 20%)
Cost vs Price
Changes to a service have to be cost neutral if no business plan is written
Take on new staff to set up new test on a fixed-term basis
Rental of equipment may be more viable than purchasing - also allows upgrades.
What is the role of the BSGM?
Made up of CGS, ACGS and AGNC.
Oversees provision of genetics services at a national level and advises on important issues. Aims to advance the science of human genetics and implementation in the NHS
What is the role of the ESHG?
Similar to that of BSGM, but Europe-wide. Also in translating research into diagnostics.
Produces guidelines on testing
What is the role of the ACGS?
A forum for clinical scientists and other lab staff. Made of 4 committees: Quality Scientific Workforce development Communications
What other relevant professional bodies are there in Genetics?
CGS
NSHCS
ACS
ACB
Related to quality: HCPC RCPath CPA UKAS ISO CQC EMQN
Others: UKGTN HUGO GEL NCG UKNSC MHRA HTA HFEA
What legislation is present to protect consumers undergoing genetic testing?
EU regulations cover: Genetic data protection, Informed consent, Discrimination. The new General Data Protect Regulation will come into force in May 2018 and will dictate how genetic data can be handled, stored, shared etc.
The Council of Europe Committee have guidelines on the use of genetic data and reporting of incidental findings.
The Biomedicine convention does not apply to the UK, but Article 12 covers the improper use of genetic tests.
the Human Genetics Commission (UK) has published a report on DTC and pre-conception testing.
ESHG publishes many guidelines, however has recommendations to companies offering DTC testing about what they must/mustn’t do.
ISO - part of 15189 covers ethical conduct to ensure standards and reputation of organisation are maintained. Annex C covers informed consent, professional registration, confidentiality and data protection
What UK government legislation is there, relating to the ethics of genetic testing?
Legislation by the HTA (infomred consent) and HFEA
New HFEA guidance on mitochondrial donation were released in 2915
Data Protection Act
How does ethics guidance and legislation impact on genetic testing?
Reporting of VUSs and Incidental findings
Informed consent
Data sharing
Re-analysis of data
Re-contacting patients in the future if new results available
How much information should be given to patients to allow them autonomy, without causing too much worry.
What are essential for the IQC processes and ISO 15189 accrediation?
Validation and Verification
What are IQC?
They are internal standards used to assess the quality of a result before the result is reported to the patient. gives users confidence that the test is performing well.
What is required for a comprehensive validation/verification procedure?
Define the process to be validated Set the aims and objectives Set acceptance criteria Detail the number and type of samples to be tested Assess uncertainty measurement Record and review
What metrics should be defined during the validation/verification of a test?
Sensitivity Specificity Accuracy Trueness Reproducibility Repeatability
With respect to validation and verification, what does ISO15189 specify?
Validated procedures should be used to confirm the new methods are suitable for use
The validation shall be as extensive as necessary
Procedures should be periodically re-reviewed
The lab will design IQC systems to verify results
Define accuracy
How often a test gives the correct result
Define Trueness
How close the test result is to a known reference value
Define Precision
The degree to which two separate measurements differ - expressed as a confidence interval
Define repeatability
The level of agreement when a test is re-run using the same sample and conditions.
Define reproducibility
The level of agreement when a test is re-run using the same sample but different conditions (e.g. different operator)
Define robustness
How well a test maintains precision when parameters are changed.
Define “incidental finding”
A finding that has potential health or reproductive impact discovered in the course of a study, but not related to the reason for the study.
What should be considered when deciding whether or not to report an incidental finding?
Is it clinically relevant? Is he disease risk clearly proven? Prenatal/postnatal? Would reporting the finding lead to a better clinical outcome? Patient consent Patient autonomy Would witholding the information open up the department to litigation? Is it ethical to withold?
What sorts of Incidental Finding might be detected on karyotype?
45,X in a prenatal referred for Down’s screen
Balanced translocation in a bone marrow sample
Would an inv(9) incidental finding on a karyotype be suitable to report as an IF?
no
What sort of IF can be detected by aCGH?
Carrier status for recessive conditions - may be justified in reporting if clinically relevant gene is present in the interval and there is a high carrier rate.
Recessive disorders related to patient phenotype - report as further investigation is warranted
Mutations resulting in adult onset disorders (AZF deletions in a LD child)
CNVs with a risk of neoplasia (BRCA1, TP53 deletion)
What is special about reporting IF findings in prenatal arrays?
Follow-up parents to allow reproductive risk to be determined, even if not relevant to current pregnancy - this is a tricky counselling issue as disorder might be adult onset (BRCA deletion - mum is at risk)
Significance to the parents must also be taken into account
What incidental findings can be identified in molecular tests?
Klinefelter syndrome in FRAX test
Some MLPA control probes are disease associated genes.
What other incidental findings can be identified?
Non-paternity. ethically it is not clear if the father and/or child should be informed. Report as low risk of recurrence.
How do Berg et al (2011) propose IFs should be classified?
1 - findings that can be acted upon
2A - low risk, clinically-valid variants (PGx markers)
2B - Risk of intermediate disease with no treatment (APOE4)
2C - Severe disease with no treatment (HD, CJD)
3 - little evidence, no clinical utility
What IF do the ACMG recommend are reported
Any previously-reported mutation in the ACMG 59 gene list, regardless of age when testing is performed on constitutional tissue
Beware of nonsense mutations in GoF genes
According to the ACMG, whose responsibility is it to provide counselling and gain appropriate consent?
Clinicians. They should be familiar with the test and reporting strategy that they can offer accurate pre-test information
What IFs can be identified in NIPD?
ctDNA from the mother, mosaicism for sex chromosome abnormalities in the foetus, or mother.
Non-paternity if exclusion testing is performed.
What problems can occur in GWAS/other research studies, regarding IFs?
Genetic variants are of uncertain predictive value.
GWAS doesn’t establish causality
Research is not the same as diagnostics
Origin of samples and consent, use in multiple studies
Samples originating in other countries may not have the same follow-up procedures - what is clinically actionable in the UK may not be the same in China,
Who should the result be fed back to?
Attitudes to additional feedback may change over time
A management pathway needs to be drawn up
According to the ACMG, when should WES/WGS be considered?
When a mendelian disorder has a common phenotype and no associated genes
When standard testing has proven negative
When a disorders is heterogeneous
According to the ACMG, when should WES/WGS NOT be considered?
For first line prenatal testing
For first line newborn screening
What does the PHG foundation recommend is discussed during the consent process, above and beyond the ACMG recommendations?
The nature of the test
Generation, interpretation and disclosure of incidental findings and VUSs
Data sharing
Potentialfor reanalysis
What do both the ACMG and PHG both agree should be strongly encouraged?
Data sharing between labs.
What are the ESHG and PHG’s opinions on the ACMG’s recommendations of testing an additional subset of genes?
This should not happen, and labs should start with a more targeted approach, suitable to the age of the patient and referral reason.
Gene lists for each disorder should be standardised to provide equity of service.
What technical details should be included in an NGS report?
Coverage of target genes Process of data analysis and pipelines, including version numbers Scope of test diagnostic yield, where appropriate Sensitivity and specificity Result confirmation method What variants the test will detect
