Article Text
PDF
Abstract
Background and problem The WHO Surgical Safety Checklist has been shown to improve patient safety as well as improving teamwork and communication in theatres. In 2009, it was made a mandatory requirement for all NHS hospitals in England and Wales. The WHO checklist is intended to be adapted to suit local settings and was modified for use in Gloucestershire Hospitals NHS Foundation Trust. In 2018, it was decided to review the use of the adapted WHO checklist and determine whether improvements in compliance and engagement could be achieved.
Aim The aim was to achieve 90% compliance and engagement with the WHO Surgical Safety Checklist by April 2019.
Methods In April 2018, a prospective observational audit and online survey took place. The results showed compliance for the ‘Sign In’ section of the checklist was 55% and for the ‘Time Out’ section was 91%. Engagement by the entire theatre team was measured at 58%. It was proposed to move from a paper checklist to a wall-mounted checklist, to review and refine the items in the checklist and to change the timing of ‘Time Out’ to ensure it was done immediately prior to knife-to-skin.
Results Following its introduction in September 2018, the new wall-mounted checklist was reaudited. Compliance improved to 91% for ‘Sign In’ and to 94% for ‘Time Out’. Engagement by the entire theatre team was achieved 100% of the time. Feedback was collected, adjustments made and the new checklist was rolled out in stages across all theatres. A reaudit in December 2018 showed compliance improved further, to 99% with ‘Sign In’ and to 100% with ‘Time Out’. Engagement was maintained at 100%.
Conclusions The aim of the project was met and exceeded. Since April 2019, the new checklist is being used across all theatres in the Trust.
- checklists
- anaesthesia
- surgery
- patient safety
- healthcare quality improvement
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Background and problem
The WHO Surgical Safety Checklist was introduced in the NHS in 2009 and was made a mandatory requirement for all hospitals in England and Wales.1–3 The checklist consists of three sections—a check prior to anaesthesia (Sign In), a check before the start of surgical intervention (Time Out) and a check before any member of the team leaves the theatre (Sign Out). The checklist has been shown to improve patient safety, reducing mortality from 1.5% to 0.8% and surgical complications from 11% to 7% as well as improving teamwork and communication in theatres.4 However, the ability to bring about these improvements appears to be related to the style of implementation used and the engagement of clinical teams, rather than just the introduction of the checklist alone, as despite its widespread use, ‘never events’ do still occur. In the NHS between April 2017 and March 2018, there were 209 wrong-site surgeries.5
The WHO checklist can be implemented in different ways, and local modifications to layout and wording are encouraged to make the checklist more relevant to different settings.1 In Gloucestershire Hospitals NHS Foundation Trust, the checklist used was a modification of the original WHO Surgical Safety Checklist. Despite the checklist being carried out before every operation, compliance and engagement with the checklist were unclear. Therefore, in 2018, a Checklist Review Group, consisting of a consultant anaesthetist, two anaesthetic registrars and an anaesthetic core trainee, was set up to review the checklist.
Aim
The aim was to achieve 90% compliance and engagement with the WHO Surgical Safety Checklist by April 2019.
Methods
Context
Gloucestershire Hospitals NHS Foundation Trust provides acute hospital services from two large district general hospitals, Gloucestershire Royal Hospital and Cheltenham General Hospital. The Trust has 960 beds and provides secondary healthcare for 600 000 people across Gloucestershire. There are 14 operating theatres at Gloucestershire Royal Hospital and 12 operating theatres at Cheltenham General Hospital. There are approximately 32 000 surgeries each year at Gloucestershire Royal Hospital and Cheltenham General Hospital. Theatre teams consist of anaesthetists and surgeons and their trainees, scrub nurses, anaesthetic assistants and healthcare assistants.
Baseline measurement
To assess compliance and engagement with the existing paper checklist, data were collected by a prospective observational audit. Data were collected at Gloucestershire Royal Hospital between 30 April 2018 and 11 May 2018. During the study period, there were approximately 600 surgeries carried out. All the operations included were during the weekdays and from a range of surgical specialties (General Surgery, Gynaecology, Trauma and Orthopaedics, Urology, Ear, Nose and Throat (ENT) and Maxillo-facial). An observer (member of the Checklist Review Group) was present in theatre while the checklist was carried out. Data were collected for the ‘Sign In’ and ‘Time Out’ sections of the checklist (figure 1—observational audit form).
Observational audit form.
Compliance was measured by recording whether each item of the checklist was discussed. The item had to be stated and verbally responded to by an appropriate member of the theatre team for compliance to be achieved. Items that were discussed outside of when the checklist was done, were not considered to have occurred as part of the checklist, and recorded as not completed.
Engagement was measured for the ‘Time Out’ section of the checklist. Engagement was measured by recording what each team member was doing while ‘Time Out’ was taking place. Engagement was only achieved if all other tasks were stopped, and individuals were fully focused and listening to the checklist. Engagement was calculated by dividing the total number of staff engaged by the total number of staff present. The ‘Time Out’ section of the checklist was chosen to measure engagement, as this was when the entire theatre team was present.
In addition, the timing of certain items on the ‘Time Out’ section of the checklist was noted. It was recorded whether antibiotics, venous thromboembolism (VTE) prophylaxis and patient warming were started prior to knife-to-skin. This was chosen as an indication of the effectiveness of the checklist—to show whether completing the checklist resulted in the intended action.
An online survey, consisting of eight questions, was sent to all theatre staff to collect feedback on the existing checklist process. Respondents were asked to select to what extent they agreed/disagreed with the eight questions on a 5-point Likert scale from ‘strongly agree’ to ‘strongly disagree’. Respondents were anonymised, but their role was identified.
Results of baseline measurement
97 ‘Sign Ins’ and 97 ‘Time Outs’ were captured during the study period.
Compliance
Compliance for the ‘Sign In’ section was 55%. The compliance for the individual checklist items varied. Compliance with checking the wrist band, consent form and procedure was between 98% and 100%. Compliance with discussing allergies was 85%, and 89% for checking the operative site had been marked. However, compliance with discussing blood availability, airway assessment and anaesthetic equipment was between 10% and 14%, and for checking the operating list was just 4%.
Compliance for the ‘Time Out’ section was 91%. Compliance for most items was between 91% and 100%. The only exception to this was again checking the operating list, where compliance was just 2%.
Airway assessment and anaesthetic equipment in ‘Sign In’ were often discussed prior to the arrival of the patient, and this might reflect the lower compliance rates seen. Checking the procedure against the operating list was not part of the original WHO checklist but had been added as a local modification. The procedure is also confirmed with the patient and with their consent form. This additional duplication may reflect why compliance was low.
During ‘Time Out’, not all of the interventions were done before knife-to-skin. Compliance with discussing antibiotics was 99%, however only 62% of the time were they administered before knife-to-skin. Compliance with discussing patient warming was 100%, but only 48% of the time was warming (blankets or Bair Hugger) put on before knife-to-skin. Compliance with discussing VTE prophylaxis was 100%, and 96% of the time calf pumps or antiembolism stockings were on before knife-to-skin. This demonstrates that if the checklist is performed too early (soon after entering the operating theatre), it creates a time-window for the team to forget to do the tasks specified, thereby making the checklist less effective—acting more like a to-do list than a checklist.
Engagement
Engagement with ‘Time Out’ was only achieved 58% of the time. Engagement varied for the different staff groups: 43% for anaesthetic assistants, 47% for anaesthetists, 59% for surgeons, 65% for healthcare assistants and 75% for scrub nurses. Reasons for non-engagement included scrubbing up, adjusting monitoring, administering medication, writing in the notes, preparing the sterile equipment and talking. Having ‘Time Out’ soon after transferring into theatre was thought to contribute to engagement not being achieved, as the team was occupied with other patient-related tasks during that time.
Online survey
One hundred and ten people responded to the online survey. Respondents were from a range of staff groups and included anaesthetic and surgical consultants, anaesthetic and surgical trainees, anaesthetic assistants, nurses and healthcare assistants. The survey showed that although 100% of people agreed that the checklist was done before every operation, only 56% of people agreed that the checklist was effective, only 23% of people agreed that the entire team gave their full attention and only 48% agreed that they could hear every question. Fifty-two per cent of people agreed that if antibiotics were required, they were given before knife-to-skin, and only 36% agreed that if patient warming was required, it was switched on before knife-to-skin. In addition, 52% of people agreed that they were often distracted while the checklist was taking place, and 52% agreed that the questions were often answered by the person who was reading the checklist (figure 2—online survey results).
Online survey results.
Given these results, a strategy was proposed to improve practice.
Strategy and improvement cycles
Plan do study act (PDSA) cycle 1
It was proposed to:
Move from the paper checklist to a wall-mounted poster size checklist.
Review and refine the items included in the checklist.
Change the timing of ‘Time Out’ to immediately prior to knife-to-skin.
Moving to a large wall-mounted checklist would enable the entire team to see and follow the checklist. Rewording the items in the checklist allows the team to focus on the most important aspects. Ensuring ‘Time Out’ is immediately prior to knife-to-skin and not earlier, enables the entire theatre team to give their full attention to the checklist without being distracted by other patient-related tasks. The emphasis was on moving from a to-do list to a true checklist, with all tasks completed prior to going through the checklist (figure 3—driver diagram of change ideas).
Driver diagram of change ideas.
The new wall-mounted checklist was designed by the Checklist Review Group. For all sections, the wording of the items on the checklist was made more concise. Moving from statements such as ‘Is difficulties with airway anticipated or a rapid sequence induction needed?’ to ‘Airway plan’ and from ‘Does the patient have any allergies’ to ‘Allergies’. Checking the procedure against the operating list was removed from both ‘Sign In’ and ‘Time Out’ to avoid unnecessary duplication. A ‘Stop before you block’ section was incorporated into the ‘Sign In’ section, as a wrong-sided block is a ‘never event’. Allergies were incorporated into the ‘Time Out’ section, as a second check.
In September 2018, the new wall-mounted checklist was trialled in an afternoon simulation session to gather feedback, primarily on safety, before use with real patients. The new checklist was then trialled in one of the General Surgical theatres in Gloucestershire Royal Hospital.
One week after its introduction, the checklist was reaudited and feedback was collected. Data was collected over a 5-day period, as per the initial audit method. Compliance and engagement were remeasured. 18 ‘Signs Ins’ and 18 ‘Time Outs’ were captured. Feedback was collected from staff by forms available in theatre.
The checklist was amended based on the feedback from the theatre teams. This included increasing the font size and changing the wording of some of the checklist items. ‘VTE prophylaxis’ and ‘Diathermy’ were changed to ‘VTE prophylaxis on’ and ‘Diathermy on’, emphasising tasks should be completed before the checklist is performed.
PDSA cycle 2
In December 2018, the updated wall-mounted checklist was trialled in all the General Surgical theatres. It was then trialled in Orthopaedics, Gynaecology, Urology and ENT theatres. Feedback was collected from each surgical specialty by forms available in theatre and at departmental governance meetings.
The checklist was reaudited. Data were collected from orthopaedic theatres (four theatres) over a 5-day period, as per the initial audit method. 23 ‘Sign Ins’ and 23 ‘Time Outs’ were captured.
Based on feedback, slight adjustments were made to the checklist. The items were made more concise and the order was changed. In ‘Time Out’, anaesthetic and surgical concerns were initially separate points—these were replaced by ‘Anyone not happy to start?’. ‘Site’ was changed to ‘Site and side’, and ‘glycaemic control’ was removed. The checklist was presented at each surgical specialty governance meeting and at the Surgical Division Board before agreement on final wording and implementation.
PDSA cycle 3
To provide training and overcome any barriers and resistance to change, there was a planned effort to involve all staff groups throughout the rollout process. Initiatives included:
Audits and reaudits were presented at anaesthetic and surgical governance meetings to explain the new process, gain feedback and answer questions.
Posters were produced outlining the key changes to the checklist and displayed in the theatre department.
Emails were sent to anaesthetic and surgical teams providing regular updates on the rollout.
Simulation sessions using the checklist were arranged at departmental governance meetings so staff became accustomed to the new process.
Training video of the new checklist in use was produced and made available on the intranet.
In April 2019, the final wall-mounted checklist was rolled out to all theatres across the Trust. Documentation of completion of the WHO checklist is now included in the theatre register (figure 4—wall-mounted checklist).
Wall-mounted checklist.
Results
The wall-mounted checklist was well received and led to a dramatic increase in compliance and engagement. The reaudit in September 2018 (PDSA cycle 1) showed compliance improved from 55% to 91% for ‘Sign In’ and from 91% to 94% for ‘Time Out’. Engagement by the entire theatre team was achieved 100% of the time. There was an improvement in interventions occurring before knife-to-skin. 67% of antibiotics were administered before knife-to-skin (increase from 62%), 75% of the time patient warming was started before knife-to-skin (increase from 48%) and 100% of the time VTE prophylaxis was on before knife-to-skin (increase from 96%).
The reaudit in December 2018 (PDSA cycle 2) showed compliance improved further to 99% for ‘Sign In’ and 100% for ‘Time Out’. Full engagement was maintained at 100%.
There was a further improvement in interventions occurring before knife-to-skin. All 100% of antibiotics and 100% of patient warming were started before knife-to-skin, and 96% of the time VTE prophylaxis was on before knife-to-skin.
The wall-mounted checklist is now being used in all theatres in Gloucestershire Royal Hospital and Cheltenham General Hospital.
Discussion
The goal of this project was to improve our WHO Surgical Safety Checklist. The original paper checklist was complied with but had variable impact in terms of efficacy and engagement. The replacement wall-mounted checklist resulted in a dramatic improvement in engagement and an improved team dynamic within theatres. The improved checklist is also more effective with fewer episodes of delay in administering antibiotics or warming patients.
Lessons and limitations
From the beginning, it was recognised that any changes to the checklist needed to be carefully introduced, and stakeholders identified early to establish their buy-in, overcome any barriers to its implementation and lead to sustainable change. Such barriers included change from the established process and the uncertainty of a new system. To help achieve this, the checklist was introduced in a staged stepwise approach, regular feedback collected and clear communication given to all those involved as the project progressed. This kept the profile of the work high throughout the roll-out period.
Several limitations are acknowledged. First, the reaudits were done shortly after the new checklist process was introduced. Theatre teams were aware of the ongoing audit cycles, and as such, compliance and engagement may have been higher during this time (Hawthorne effect).
Second, the improvement strategies were introduced simultaneously—moving to a wall-mounted checklist, changing the timing of ‘Time Out’ to just before knife-to-skin and refining the points included. It is therefore hard to evaluate which change had the greatest impact.
Third, the reaudits had smaller sample sizes, n=18 and n=23. It is worth noting though the major changes to the checklist were in PDSA cycle 1, where 18 ‘Sign In’ and 18 ‘Time Out’ were analysed. Only minor refinements were made in PDSA cycle 2, where the sample size was 23 ‘Sign Ins’ and 23 ‘Time Outs’. So the major changes had gone through two PDSA cycles with a combined sample size of 41.
Fourth, an online survey of staff perception of the paper checklist was conducted at baseline, and although feedback from staff was collected throughout, it is acknowledged that the online survey could have been repeated after the Trust-wide roll out to reassess staff perception of the new process.
Finally, in the baseline audit, although data were collected from all surgical specialties, the proportion from each surgical specialty was not recorded, and so any baseline variability in compliance and engagement was not identified. Although this would have been of interest, the rollout of a single checklist was planned for all theatres across all surgical specialties.
Conclusions
Although the paper checklist was completed before every operation, there was often a lack of engagement by the entire team during this process. Wall-mounted checklists are present in other hospitals, and there was a drive to introduce such a change here, so the benefits of the checklist could be fully achieved. The implementation of the new checklist and process led to an improvement in both compliance and engagement. Compliance improved from 55% to 99% with ‘Sign In’ and from 91% to 100% with ‘Time Out’. Engagement improved from 58% to 100%. The project aim was achieved. Following the roll out across the Trust, the next step will be to reaudit the checklist, specifically for engagement and for signs of checklist fatigue and re-assess staff perception through repeating the online survey to ensure that the improvement in standards is being maintained.
Acknowledgments
The authors would like to thank the Quality Improvement Academy at Gloucestershire Hospitals NHS Foundation Trust, and Andrew Seaton, Director of Safety, for their help and guidance.
Footnotes
Contributors TK: project lead, supervised data collection, project progression and final approval of the version to be submitted. HM, LK: data collection, interpretation and analysis. CC: data collection, interpretation and analysis, drafting of the initial paper and final approval of the version to be published.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Ethics approval This work did not require ethics review. The work intended to improve local care. The authors sought to evaluate the improvement in compliance and engagement with the WHO Surgical Safety Checklist at Gloucestershire Hospitals NHS Foundation Trust.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.