Problem

For over a century surgical care has been a vital aspect of medicine1 and is often the primary method of management in many conditions. Around the globe millions of people require surgical intervention every year which, although always intended to save lives, may cause substantial harm where there is unsafe practice.2 With this known history and the significant need for good practice it would thus seem clear that high quality of global surgical development is a must. However, despite this, and the growing evidence of the crucial part surgery can play in achieving health goals, surgery has not adequately been received as a global health priority.3 Indeed, as a whole, non-communicable diseases (NCDs) have been poorly recognised and surgical systems in many countries have consequently been left woefully inadequate for the growing population need.4

Much needed recognition of this problem came in 1980 when the then WHO Director-General, Halfden Mahler, challenged the world to rectify the global inequalities in access to safe surgical and anaesthesia care.5 Despite this early call, the need for development of surgical and anaesthetic services is only recently being recognised as the global burden of disease shifts towards the realm of non-communicable. Emphasis has been made here by the Lancet Commission on Global Surgery (in 2015)6 and the WHO Second Global Patient Safety Challenge, “Safe Surgery Saves Lives” (launched in 2006)2, detailing the need to improve surgical and anaesthetic care worldwide. A Roadmap has been set out to achieve, by 2030, the recommendations made by the Lancet Commission. To achieve this goal, and reduce the surgical inequality, action will need to be undertaken in a wide range of areas.7

One such action is the introduction of the World Health Organization (WHO) Surgical Checklist to Operating Rooms (OR). This checklist was developed in 2008 from WHO published guidelines that aimed to improve surgical safety worldwide. The checklist was designed to be a simple tool that could be used globally to reduce major surgical complications.8

This study took place at Felege Hiwot Specialised Referral Hospital (FHSRH), a major government referral hospital in the city of Bahir Dar in the Amhara region of Ethiopia. Ethiopia is a country of 82.8 million people that is undergoing significant development and expansion of its healthcare services. However, despite this, there remains only 0.2 physicians per 10,000 people.9 At FHSRH, there are at present, on average, approximately 150 major elective surgical procedures and 200 major emergency surgeries performed per month. With such a high case load the need for the successful introduction of tools that improve surgical safety, such as the WHO checklist, is imperative.

This study thus aimed to improve the use of the WHO Safe Surgery Checklist from it's basic use at introduction to complete and comprehensive use over a one year time period via continued review and training of Checklist practice.

This aim is to be applied to the four ORs of the General Surgery department and 2 ORs of the Obstetrics & Gynaecology department of FHSRH.

This project was conducted as part of the partnership between FHSRH and The University of Aberdeen/Soapbox Collaborative. Members of the Project team consisted of students and doctors (surgical and anaesthetic) from both the UK and Ethiopia.

Background

Global Surgical & Anaesthesia Care

The global community continues to move forward and develop new targets as demonstrated by the new Sustainable Development Goals (SDGs), which aim to improve the well-being of the global population through broad reaching interventions. Considering that an estimated 234 million operations are being performed every year throughout the world10, surgical development is crucial to achieving these SDG targets. Surgical work is vital to the healthcare of every country around the world and for every form of community. This need is further illustrated by a report from the World Bank stating that, in 2002, approximately 164 million disability-adjusted life-years were attributable to surgically treatable conditions.11 Surgical conditions alone claim responsibility for approximately one-third of the world's burden of disease.12 Moreover, surgical intervention is required to some degree in all subcategories of global disease burden.13 This is also significantly true for obstetric surgery. It has been reported that the improvement of obstetric surgical care in low-middle income countries (LMICs) could result in great reductions in the burden of conditions for both maternal and neonatal care.14

The cost-effectiveness of surgical intervention should not be a barrier to the promotion of surgery globally. Indeed, it has been written that surgical development in universal health care can, in respect to economic benefit:-cost ratio, have a 10:1 bias.15 Thus it must be universally accepted that the SDGs will not be fully accomplished without targeted funding to surgical and anaesthesia care development.

However, the benefits that can be gained from surgical care must be countered with the risks of complications. It is the recognition and reduction of these risks that, if successfully managed, will lead the improvement in global surgical care. It has been reported that adverse events in hospitals most commonly occur in operating theatres with 43% of these events being preventable if current standards of care are rigorously applied.16 Indeed, in industrialized countries, studies have demonstrated a perioperative rate of death from inpatient surgery to be 0.4 to 0.8% with major complications at 3 to 17%.17 ,18 Unfortunately, the quantification of this risk in many parts of the world is not adequately realised.8 This is a point in itself that calls for further research in global surgery. However, there is a clear understanding that such rates are likely to be much higher in developing countries.19 ,20 For example, if known surgical complication rates are extrapolated to the global population, it is a plausible to estimate 7 million complications and 1 million deaths every year from surgery alone.21 To give this further perspective, this estimation is twice the number of recorded maternal deaths.21 This produces a picture of a service that is much needed globally to significantly reduce a rising burden of disease, which is subject to high rates of complications that are not sufficiently recognised. Thus it is quite clear that surgical care is in much need of attention from the global health community to further advance its development.

This is a situation, however, that is entirely amenable to improvement which creates a great potential for significant development. Reports have noted that approximately 50% of all surgical complications are entirely avoidable.22 Specific previous efforts have demonstrated success in complication reduction via programs that decrease the rate of surgical-site infection and promote safe anaesthesia.23 ,24 To achieve such safety improvements requires an understanding of where errors are likely to occur and have the ability to recognise and pre-empt such errors.25 In low resource settings it is important to design quality improvement around evidence-based interventions that are applicable to the environment.26 One such way to achieve this is via the introduction of “tried and tested” checklists that are transferable to different healthcare environments. Such a strategy emphasises the concept of “getting the basics right, first time, every time” as has been detailed in other reports.27 One such checklist that follows the above strategy and has been developed for the purpose of improving surgical safety is the WHO Surgical Safety Checklist.

WHO Surgical Checklist

The WHO surgical safety checklist, published in 2008, was developed from guidelines that made a number of recommendations for practices that ensure safety of surgical patients worldwide as part of the Safe Surgery Saves Lives initiative.28 Following this development a large pilot study was completed in 2008 which applied the checklist to diverse and economically disparate surgical populations (four high income countries and four LMICs). This study found that implementation of the checklist was associated with reductions in both surgical complications (11% to 7%) and morbidity (1.5% to 0.8%).8

Following this initial pilot other studies from a variety of locations have also demonstrated benefit from the application of the checklist. One study in Chisinau, Moldova demonstrated successful checklist introduction with a significant reduction of overall complications from 21.5% to 8.8%.29 A further study in Uganda illustrated that improving surgical practice with use of the checklist can be achieved via stepwise quality improvement projects.30 Finally, a large and recent study, of 357 hospitals in 58 countries, has documented 38% lower odds of death with use of the WHO Checklist.31

The need for surgical and anaesthetic development is thus clear and the benefit of the WHO Surgical Safety Checklist to achieve this has also been well established. The Checklist is endorsed surgically (2009 WHO Safe Surgery Guidelines).28 anaesthetically (2010 International Standards for a Safe Practice of Anaesthesia)32 and by the 2010 Ethiopian Hospital Reform Implementation Guidelines (EHRIG).33 The introduction of the Checklist to FHSRH is thus both significant and appropriate. This study will report on the introduction of the Checklist to FHSRH and, more widely, report on the effectiveness of introducing the Checklist to a low resource setting.

Baseline measurement

This project was conducted, from July 2015 to August 2016, within the stated wider partnership between Felege Hiwot Hospital and The University of Aberdeen. This partnership enabled the development of good relations between UK and Ethiopian staff for collaboration in the review of the Checklist development and implementation. Firstly, the published WHO Surgical Safety Checklist and guidance on its use was reviewed by senior surgical staff of FHSRH. The checklist was then modified for appropriate and locally specific use within the OR of FHSRH. During modification it was ensured that the ten objectives identified as essential for safe surgery remained prominent points of the checklist.26 These ten objectives are: Safe anaesthesia, adequate airway check, risk assessment of blood loss, precautions against surgical site infection, accurate check of correct site of surgery, strict checks for any allergies, accurate and strict swab/instrument counts, identification of specimens, good communication between the whole OR team, follow up of surgical outcomes.

Following this site specific modification, the Checklist was introduced to all OR's and staff were instructed on its use by local surgical leaders. This instruction was given to surgical, anaesthetic and nursing teams working in the OR. Base measurement of the Checklist was then conducted prospectively via observational study within the OR and review of patient notes and completed checklists. Baseline Phase I study was conducted for the General Surgical OR (GS-OR) and the Obstetrics & Gynaecology OR (OBG-OR). This initial assessment collected data regarding overall Checklist completion, completion of each point on the Checklist and average completion of categorised points. Three categories were formulated from individual Checklist points and observed practices (only for General Surgery) for surgery, anaesthetics and infection prevention. Average completion of all points for each category was then calculated. The groupings were as follows:

Surgical: Patient Confirmation, Patient Consent, Site Marked, Allergies Checked, Blood loss risk, Introductions, Surgical Pauses, Swab Counts before and after

Anaesthetic: Pre-op assessment, Anaesthetist pause, Airway check, Oximeter Check

Infection Prevention: Antibiotic prophylaxis, Use of PPE, Equipment Sterility, Skin Preparation

Surgical cases from morning surgical sessions were reviewed for Checklist completion. The reviewer attended each case to analyse the use of the Checklist and it's completion. Mornings were selected randomly with at least 2 consecutive cases reviewed from each session. At least 5 different surgical sessions for each phase were attended to ensure different combinations of theatre teams were surveyed.

As the study aimed to solely investigate actual use of the Checklist, phase I review begun once the Checklist was introduced and measured the development of it's use over time from this starting point.

The initial Phase I review for GS-OR collected data from 30 patients between 13th July to 24th July 2015. Of these 30 operations 67% were elective and 33% emergency. The Checklist was used 50% of the time (27% partial completion and 23% full completion). The most poorly completed components of the Checklist were “Site Marked” and “Allergy” which both had 0% completion. Other poorly completed areas included components of surgical pause, including anaesthetic concerns, (7 – 17%) and “equipment count” (13%). The average completion for category groupings resulted as follows: surgical, 33%, anaesthetic, 69%, infection prevention, 89%.

The baseline Phase I review for OBG-OR collected data from 14 patients between 3rd August to 10th August 2015. Of these 14 operations 36% were elective and 64% were emergency. The Checklist was used 50% of the time and all these cases were partial completion with no cases observing full completion. The most poorly completed items were components of surgical pause, including anaesthetic concerns, “Allergy” and “Recovery Concerns” which all recorded 0% completion. “Equipment count” was also poor with only 7% completion.

Design

The greatest strength of this study was that the tool being introduced, the Checklist, was already recognised by the senior surgeons at FHSRH as a valuable instrument that can improve the safety of surgical services. The importance in design was thus establishing the most effective way to work with the various staff in the OR to develop the consistent use of the Checklist.

Previous work regarding introduction of the Checklist has highlighted a number of successful approaches.29 ,30 ,34 ,35 These approaches emphasise the importance of consulting with local leaders and establishing local ownership of the Checklist. This ownership can be established with carefully designed multidisciplinary training and promotion of the effectiveness of good Checklist use. The implementation of the Checklist itself can then be performed by locally trained staff with regular on site supportive supervision and encouragement. This method requires a good working relationship with the local OR team and development of progressive incremental changes until the Checklist becomes standard practice. It has also been noted that development of improved teamwork between all staff in the OR is vital to any successful introduction of the Checklist. Regular interviews and discussions with staff are also important to review any difficulties and identify where improvements can be made.

All these previous successful intervention designs were thus pooled together for the introduction at FHSRH. As noted, the Checklist was first modified and training developed by local senior surgical staff. Following its introduction, and base measurement of its use, there were a number of discussions with surgical, anaesthetic and nursing staff. Problems that were relevant generally and specifically to the different departments were then addressed. Plans were made for gradual standardisation of use over a year and further review following this period. Further review would then be coupled with further multidisciplinary training, discussions, support of use and analysis of problems/difficulties.

Expected problems during intervention including equipment malfunctions, under-staffing, team work difficulties and acceptance of new protocols by all staff. These problems aimed to be addressed during training and discussions. However, primary introduction of the Checklist was via FHSRH staff which promoted local ownership and sustainability of the intervention.

Strategy

Initial baseline measurement (Phase I) of the Checklist and discussions with OR staff identified a number of areas where improvement could be made. Lessons learned that needed to be addressed to improve Checklist implementation included:

• Further training of circulating nurses to ensure Checklist is managed and used correctly

• Promotion of greater communication between circulating nurse, scrub nurse, anaesthetist and surgeon

• Ensure equipment and swab counts are made and develop method to ensure this is readily and easily done

• Stress importance of surgical pause

• Improved anaesthesia checks

• Promotion of multidisciplinary approach to surgery

The first improvement cycle aimed to address the above lessons learnt through local leadership, training of OR staff and presentations/discussions at morning meetings. Focus was placed on improving communication and interaction (including promotion of surgical pause) between team members when using the Checklist. Copies of the modified Checklist were also made readily available in all OR's and its completion was made a primary concern for circulating nurses. A table for swab counts was also added to the back of the Checklist paper copy to act as a prompt and recording method. A time lapse of 11 months was allowed to enable changes to develop and to enable Checklist use to become standard practice.

From this first cycle of improvement it was predicted that the Checklist would become standard practice in the OR and thus Phase II would measure an improvement in use of the Checklist. However, it was anticipated that certain points in the Checklist may be adopted more slowly and thus some points may remain poorly completed. This prediction was accurate with improvement across almost all individual points noted. However, as predicted, some points improved less than others. This was particularly true for GS-OR where “full completion” of Checklist fell by 3% compared to a 40% increase in “full completion” for OBG-OR.

The weaker points from Phase II measurement were then noted for a focused second round of improvement for GS-OR. Focused group discussions with surgical, anaesthetic and nursing staff were conducted to again identify general challenges and areas for improvement. These discussions were also used to address specific problems identified during Phase II measurement for focused and directed improvement of weak points.

Following this second round of improvement a period of encouragement and assistance in Checklist use was conducted within OR's for a month before Phase III measurement. This consisted of senior surgical leaders (when not operating) intermittently attending the OR to review use of the Checklist by nurses and junior surgeons and providing brief support where needed. It was predicted that the Checklist use would improve again but with further improvements in the weaker points that were specifically addressed to make the whole Checklist standard practice. This was again achieved and the results presented back to the department. Any outstanding weaker points were again identified for continued improvement.

Results

Phase II and III measurement followed the same methods as baseline measurement with prospective observational study and patient note analysis. These two phases each followed an improvement cycle as documented above. Phase II was conducted for GS-OR and OBG-OR and Phase III for GS-OR only.

Phase II GS-OR was conducted between 10th June and 14th July 2016. 30 cases were collected with 50% elective and 50% emergency. Phase III GS-OR was then conducted between 8th August and 22nd August 2016. 15 cases were collected with 33% elective and 50% emergency. Phase II OBG-OR was conducted between 8th August and 22nd August 2016. 10 cases were collected with 30% elective and 70% emergency.

Use of the Checklist in GS-OR increased between Phase I and 2 from 50% to 97%. This use remained high at 94% in Phase III. The main increase between Phase I and Phase II was in partial completion (27% to 77%) whereas full completion was relatively static (23% to 20%). The third phase however resulted in an increase in full completion (20% to 60%). At the end of the study the least fully completed section of the Checklist was “Sign In” (53%), then “Sign Out” (73%) and “Time Out” was best (87%). At study end the single most poorly checked item was “Site Marked” (60%) but this was markedly increase from Phase I (0%). All three developed categories demonstrated increased average completion improvement as follows: surgical (33%, 76%, 89%), anaesthetic (69%, 89%, 87%), infection prevention (89%, 94%, 100%).

Use of the checklist in OBG-OR increased between Phase I and Phase II from 50% to 100%. This improvement included a slight increase in partial completion (50% to 60%) and a great increase in full completion (0% to 40%). The least fully completed section was “Time Out” (50%), then “Sign Out” (80%) and “Sign In” was the best (90%). At study end the most poorly checked item was “Recovery Concerns” (70%) which also showed improvement from Phase I (0%).

From focused interviews with surgical, anaesthetic and nursing teams it was found that there was much enthusiasm for use of the checklist and appreciation of benefit gained in using it. The greatest challenges in completing the checklist were revealed to be communication difficulties between teams and high turnover of staff in the OR (where trained users and leaders were often moved) (See supplementary file).

Lessons and limitations

A collection of key lessons can be extracted from this project that will be useful in designing future introductions of the Checklist to similar settings. Firstly, there were a number of positive points that contributed to the successful introduction. Local ownership and drive to making the Checklist standard practice is the first and foremost point to any successful and sustainable introduction. The Checklist introduction in this project was driven by local surgical leaders and the developed partnership was then used to assist in this introduction and analyse its use with identification of areas in need of improvement. Early analysis of the Checklist was important to identify immediate problems and difficulties which could be directly managed in partnership with local staff. Tackling such early misunderstandings and misconceptions was vital before allowing a significant amount of time to pass for its use to become standard practice. This early improvement needed both general focus (such as development of greater interdisciplinary communication and interaction) and more specific focus (such as explaining the concept and benefit of the surgical pause). Direct alterations could also be made to target poorly performing areas (such as adding an equipment/swab count table to the reverse of the Checklist). These early improvements and standardisation of use over a year then enabled more precise improvements over a focused period of two months.

However, these methods also developed a number of challenges and difficulties. A major difficulty was the high turnover of staff within the OR, particularly among the nursing staff and younger Resident surgeons in training. This meant that, although a number of surgical leaders were enthusiastic for its introduction, the nurses and surgeons newly arriving in the OR may not fully understand the importance of the Checklist nor have adequate training in its use. It was indeed noted in group discussions that frequent rotation had an effect on good team dynamics and communication which in turn has an effect on Checklist use. This was a limitation of the focused intervention for the second wave of improvement in that over two months the teams were relatively stable. For although this study did show improvement over the one year period, the challenge will come in maintaining the greater improvements from the focused period over a longer time-frame when staff again move and teams change.

To take on this above challenge future study must consider increasing the number of data points collected. For example, establishment of monthly review and data collection over the course of twelve months will enable greater awareness of compliance throughout the year. This expansion in data points will also reduce the possibility of chance affecting the results. With only three data points across the year there exists the possibility that random fluctuations produced the results gathered. By collecting more data, such as twelve data points over a year, the study will be more reliable in its demonstration of any improvement.

Furthermore, this study incorporated a number of interventions that had been reported as successful elsewhere. This was done to maximise the potential for improvements to be made. Although this has shown success it would be valuable to know which single interventions had the greatest benefits. This study does not allow single points of intervention to be broken down to judge their relative impact. Future study could use greater data collection, as described above, to systematically test emphasis on individual interventions. Interventions may then be compared for relative benefit and selected for maximum efficiency and prospect of improvement.

The project described here is sustainable in being locally introduced and gradually standardised. However, to sustain the improvements made here will require close attention to training all new staff rotating into the OR's, promoting the importance of its use to all incoming surgical trainees and ensuring communication between all disciplines remains open. Greater work in this project could have been done towards identifying leaders from all disciplines who could be permanent members of the OR and be key teachers and promoters of the Checklist. Such an added intervention would contribute greatly to the long term sustainability of the improvements achieved here. However it is important to note that these conclusions can only be accurately applied to the specific locality in this study. Thus, although other localities may take lessons from this study that may be adapted elsewhere, the specific methods and results here cannot be generalised to other clinical environments.

Conclusion

To reach the surgical development targets set for 2030 there needs to be a significant speeding up of progress being made.7 Despite this recognition global health research activity in surgery sits at only 4.1%36 resulting in the continuation of many knowledge gaps. One such gap that has been reported is the feasibility of introducing the WHO Surgical Safety Checklist in other hospitals. The report demonstrating the success of the Checklist by Haynes et al8 commented on the potential of the Checklist to make a significant improvement to surgical care globally with low cost implementations. However, the authors commented that further study is required to assess the feasibility of this implementation in other hospitals. Unfortunately there remains limited reports from low-resource settings regarding such implementation. One study commented that any successful implementation at different institutions will have variations and are likely to be very much context-dependent.37 This highlights the need for further studies at different institutions and locations to determine the successful strategies for Checklist implementation.

This study thus added to this limited but growing body of work studying the WHO Surgical Safety Checklist implementation. This study demonstrated and effective implementation via local leadership, training, direct observation, changes in standards and regular measurement with feedback and discussion. An increase in Checklist use was observed over a one year period in both General Surgical and Obstetrics & Gynaecology OR's. This included apparent improvements in this specific location over long term use and further apparent substantial improvements over a short term focused improvement period. Further analysis will be required to ensure these improvements are certain and lasting.

This project has been successful in illustrating that such improvements can be made with the described methods in different OR environments. The developed category groupings for General Surgery also enabled focus upon where greatest improvement needed to be made. Attention to both surgical and anaesthetic teams enabled improvements in both these categories. Furthermore, the “infection prevention” category was increased to 100% compliance following the focused intervention. This included an increase in antibiotic prescribing from 63% to 100%. Previously reports have demonstrated that an increase in this single point alone has been noted to reduce post-surgical infection by 55%.38 ,39 Thus using the Checklist to focus on infection prevention can potentially make significant contributions to the developing interests in surgical-site infection reduction.

Further work will be required here though in assessing whether these improvements will be sustainable over a longer time period following the end of the short-term focused intervention. There is possible bias in that the improvements noted were a result of enhanced performance of subjects in knowledge of being observed. The danger here being that following study end and loss of follow up the performance improvements begin to decline. It has been noted that such an effect in low-resource settings is more usual following short-term volunteer visits with myopic scope for further review and support.35 The strengths of this study lie in the long-term partnership that exists that will enable further follow up and support to promote the sustainability of these recorded improvements. It is the establishment of such a long-term relationship to enact sustainable development that this study most highly promotes.

Thus ultimately, although this project is only specific to this particular institution and location, it does add to the growing evidence of successful Checklist implementation in low-resource settings. Such implementation requires site specific Checklist modification, local ownership, group education, long-term assistance, monitoring, audit and feedback. These developments require long-term input and good working relationships. The development of more such relationships, and quality improvement work in OR's across the globe, will surely contribute to addressing the need for improved global surgical services and progressing towards the targets set for 2030.