Article Text
PDF
Abstract
Background In 2009, the WHO introduced the surgical safety checklist (SSC) as one of the interventions for improving patient safety. The systematic use of structured checklists during surgery has been shown to reduce perioperative morbidity and mortality. However, SSC utilisation has been challenging in low-income and middle-income countries, including Ethiopia. Jhpiego Ethiopia implemented a quality improvement project (QIP) aimed to increase SSC utilisation.
Methodology A model for improvement was used to design and implement a collaborative QIP to improve SSC utilisation at 23 public health facilities (13 primary health care facilities, 4 general hospitals and 6 tertiary hospitals) in Ethiopia from October 2020 to September 2021. SSC utilisation was defined as when a patient chart had SSC attached and each part of the checklist was completed. Training of surgical staff on safe surgery packages, monthly clinical mentorship and cluster-based learning platforms were implemented during the study period. We analysed bimonthly chart audit reports from each facility to assess the proportion of surgeries where the SSC was used. Shewhart charts were used to conduct a time-series analysis. Additionally, the Z-test for two sample proportions was used to determine if there is a statistically significant change from the baseline measure with a p<0.05.
Result In the postintervention period, the overall SSC utilisation improved by 39.9 absolute percentage points to 90.3% (p<0.0001) compared with the baseline value of 50.4% early in 2020. A time-series analysis using Shewhart charts showed a shift in the mean performance and signals of special cause variation. The largest improvement was observed in primary health care facilities in which the SSC utilisation improved from 50.8% to 97.9% (p<0.0001).
Conclusion This study demonstrates that onsite clinical capacity building, mentorship and collaborative cluster-based learning platforms can improve SSC utilisation across all levels of facilities performing surgery.
- Checklists
- Patient safety
- Surgery
- Continuous quality improvement
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Utilisation of the WHO surgical safety checklist during surgery has been shown to improve surgical patient safety. But, proper utilisation has been challenging especially in low-income and middle-income countries.
WHAT THIS STUDY ADDS
The use of quality improvement methods to analyse performance gaps and design interventions was effective in bringing significant practice change in surgical safety checklist utilisation among the surgical team.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This evidence may inform health facilities in Ethiopia and other low-income and middle-income countries of effective interventions to improve surgical patient safety.
Background
Globally, over 313 million patients undergo surgical procedures annually.1 Surgery-related complications are common patient safety issues. Studies reported adverse events can occur in up to 14% of patients who have undergone surgery while mortality ranges between 1% and 4%.2 3 Similarly in Ethiopia, adverse events occur in 12% of surgical patients.4 However, the incidence of surgical complications has not shown a significant change over the past two decades.5 6 Several interventions have been introduced to improve surgical safety, including checklists and new policies to govern the operating room.7
In 2009, the WHO introduced the surgical safety checklist (SSC) as one of the interventions in improving surgical patient safety. The systematic use of structured checklists in preprocedural and postprocedural has demonstrated the potential to be effective at reducing surgical complications and mortality rates.6 8–11 The SSC includes three pause points prompting discussion among team members and establishing shared mental model where the team identifies possible risks for errors in the sign-in, time-out and sign-out domains.12 In a prospective interventional study conducted in eight hospitals participating as pilot sites in the Safe Surgery Saves Lives programme, proper utilisation of the SSC reduced mortality by 50% (mortality of 1.5%–0.8%), (p=0.03) and reduce complications from 11% to 7% (p<0.001).13 Additionally, the use of the SSC increased appropriate antibiotic use from 56% to 83%, which could potentially result in a drop in associated infections by greater than 33%.13
Although SSC utilisation has been associated with increased detection of potential safety hazards, studies show that SSC utilisation rate is variable across healthcare institutions.6 Systematic reviews indicate there is a significant difference in the utilisation of SSC among health facilities ranging from 29.8% to 88.8%.14 A multicentre study conducted in England reported that surgical teams failed to pause or focus on the checks in more than 70% of cases.15 Similarly, in a study conducted Gondar university hospital in Ethiopia shows 39.7% SSC utilisation.16 Reasons for the low SSC utilisation include a lack of positive role models or less than enthusiastic team members, hierarchical barriers, limited knowledge of correct usage and inappropriate implementation procedures. Lack of knowledge, lack of resources and a hierarchical culture are particularly common barriers to checklist implementation in low-income and middle-income countries.9 10 17
To address major surgical safety issues, Jhpiego and Ministry of Health of Ethiopia designed Strengthening Systems for Improved Surgical Outcomes (SSISO) to be implemented in 23 public health facilities. Using global safe surgery experience, a safe surgery package was prepared to be implemented in the intervention facilities through blended training and ongoing coaching using a mentor-mentee (hub-and-spoke) cluster model.18
Aim statement
The aim of this quality improvement project (QIP) is to increase SSC Utilisation from 50.4% to 95% at 23 public health facilities selected as part of the SSISO project from 1 October 2020 to 30 September 2021.
Methods
Study area and period
This QIP was implemented in 23 SSISO project intervention health facilities from October 2020 to September 2021. Health facilities were in three regions (Amhara, Oromia and Addis Ababa). Among the facilities, there were six tertiary, four secondary and thirteen primary healthcare units (list of health facilities in online supplemental annex 1).
Supplemental material
Design
A longitudinal/time-series study design was used to implement collaborative QIP to address the gaps in surgical care safety and particularly address gaps in SSC utilisation. The facilities developed a QIP with the same aim, which tested project-based change ideas and contextualised facility-based change ideas to improve SSC utilisation.
Population
The source population is all 23 health facilities SSISO project intervention facilities.
Inclusion criteria
Health facilities enrolled in and supported by the SSISO project.
Major surgeries in surgical service outlets.
Exclusion criteria
All service outlets and medical records of clients who undergo minor surgeries will be excluded from the survey.
Theory of change and change ideas tested
Taking W. Edwards Deming’s system of profound knowledge as the theoretical framework, a facility-based root cause analysis was carried out using fishbone diagram (online supplemental annex 2) to understand the root causes and opportunities for improvement .19 A multidisciplinary QI team was established to ensure staff participation and ease follow-up. Plan-do-study-act cycles were used to introduce interventions and build knowledge impact of changes that were tested.
Supplemental material
The following package of interventions was implemented:
Intervention 1: onsite clinical capacity building on safe surgery packages
Jhpiego leveraged global experience and instruction design know-how to develop and adapt safe surgery training package. Surgical staffs received regular onsite clinical capacity building training using this training package.
Intervention 2: monthly clinical mentorship
A hub-and-spoke mentoring model was implemented to build the capacity on the safe surgery package for health facilities grouped in a mentor-mentee cluster of facilities. This model allowed greater engagement of mentor tertiary-level care teaching facilities and leveraging of surgical expertise in the teaching facilities to coach surgical team in mentee facilities, mainly a primary care hospital and health centres. Following leadership, QI and safe surgery package training for selected expert mentors, these mentors then provided monthly peer and mentee site mentorship using a structured checklist in each facility. A multidisciplinary team of mentors was used to cascade the mentorship which includes surgeons, anaesthesiologist, nurse and QI experts.
Intervention 3: supportive supervision and regular feedback on performance
In collaboration with experts from the regional health bureau and MoH, the SSISO project team provided regular supportive supervision for health facilities and provided regular feedback on the progress made at the health facility. Additionally, facilities were trained on data utilisation to improve surgical data utilisation for decision-making.
Intervention 4: create an experience sharing and learning platform
The SSISO project team conducted review meetings with facilitates with the aim to create a learning platform for facilities to learn from each other best practices. Facilities were given opportunities to discuss successes and challenges as peers across the intervention facilities.
Data source
The readiness and baseline assessment (RABA) study was conducted prior to the intervention. During the study period, bimonthly retrospective audits of surgical patient charts during the project period were used to measure SSC utilisation. A total of 30-time points were collected.
Data collection
One QI focal person who was a health professional practising in the facility was selected from each intervention facility as a data collector. Respective health facility leadership was engaged in the data collector selection process. A 3-day safe surgery training and an orientation on the structured chart audit tool was provided for data collectors. At each facility, data collectors randomly selected 19 surgical patient charts using the lot quality assurance sampling method for those who had undergone a surgical procedure in the reporting period.20 21 A retrospective chart audit was used to see SSC utilisation. The charts were audited using the structured audit tool every 2 weeks and uploaded online using Microsoft forms. In health facilities with fewer than 19 surgeries in the prior 2 weeks, all surgeries in the respective period were audited.
Measures
QI measures (table 1) were used to follow the QIP. The facility name, date of surgery, type of surgery (emergency/elective), type of procedure and SSC completeness in each phase of the checklist (sign-in, time-out and sign-out) and the outcome of the surgery was collected from chart audit. Additionally, the number of surgical staffs trained on safe surgery and the number of mentorship visits were collected from the monthly mentorship reports.
Quality improvement family of measures tracked during the project implementation period
Data analysis
Data collected from the chart audit using a standard checklist were entered in online Microsoft form. Then the aggregate data were exported MS Excel 2019. The SSC utilisation was expressed as yes/no for each step of the SSC and the overall utilisation using all or none principle. The proportion of SSC utilisation was calculated from the sum of all charts audited and reported every 2 weeks.
The Z-test for two sample proportions was used to compare the change in performance from the baseline assessment. Statistical process control charts(Shewhart charts) was used to analyses variation in the system over time to assess whether changes resulted in improvements Box 1.22 In the time-series analysis using Shewhart control charts, the study was divided into the initial months (baseline), the intervention phase and implementation phase. Separate calculation of mean performance and control limits for the three periods conducted.
Operational definition
SSC utilisation
For the purpose of this study defined as the patient charts where SSC is attached and each part of the checklist is completed. One hundred percent of the checklist steps need to be completed to define as a complete checklist.
Shewhart charts
The Shewhart chart (or control chart) is an extension of a line graph with a median used to distinguish between variations in a measure of quality due to common causes and variation due to special causes. These charts are most used in time series interventional studies. These in this context were used to indicate a common cause and special cause variations.
Common cause variations are those inherent to the system. Those variations cannot be avoided unless the system as a whole is changed.
Special cause variations are those variations that are not inherent to the system. It indicates there is a variation that is not normal to the existing system. These variations need to be explained by inserting annotations into the charts.
Change ideas are actionable specific ideas that, if introduced, may lead to an improvement. Literature reviews, logical thinking about current systems, creative thinking, benchmarking and technology are common sources of new ideas in QI.
Results
Prior to implementation, a RABA was done in SSISO intervention facilities(n=23), where 539 major surgical patient charts were reviewed, and it was found that SSC was used in 272 (50.4%) of the charts. During the project period, a total of 5268 randomly selected patient charts archived in 23 heath facilities were audited. Among randomly audited charts, 4054 (77.12%) were emergency surgeries (table 2). Looking at the types of procedures, caesarean section accounted for 3667 (69.7%) of surgical operations and the rest 1601 (30.4%) include other major surgical procedures (gastrointestinal, paediatric surgery, neurosurgery, urology, etc) (table 2).
Facility and operative characteristics of the data used for time-series analysis
Process measures
Change ideas prioritised to increase the SSC utilisation were tracked during the study period. During the study period, a total of 188 mentorship visits were conducted, representing 97% completion of all planned monthly mentorship visits by the multidisciplinary team (figure 1).
Coverage mentorship visits using hub-and-spoke model in eight cluster facilities from October 2020 to September 2021.
The initial offsite safe surgery package training was provided to 142 project mentors and surgical team leaders from each facility. Additional surgical system leadership and basic QIP training was provided to equip mentors with the necessary skills during the mentorship visit.
Regular onsite clinical capacity building training programme was conducted for the surgical staff working in the intervention’s health facilities. The content of the training was based on the gaps observed during mentorship visits. During the intervention period, the training was provided for more than 1500 staffs working in the intervention health facilities.
Outcome measure
We compared a baseline 3-month (March–April 2020) performance prior to the intervention to the last 3-month performance (July 2021–September 2021) to see if there was a statically significant improvement in SSC utilisation after the implementation of the interventions. In the postintervention period, the overall SSC utilisation during surgeries improved by 39.9 absolute percentage points to 90.3% compared with the baseline value of 50.4% early in 2020 (p<0.0001). When analysed by level of care, there was a 31.0%, 37.4% and 47.1% increase in SSC utilisation in tertiary, secondary and primary healthcare facilities, respectively (table 3).
Comparison of proportion of charts with completed SSC from preintervention 3 months (2020) and postintervention 3 months (2021) among the 23 health facilities
In time-series analysis, the shewhart chart (p-chart) shows a shift in the mean performance towards the target (figure 2). A shift which is special cause variations was observed from June 2021 to September 2021 (figure 2), showing a change in system performance. After the initial training for project mentors, monthly mentorship and capacity buildings, a positive performance shift was observed. Overall, the SSC utilisation target (95% utilisation) was achieved within 8 months following the initiation of the project. The improvement was tracked for 3 months after the interventions ended in September 2021 and improvements were sustained (figure 2).
Proportion of major surgeries with completed SSC in 23 intervention health facilities, from 1 October 2020 to 30 September 2021, P chart. SSC, surgical safety checklist; UCL, upper control limit; LCL, lower control limit; SSI, surgical site infection; AA, Addis Ababa.
It took approximately 7 months to achieve the target (95%) for mentee (spokes) facilities, compare to the mentor facilities that took 11 months to reach the maximum performance (~90%) which was below the target set for the project 95% (figure 3). The improvement in SSC utilisation was greater in the mentee facilities than in the mentor facilities may be attributed to a smaller number of OR rooms, number of surgical workforce and the number of specialties for the difference in the performance.
Percentage of surgical procedures with SSC used in 23 health facilities from October 2020 to September 2021, by mentor and mentee facilities. SSC, surgical safety checklist.
Discussion
There were observed improvements in SSC utilisation following implementation of the tested change interventions. These interventions include onsite clinical capacity building on safe surgery packages, monthly clinical mentorship, supportive supervision and regular feedback on performance, create an experience sharing and learning platform. The results showed improvements in checklist utilisation through behaviour change identified during facility based RCA described in earlier work.23 24
The overall SSC utilisation improved from 50.4% at the baseline to 90.3% in the postintervention period. Compared with other studies with similar interventions, our study had similar improvements, even though the magnitude improvement differs. In study conducted in Tanzania, SSC utilisation has improved from 0% to 98% using a team-based approach to introduce SSC.23 A similar finding was observed in a prospective longitudinal study conducted university hospital in Queensland, Australia which shows utilisation rate ranging from 79.3% to 94.5%.24 This increase in SSC utilisation is expected to result in a significant improvement in the reduction of adverse events following surgical procedures.2 13
Additionally, better performance was observed in mentee facilities (spokes) compared with mentor facilities. This could be attributed to the fact that in mentee facilities, there are fewer staff, which created the opportunity to provide clinical capacity building to all members. This likely resulted in better communication and flexibility, which have been associated with the performance difference in other studies.18 Conversely, having large number of surgical staffs, rotation and turnovers during the intervention period had negatively impacted the performance through dilution of knowledge and capacity.18 These issues were more common in mentor facilities, which likely contributed to the observed performance difference. The project tried to mitigate this challenge through regular onsite capacity building trainings.
Limitations
Although we trained QI focal and mentors on data collection procedures, there is potential risk to unconsciously bias chart auditing results as supporting ones’ expectations like reporting an unfilled checkbox as ‘yes’. To address this, we tried cross check reports during supportive supervisions, avoided blaming for bad performance and used the data strictly for improvement. Since we introduced the training and mentorship simultaneously, we were not able to determine which intervention had the greatest impact on the results. The study cannot rule out additional cofounders that can positively or negatively affect our results, such as advocacy works on safe surgery by governmental bodies. Since the bimonthly data collection extends for more than 15 months, there is potential for data collectors to bias sampling selection for convenience. The monthly mentorship visits and feedback nature of the study may have given rise to the Hawthorne effect, whereby providers delivered care differently than they would if they were not being observed. But, we expect this decrease over time as participants became accustomed to observation and feedback. Additionally, this study did not include a link improved SSC utilisation with patient-level outcome.
Conclusion
By using QI methods to understand the performance gap, design interventions and successfully implementing these interventions (clinical mentorship, onsite clinical capacity building and create learning platforms), we were able to increase SSC utilisation by 39.9%, from 50.4% to 90.3%, in study facilities. Safe surgery intervention packages have been included in national surgical strategic plan and perioperative guideline, which provides an excellent opportunity for sustainability and scale up of successful practices from this project.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Johns Hopkins School of Public Health IRB (IRB # 00011312) and the Ethiopian public health Association IRB (EPHA/06/147/20). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
Our sincere gratitude goes to all the mentors, QI focals and each health facility leadership who contributed to the implementation and follow-up of this collaborative QIP.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors All authors had a role in writing the manuscript. DY is guarantor author.
Funding Jhpiego Ethiopia through funding from UBS Optimus Foundation (grant No. 48297), provided financial support.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.