Discussion
We believe that the observed improvements in SSC use following implementation of the PTB programme were due to several factors that commonly lead to successful implementation of any process-based practice. These include having a comprehensive stakeholder-driven programme, using a staged stepwise process providing clear communication and real-time feedback, engaging active leadership, and process simplification and modification.15 33 Our results showed improvements in checklist participation and item usage rates, following the implementation of the PTB programme. PTB was intended to simplify the checking process through addressing behavioural deficits and contextual factors identified during observations, described in earlier work.18 24 In other research, the most common reasons health professionals identified for non-compliance were forgetfulness in using the SSC or in addressing some of its elements,17 26 or a lack of time to complete it.15 17 33 We observed substantial improvements in the use of sign-out—improvements that were sustained 12 months after implementation of the programme. Prior to implementation of the programme, we did not observe sign-out during baseline observations, thus any improvement in performance from baseline would be significant. Vogts et al
34 suggest that sign-out use is often low because this section is not explicitly linked to a specific event in patient management, unlike sign-in and time-out—which may have catastrophic consequences if these checkpoints are not completed. Sign-out may reduce the likelihood of the most common ‘never event’, a retained foreign object during surgery.1
In our study, inadequate tissue specimen labelling was the most common clinical incident—the ramifications of which can potentially have devastating effects for patients. Labelling errors can result in inappropriate therapy or withholding treatment in patients with unrecognised malignancies.35 Observed improvements in use of the checklist did not result in concomitant reductions in clinical incidents. There are three possible explanations for this finding. The first relates to the possibility that incidents may have been under-reported. The second reason centres on the observed increase in the reporting of SAC 3 incidents following implementation of the PTB programme. This increase may be the result of simultaneous improvements in safety culture over time. The third explanation concerns the reach of the PTB programme, which was not as widespread as we would have liked. During the study period, staff changes and turnover occurred, and likely impacted programme spread across all surgical specialties and teams. The impact of team and staff factors and the stability of the workforce may also contribute to the dilution of capacity, that is, skills and attitudes.36
Despite a decrease in observed item compliance in the sign-in phase after implementation, our results attest to the overall sustainability of the evidence-informed PTB programme. Clearly, a variety of factors may create conditions that enable initial implementation, their presence or influence often diminishes over time.36 Unintentional ‘slippage’ can occur because of contextual factors such as a lack of resources, staffing conditions or competing demands.37 We attribute sustainability of the PTB programme to factors related to both outer (policies) and inner (ie, culture, structure), the programme itself (fit, adaptability), implementation processes (ie, fidelity monitoring, efforts to align the intervention and the setting) and the capacity to sustain (ie, interpersonal processes such as leadership, support). Beyond implementation of PTB, the clinical leads on the implementation team used fidelity maintenance strategies such as audit and feedback, reminders, and provided real-time informal education and training support. The PTB programme was co-constructed with stakeholders,21 and ‘normalised’ in everyday practice over time.38 The implementation team included credible leaders from nursing and surgery, which contributed to its sustainability and spread. Following evaluation of the PTB programme at this facility, the programme was implemented at two smaller satellite hospitals within the health services district. Spread of the PTB programme to these facilities was augmented by the leadership of senior nursing staff who work across these hospitals. While implementation fidelity is important, modification and evolution of any programme and intervention may need to occur in response to shifting priorities and contextual influences, or in light of new evidence.36 Nonetheless, the importance of having an ongoing education programme cannot be understated given the continuing challenges brought about by staffing changes and attrition.
Limitations
Few studies have used prospective approaches to evaluate the implementation of practice change interventions.39 This is one of the first longitudinal studies to prospectively and retrospectively evaluate a patient safety programme designed to improve surgical teams’ participation in, and use of, the SSC. Nonetheless, we acknowledge some limitations. First, the use of a single hospital site and sampling methods of prospective data collection may limit generalisability and increase selection bias. Second, we assessed surgical teams’ use of the SSC with direct observations, with results showing moderate consistency between raters, likely due to some variation in interpretation of events. To minimise this, we developed a data dictionary, and the two observers were experienced OR nurses trained in observational research methods. Third, the observational nature of the study may have given rise to the Hawthorne effect. Yet it is likely that this diminished over time as participants became accustomed to the presence of the observers. Fourth, at the time of the implementation, the PTB programme was not mandated, therefore, participation was essentially voluntary, potentially leading to self-selection. However, since its introduction, the PTB programme has been included in the health services policy, and has been subsequently introduced at two similar hospital sites across the health services district. Fifth, clinical incident data are secondary, self-reported and represent a non-random sample of errors from an unknown universe of errors. As such, these data cannot generate incidence rates or capture the entire universe (ie, denominator) of errors. Finally, there may have been some misclassification in the coding of harm and other variables. Despite these limitations, our study has considerable strengths in terms of its longitudinal nature and use of data triangulation. These results offer unique insights into SSC use and the types of clinical incidents identified in surgery. We anticipate these results will drive further improvements in checklist implementation and use beyond the facility where the PTB programme was developed and implemented.