You are here

  • Home
  • Archive
  • Volume 8, Issue 1
  • Validation of a Norwegian version of SURgical PAtient Safety System (SURPASS) in combination with the World Health Organizations’ Surgical Safety Checklist (WHO SSC)

Article Text

Article menu

Download PDFPDF

Original article
Validation of a Norwegian version of SURgical PAtient Safety System (SURPASS) in combination with the World Health Organizations’ Surgical Safety Checklist (WHO SSC)
  1. Anette Storesund1,2,
  2. Arvid Steinar Haugen1,
  3. Hilde Valen Wæhle3,4,
  4. Rupavathana Mahesparan5,
  5. Marja A Boermeester6,
  6. Monica Wammen Nortvedt7,8,
  7. Eirik Søfteland1,2
  1. 1 Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway
  2. 2 Department of Clinical Medicine, Faculty of Medicine, University of Bergen, Bergen, Norway
  3. 3 Department of Research and Development, Haukeland University Hospital, Bergen, Norway
  4. 4 Department of Clinical Science, Faculty of Medicine, University of Bergen, Bergen, Norway
  5. 5 Department of Neurosurgery, Haukeland University Hospital, Bergen, Norway
  6. 6 Department of Surgery, Academic Medical Center Amsterdam, Amsterdam, The Netherlands
  7. 7 Centre for Evidence-Based Practice, Western Norway University of Applied Sciences, Bergen, Norway
  8. 8 Accident and Emergency Department, City of Bergen, Bergen, Norway
  1. Correspondence to MCCN (Hons) Anette Storesund; anette.storesund{at}helse-bergen.no

Abstract

Introduction Surgical safety checklists may contribute to reduction of complications and mortality. The WHO’s Surgical Safety Checklist (WHO SSC) could prevent incidents in operating theatres, but errors also occur before and after surgery. The SURgical PAtient Safety System (SURPASS) is designed to intercept errors with use of checklists throughout the surgical pathway.

Objective We aimed to validate a Norwegian version of the SURPASS’ preoperative and postoperative checklists for use in combination with the already established Sign In, Time Out and Sign Out parts of the WHO SSC.

Methods and materials The validation of the SURPASS checklists content followed WHOs recommended guidelines. The process consisted of six steps: forward translation; testing the content; focus groups; expert panels; back translation; and approval of the final version. Qualitative content analysis was used to identify codes and categories for adaption of the SURPASS checklist items throughout Norwegian surgical care. Content validity index (CVI) was used by expert panels to score the relevance of each checklist item. The study was carried out in a neurosurgical ward in a large tertiary teaching hospital in Norway.

Results Testing the preoperative and postoperative SURPASS checklists was performed in 29 neurosurgical procedures. This involved all professional groups in the entire surgical patient care pathway. Eight clinical focus groups revealed two main categories: ‘Adapt the wording to fit clinical practice’ and ‘The checklist items challenge existing workflow’. Interprofessional scoring of the content validity of the checklists reached >80% for all the SURPASS checklists.

Conclusions The first version of the SURPASS checklists combined with the WHO SSC was validated for use in Norwegian surgical care with face validity confirmed and CVI >0.80%.

Trial registration number NCT01872195.

  • patient safety
  • checklists
  • surgery
  • quality improvement
  • transitions in care

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2018-000488

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Surgical complications are a global concern. A review of closed healthcare claim cases including complications showed that it would be possible to prevent 50% of the cases.1 A common problem which is known to complications is poor communication.2 Tools such as safety checklists have been introduced to enhance teamwork, communication and reduce patient safety risks.3 Use of checklists has been shown to reduce surgical complications and mortality.4–6 WHO’s Surgical Safety Checklist (WHO SSC) was introduced in the operating theatres (OTs) in two Norwegian hospitals in 2009–2010.6 However, the in-hospital surgical pathway is comprehensive and consists of multidisciplinary involvement and interactions in OTs and in the admission phase, preoperative phase, postanaesthesia care unit (PACU) and postoperative ward care.7 Transfers through different departments with loss of information throughout the clinical pathway may be a threat to patient safety.8 Complications are known to occur also in the preoperative and postoperative phases of surgery.9 Many risk factors have been described, such as failing to identify allergies,10 lack of antibiotic prescriptions11 and follow-up on venous thromboembolism risk and prophylaxis.12 To our knowledge, there is only one validated checklist concept that systematically cover the total surgical pathway with personal checklists for the involved key personnel used through all critical transfer points in the care process: the Dutch SURgical PAtient Safety System (SURPASS) checklists.13

    The SURPASS consists of 11 checklists covering the total surgical flow, from admission to discharge. Introduction of the SURPASS checklists in six Dutch hospitals reduced complications from 27.3 (95% CI 25.9 to 28.7) to 16.7 (95% CI 15.6 to 17.9). The mortality was reduced from 1.5% (95% CI 1.2 to 2.0) to 0.8% (95% CI 0.6 to 1.1).5 The WHO SSC has been implemented in all hospitals in Norway as part of the Norwegian patient safety programme ‘In Safe Hands’.14 Due to mandatory use of the WHO SSC, it was not possible to introduce all parts of the more comprehensive SURPASS system. Nevertheless, it seemed to be feasible to introduce the preoperative and postoperative SURPASS checklists in combination with the WHO SSC in clinical practice. Thus, this needed further investigation. We aimed to translate the SURPASS’ five preoperative and three postoperative checklists and validate the SURPASS version in combination with the already established Sign In, Time Out and Sign Out parts of the WHO SSC for use in Norwegian surgical care.

    Methods and materials

    Translation and validation of the SURPASS checklists content into Norwegian flow of surgical care followed the WHO guidelines,15 recommended for translation and adaption of instruments. The process consisted of six steps: (1) forward translation; (2) testing the content; (3) focus groups; (4) expert panel; (5) back translation and (6) approval of the final version.

    The study was carried out in a neurosurgical unit in a large tertiary teaching hospital in Norway, referral for 1.1 million inhabitants, performing all common neurosurgical procedures both in children and adults.

    WHO Surgical Safety ChecklistSSC

    The established WHO SSC consists of three checklists to be performed within the OT at three definite moments in surgery: before induction of anaesthesia, before incision and at the end of surgery.16 The checklist was in 2009 translated to Norwegian17 by clinical experts including surgeons, anaesthesiologists, nurse anaesthetists, OT nurses and quality improvement officers. It was back translated to English by native English-speaking personnel and became the official Norwegian version.18 The WHO SSC was implemented in five surgical departments, including neurosurgery. Effects of using the checklists have been validated through previous published work.6 17 19 Further implementation of the WHO SSC at the remaining surgical departments followed WHO’s implementation guide with adaptation to local use.20 21

    The SURPASS checklists

    The SURPASS checklists consist of five preoperative, three intraoperative and three postoperative checklists. The preoperative and postoperative checklists are individualised to fit the healthcare providers’ professional responsibility. The original version of the SURPASS checklists13 was developed in three steps: (1) literature studies on human processes and adverse events after surgical procedures, (2) observations of safety risk events in clinical practice throughout the perioperative care and (3) practical and effectiveness evaluation of the checklists. The content was validated by observing safety deviations in clinical practice in comparison with checklist items.13 This process was to ensure that practice and theory corresponded. The original preoperative and postoperative phases of SURPASS consisted of 63 checklist items. In addition, two items on the preoperative checklist for surgeons were to be used in case of local anaesthesia without anaesthesiologist.

    In contrary to the WHO SSC, which are performed by the surgical team, the preoperative and postoperative SURPASS checklists are personalised and completed by individual health professionals in charge of specific care details through the surgical care pathway. We chose to add specifically the preoperative and postoperative parts of the SURPASS checklists to the already established intraoperative WHO SSC in our hospital and combine them in one comprehensive perioperative checklist.

    Forward translation

    An English translation of the content was provided from the SURPASS copyright holders5 in addition to the original Dutch version. Translation of the checklist content into Norwegian was first carried out by professional translators (Semantix AS, Stavanger, Norway). Then, the translated and the English versions of the checklists were reviewed by three clinical experienced researchers (AS, ASH and ES). Cross-cultural adaptation of surgical workflow and logistics in checkpoints from Dutch to Norwegian standards were ensured in close collaboration with surgeons and healthcare personnel from the neurosurgical department testing the checklists. This also investigated the face validity and feasibility. Three items were left out from the original Dutch preoperative ward nurse checklist due to lack of local existing protocols and procedures at the time of investigations: screenings for decubitus; risk of patient falls; and delirium. All three screening protocols were under development and scheduled to be introduced at a later stage. One item for the discharging nurse concerning home regimen explained to patient was left out due to being covered in standard discharging procedures. Two new procedures were implemented that contribute to two new checklist items on the preoperative ward nurse checklist: body temperature controlled 1 hour before entrance to the OT (not in the original version) and patient identification tags on both wrists (in the original version: name tags and barcode on both wrists). One of the original checklists assigned to an anaesthesiologist or intensivist when transferring the patient from PACU or intensive care unit to hospital wards was changed and assigned to the PACU nurse.

    Testing the content

    Before testing the checklists, all groups of healthcare professionals received at least one educational session. The personnel involved in neurosurgery were ward doctors (neurosurgical resident/consultant in neurosurgery/final year student resident), ward nurses (registered nurses (RNs)), neurosurgeons, anaesthesiologists, OT nurses (RNs with graduate certificate in operating room processes), PACU nurses (RNs or graduate certificate in intensive care) and discharging doctors (neurosurgical resident/consultant in neurosurgery/final year student resident) and nurses (ward nurse and RNs). All personnel involved received information by email and informative posters that were displayed in the department. Training followed the principles of Conley and colleagues,22 by explaining why the checklists were tested and showing how to use the different checklists. The implementation team consisted of key clinical personnel, the research group and the middle level of management for the involved groups. Paper version checklists were used individually by personnel at each preparatory step of the surgical pathway. All the checklists had user instructions attached. The lists were designed to check whether all necessary procedures had been completed, hence different from a to-do list.23 During the test period, it was mandatory to use the preoperative parts of the checklists. In agreement with the department head, consequences of not completing the checklists resulted in delayed surgery.

    During the test period, the checklist users were asked to write feedback notes on a daily basis regarding wording of the checklist items. This was to determine whether the wording was precise and to get an understanding of optimal time-points for completion of the checklists.

    The implementation team was available to clarify doubts and follow-ups throughout the test period. All the surgeons were asked individually on their experiences of using the preoperative and discharging checklists.

    Focus groups

    After testing the checklists in clinical settings, we needed more systematic information regarding the checklist users’ perspective on usage and existing workflow in relation to checklist compliance.24 Eight focus groups were carried out by two moderators. We planned to perform interviews in small focus groups (two to five participants) with a strategic sample of healthcare professionals. Respondents being potential users of the SURPASS checklists, including surgeons, anesthesiologists, ward doctors, ward nurses, OT nurses and PACU nurses with mixed length of experiences, were selected. The interviews were scheduled to last up to 60 min. Trained interviewers and moderators (AS, HVW, ASH and ES) conducted the focus group interviews. The interviews were carried out in hospital settings close to the wards and OTs to minimise use of time away from clinical work. The checklist items formed the interview guide. Data from the interviews were noted as condensed meaning units on a paper form. The participants reported their clinical experience, sex and profession. We used qualitative content analysis to identify codes and categories to assess the items adaption to the existing work flow.25

    Expert panels

    Each item on the checklists were subsequently tested by expert panels for all the eight new SURPASS checklists using the content validity index (CVI).26 To score the CVI, we used eight panels with experts. The experts were instructed to score the content from a general surgical angle—covering all the surgical areas, not merely neurosurgery. The CVI scoring was performed to test relevance and comprehensiveness of precise and clear wording of the checkpoints.27 The experts rated each checkpoint item on a four-point scale: 1=not relevant, 2=somewhat relevant, 3=quite relevant and 4=highly relevant.28 Item content validity scores (I-CVI) were used to guide revision of wording or questions of deleting items or text. To reveal the total content validity score of the checklist or scale (S-CVI), the proportion of experts who have scored 3 or 4 were calculated.26

    Back-translation and final approval of the SURPASS checklists

    Following a forward translation, testing of the content in clinical practice, focus groups and validation by expert panels, the checklists were back-translated from Norwegian to Dutch by a native Dutch speaker. The back-translated checklists, including both the SURPASS parts and the WHO SSC were presented to the Dutch SURPASS copyright holder for approval.

    Results

    Forward translation

    The content of the original SURPASS checklists has previously been published.5 After forward translation of the checklist content, managers and the different clinical professionals ensured that the different checklist contents were assigned to the responsible healthcare professional following Norwegian standards and legislation. The item ‘obtaining written consent’ is not required by Norwegian legislation; thus, this checklist item was left out. Adjustments and cross-cultural adaptations to local workflows needed to be performed: for example, ward doctors in the Netherlands are to check on: relevant imaging present; in Norway, the surgeons assess the images and the OT nurses check for the presence of the images in the OT. Also for Dutch ward doctors: relevant laboratory checks, including cross-typing; in Norway, ward nurses check for cross-typing, while the surgeons and anaesthesiologists control the laboratory results. All healthcare professional groups engaged in neurosurgery each confirmed face validity and feasibility of their respective checklist items before the checklists were tested in clinical practice.

    Testing the content

    We tested the checklists in 29 neurosurgical procedures performed over 3 weeks in June and July 2012. In each surgical procedure, 11 checklists were used, which includes: the five new preoperative SURPASS checklists, the established three parts of WHO SSC and the three new postoperative SURPASS checklists. All the healthcare professional groups engaged in neurosurgery were represented. Compliance rates to the different checklists are presented in figure 1. The SURPASS checklists used here included 64 checklist items, in addition to two items on the preoperative checklist for surgeons to be used in case of local anaesthesia without an anaesthesiologist involved. The test revealed that some items had to be moved to other professional groups due to differences in national and local work assignments and work flow, and some items needed to be reformulated for clarity, specificity and simplicity.

    Figure 1
    Figure 1

    Compliance to the preoperative and postoperative SURPASS checklists according to professional background when testing the content in 29 neurosurgical procedures, June–July 2012, in one Norwegian hospital. PACU, postanaesthesia care unit.

    Focus groups

    The focus groups involved professionals having been assigned the five preoperative and three postoperative SURPASS checklists, with 2–5 professionals in each group. All the interviews, except one, had both an interviewer and a moderator. Two interviews had one healthcare provider involved, all together 25 different professionals participated. The participants had a wide range of working experience, from 6 months to 35 years, with 52% being females. Three identified codes ‘change of wording’; ‘responsibility’ and ‘organisation (of when to do the checklist)’ constituted the main categories of ‘Adapt the wording to fit clinical practice’ and ‘The checklist items challenge existing workflow’ (figure 2).

    Figure 2
    Figure 2

    Qualitative content analyses to understand eight focus groups’ perspectives on the tested preoperative and postoperative SURPASS checklist content for neurosurgical procedures in one Norwegian hospital. OT, operating theatre; PACU, postanaesthesia care unit.

    Expert panels

    Following careful text adjustments after testing the checklists in clinical practice, and adjusting items according to the suggestion from focus groups, the next step in the validation process was the CVI scoring. The expert panels’ characteristics are shown in table 1.

    View this table:
    Table 1

    Characteristics of neurosurgical personnel scoring content validity index (CVI) of the preoperative and postoperative SURPASS checklists after testing, focus groups and adjustments according to feedback in the SURPASS validation study in a tertiary teaching hospital, in Norway, 2012

    Altogether 35 different healthcare personnel scored CVIs. Six surgeons and six ward nurses scored on both the preoperative and discharging checklist. The scorings on I-CVI and S-CVI are represented in table 2.

    View this table:
    Table 2

    The item content validity index (I-CVI) and scale content validity index (S-CVI) scores by the neurosurgical experts evaluating the preoperative and postoperative SURPASS checklists after testing, focus groups and adjustments according to feedback in the SURPASS validation study in a tertiary teaching hospital, in Norway, 2012

    Examples of items having a low score (1 and 2): for surgeons: preoperative marking of the incision site; and preoperative hair removal. For ward nurses: marking of the incision site.

    Back translation of the Norwegian validated version

    Following careful adjustments after validation, the Norwegian version of the preoperative and postoperative parts of the SURPASS checklists finally consisted of 60 checklist items distributed on five preoperative and three postoperative checklists. In addition, one item was to be performed preoperatively by surgeons in case of local anaesthesia without an anaesthesiologist involved. All the original checklist items excluding the three ward nursing screenings and obtained consent were included in the Norwegian version. The content of the tested checklists and the corresponding content having been back translated are shown in online supplementary digital content 1. The back-translated checklists, including both the SURPASS parts and the WHO SSC, were approved by the Dutch SURPASS copyright holder.

    Supplemental material

    Discussion

    The English version of SURPASS’ five preoperative and three postoperative checklists were validated together with the established three parts of WHO SSC in a neurosurgical department in a tertiary hospital in Norway. The validation process consisted of six steps, including forward translation, testing the content, focus groups, expert panels, back translation and approval of the final version. There was a general positive attitude towards using checklists, although critique, reluctance and questions regarding the checklists themselves and on safety-effects were also raised. Checklist scepticism has also been documented for years in other healthcare settings.22 29–32

    Before testing the content and the flow of checklists, there was a close collaboration with management and health personnel within each profession for all checklist parts. The Dutch and Norwegian standards of healthcare are very similar, but some differences in healthcare providers’ responsibilities were disclosed. To overcome this, some items were assigned to other professions’ checklists. From the literature and our previous experience on implementation of the WHO SSC, we observe that including key stakeholders at an early stage for buy-in and to increase ownership in the process is recommended.33–35 Face validity and feasibility were confirmed before testing the content in clinical practice.

    Testing the checklists in clinical practice revealed that there were still challenges concerning wording and the existing workflow. Several studies have identified that change of workflow following checklist implementation may represent a barrier to engage the healthcare providers.35–38 Although many of the clinicians found a paper checklist most convenient for testing the content, there were logistic challenges that resulted in low compliance rates for the OT nurses. Some of the personnel were enthusiastic about systematically having a last check-up before transferring the patient. Some were engaged to give the test period a fair chance to succeed and were open-minded. Others were open on concerns, that is, another thing to spend time on in an already time-constraint environment. The managers were engaged and pointed out dedicated staff to follow up the test period. The implementation team involved and engaged the personnel thoroughly, on both group and individual levels and monitored the process closely. The WHO SSC was implemented in this hospital in 2009. It is mandatory to use, and it has a good compliance rate. However, discussions on issues regarding the WHO SSC were important, but the main focus was on testing the new SURPASS checklists.

    To get a further insight into the challenges with the existing workflow and identify wording to be improved, we conducted focus group interviews. The focus groups had several suggestions for rephrasing list contents to adapt the wording and item content into clinical practice and workflow.

    All the expert panels were instructed to score the CVI from a general surgical perspective. Still, the ‘low relevance’ scorings of specific checklist items were explained as not being important for the expert panel’s surgical discipline. However, these items could be judged as highly relevant checkpoints for other surgical departments and should be tailored to these settings accordingly. Thus, despite a low score, these items were not removed from the checklists being back translated due to generalisation to other specialities. However, the items were removed from neurosurgery checklists as a local adjustment. All the eight checklist scores had a CVI >0.80. A 90% agreement on CVI is regarded satisfactory with some authors,27 while others urge to have total agreement by all the experts if five or fewer experts.39 However, if six or more experts are scoring, the I-CVI is regarded as valid when 80% reach agreement.39 40 All the checklists reached an acceptable CVI score.

    We recommend local adaptation and testing the content in new settings to disclose and terminate barriers before implementation of additional surgical checklists.

    Strengths and limitations

    A strength of this study is the inclusion of interprofessional key stakeholders in the early process of adjusting the content to Norwegian work assignments and flows. Another is the continuous process of testing the checklists in practice with all health professional groups represented. Generally, the similarities between Dutch and Norwegian surgical safety standards increased likelihood that the checklist contents followed existing workflow and procedures. Still, three items were not included on this checklist version due to lack of protocols and work processes corresponding to these items. All new protocols and work processes should of course be implemented properly before the checklists are introduced. Prior to checklist implementation, a thorough evaluation of context, assessing corresponding work processes and procedures to checklist items has also been recommended in the literature.35

    It may be a possible limitation that the Norwegian version of the SURPASS checklists was validated in one department only. However, the original SURPASS checklists was developed through a great variety of surgical procedures and settings, to make adaptation of the checklists to other hospital departments feasible.13 Use of highly experienced and expert personnel when testing the checklists may be seen as a strength. Advices as to adaption and tailoring the content to the setting were followed.21

    Conclusion

    The SURPASS’ preoperative and postoperative checklists were successfully validated for use in Norwegian surgical care with high face validity and content validity (CVI >80%) and in combination with the WHO operative checklist. Adding new checklists in combination with the already established Sign In, Time Out and Sign Out parts of the WHO SSC was feasible in neurosurgery.

    Acknowledgments

    We would like to thank the interprofessional personnel working with neurosurgical patients who have contributed with valuable information and input on the checklist content. Also, we are grateful for the moderating done by Anne Mette Espe during the focus groups and the checklist back translational work done by anaesthesiologist Geert de Pater.

    References

    1. 1.↵
      2. Griffen FD ,
      3. Stephens LS ,
      4. Alexander JB , et al
      . The American College of Surgeons' closed claims study: new insights for improving care. J Am Coll Surg 2007;204:561–9.doi:10.1016/j.jamcollsurg.2007.01.013
      OpenUrlCrossRefPubMedWeb of Science
    2. 2.↵
      2. Kohn LT ,
      3. Corrigan JM ,
      4. Donaldson MS , et al
      . To err is human: building a safer healthcare system. Institute of Medicine, ed. Washington, DC: National Academy Press, 2000.
    3. 3.↵
      2. Lingard L ,
      3. Regehr G ,
      4. Cartmill C , et al
      . Evaluation of a preoperative team briefing: a new communication routine results in improved clinical practice. BMJ Qual Saf 2011;20:475–82.doi:10.1136/bmjqs.2009.032326
      OpenUrlAbstract/FREE Full Text
    4. 4.↵
      2. Haynes AB ,
      3. Weiser TG ,
      4. Berry WR , et al
      . A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9.doi:10.1056/NEJMsa0810119
      OpenUrlCrossRefPubMedWeb of Science
    5. 5.↵
      2. de Vries EN ,
      3. Prins HA ,
      4. Crolla RM , et al
      . Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med 2010;363:1928–37.doi:10.1056/NEJMsa0911535
      OpenUrlCrossRefPubMedWeb of Science
    6. 6.↵
      2. Haugen AS ,
      3. Søfteland E ,
      4. Almeland SK , et al
      . Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial. Ann Surg 2015;261:821–8.doi:10.1097/SLA.0000000000000716
      OpenUrlCrossRefPubMed
    7. 7.↵
      2. Manser T ,
      3. Foster S ,
      4. Flin R , et al
      . Team communication during patient handover from the operating room: more than facts and figures. Hum Factors 2013;55:138–56.doi:10.1177/0018720812451594
      OpenUrlCrossRefPubMedWeb of Science
    8. 8.↵
      2. Nagpal K ,
      3. Vats A ,
      4. Lamb B , et al
      . Information transfer and communication in surgery: a systematic review. Ann Surg 2010;252:225–39.doi:10.1097/SLA.0b013e3181e495c2
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. de Vries EN ,
      3. Ramrattan MA ,
      4. Smorenburg SM , et al
      . The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17:216–23.doi:10.1136/qshc.2007.023622
      OpenUrlAbstract/FREE Full Text
    10. 10.↵
      2. Bochner BS ,
      3. Lichtenstein LM
      . Anaphylaxis. N Engl J Med 1991;324:1785–90.doi:10.1056/NEJM199106203242506
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      2. Bull AL ,
      3. Russo PL ,
      4. Friedman ND , et al
      . Compliance with surgical antibiotic prophylaxis--reporting from a statewide surveillance programme in Victoria, Australia. J Hosp Infect 2006;63:140–7.doi:10.1016/j.jhin.2006.01.018
      OpenUrlPubMed
    12. 12.↵
      2. Cohen AT ,
      3. Tapson VF ,
      4. Bergmann JF , et al
      . Venous thromboembolism risk and prophylaxis in the acute hospital care setting (ENDORSE study): a multinational cross-sectional study. Lancet 2008;371:387–94.doi:10.1016/S0140-6736(08)60202-0
      OpenUrlCrossRefPubMedWeb of Science
    13. 13.↵
      2. de Vries EN ,
      3. Hollmann MW ,
      4. Smorenburg SM , et al
      . Development and validation of the SURgical PAtient Safety System (SURPASS) checklist. Qual Saf Health Care 2009;18:121–6.doi:10.1136/qshc.2008.027524
      OpenUrlAbstract/FREE Full Text
    14. 14.↵
      Norwegian Directorate of Health. In safe hands 24/7. 2015 http://www.pasientsikkerhetsprogrammet.no//om-oss/english (accessed 23 Jul 2018).
    15. 15.↵
      World Health Organization. Process of translation and adaption of instruments. 2018 http://www.who.int/substance_abuse/research_tools/translation/en/# (accessed 23 Jul 2018).
    16. 16.↵
      World Health Organization. WHO Surgical safety checklist. 2009 http://apps.who.int/iris/bitstream/10665/44186/2/9789241598590_eng_Checklist.pdf (accessed 23 Jul 2018).
    17. 17.↵
      2. Haugen AS ,
      3. Søfteland E ,
      4. Eide GE , et al
      . Impact of the World Health Organization’s Surgical Safety Checklist on safety culture in the operating theatre: a controlled intervention study. Br J Anaesth 2013;110:807–15.doi:10.1093/bja/aet005
      OpenUrlCrossRefPubMedWeb of Science
    18. 18.↵
      Norwegian Knowledge Centre for the Health Services. Sjekkliste for trygg kirurgi. 2010 http://www.who.int/patientsafety/safesurgery/tools_resources/SSC_bokm_al.pdf?ua=1 (accessed 23 Jul 2018).
    19. 19.↵
      2. Haugen AS ,
      3. Wæhle HV ,
      4. Almeland SK , et al
      . Causal Analysis of World Health Organization’s Surgical Safety Checklist Implementation Quality and Impact on Care Processes and Patient Outcomes: Secondary Analysis From a Large Stepped Wedge Cluster Randomized Controlled Trial in Norway. Ann Surg 2017 [Epub ahead of print 6 Nov 2017].doi:10.1097/SLA.0000000000002584
    20. 20.↵
      2. Haugen AS ,
      3. Bakke A ,
      4. Løvøy T , et al
      . Preventing complications: the preflight checklist. Eur Urol Focus 2016;2:60–2.doi:10.1016/j.euf.2016.01.014
      OpenUrl
    21. 21.↵
      World Health Organization. Implementation manual - WHO surgical safety checklist. 2008 http://www.who.int/patientsafety/safesurgery/tools_resources/SSSL_Manual_finalJun08.pdf (accessed 23 July 2018).
    22. 22.↵
      2. Conley DM ,
      3. Singer SJ ,
      4. Edmondson L , et al
      . Effective surgical safety checklist implementation. J Am Coll Surg 2011;212:873–9.doi:10.1016/j.jamcollsurg.2011.01.052
      OpenUrlCrossRefPubMed
    23. 23.↵
      2. Thomassen Ø ,
      3. Storesund A ,
      4. Søfteland E , et al
      . The effects of safety checklists in medicine: a systematic review. Acta Anaesthesiol Scand 2014;58:5–18.doi:10.1111/aas.12207
      OpenUrlCrossRefPubMedWeb of Science
    24. 24.↵
      2. Krueger RA ,
      3. Casey MA
      . Focus groups: a practical guide for applied research. Los Angeles: Sage, 2015.
    25. 25.↵
      2. Graneheim UH ,
      3. Lundman B
      . Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004;24:105–12.doi:10.1016/j.nedt.2003.10.001
      OpenUrlCrossRefPubMedWeb of Science
    26. 26.↵
      2. Polit DF ,
      3. Beck CT
      . The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health 2006;29:489–97.doi:10.1002/nur.20147
      OpenUrlCrossRefPubMedWeb of Science
    27. 27.↵
      2. Waltz CF OLS ,
      3. Lenz ER
      . Measurement in nursing and health research. 3rd edn. New York: Springer Publishing Company, 2005.
    28. 28.↵
      2. Davis LL
      . Instrument review: getting the most from a panel of experts. Applied Nursing Research 1992;5:194–7.doi:10.1016/S0897-1897(05)80008-4
      OpenUrlCrossRef
    29. 29.↵
      2. Thomassen O ,
      3. Brattebø G ,
      4. Heltne JK , et al
      . Checklists in the operating room: Help or hurdle? A qualitative study on health workers' experiences. BMC Health Serv Res 2010;10:342.doi:10.1186/1472-6963-10-342
      OpenUrlCrossRefPubMed
    30. 30.↵
      2. Fourcade A ,
      3. Blache JL ,
      4. Grenier C , et al
      . Barriers to staff adoption of a surgical safety checklist. BMJ Qual Saf 2012;21:191–7.doi:10.1136/bmjqs-2011-000094
      OpenUrlAbstract/FREE Full Text
    31. 31.↵
      2. Russ SJ ,
      3. Sevdalis N ,
      4. Moorthy K , et al
      . A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: lessons from the "Surgical Checklist Implementation Project". Ann Surg 2015;261:81–91.doi:10.1097/SLA.0000000000000793
      OpenUrlCrossRefPubMed
    32. 32.↵
      2. Borchard A ,
      3. Schwappach DL ,
      4. Barbir A , et al
      . A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg 2012;256:925–33.doi:10.1097/SLA.0b013e3182682f27
      OpenUrlCrossRefPubMed
    33. 33.↵
      2. Leape LL
      . The checklist conundrum. N Engl J Med 2014;370:1063–4.doi:10.1056/NEJMe1315851
      OpenUrlCrossRefPubMedWeb of Science
    34. 34.↵
      2. Russ S ,
      3. Rout S ,
      4. Sevdalis N , et al
      . Do safety checklists improve teamwork and communication in the operating room? A systematic review. Ann Surg 2013;258:856–71.doi:10.1097/SLA.0000000000000206
      OpenUrlCrossRefPubMed
    35. 35.↵
      2. Hull L ,
      3. Athanasiou T ,
      4. Russ S
      . Implementation science: a neglected opportunity to accelerate improvements in the safety and quality of surgical care. Ann Surg 2017;265:1104–12.doi:10.1097/SLA.0000000000002013
      OpenUrl
    36. 36.↵
      2. Bergs J ,
      3. Lambrechts F ,
      4. Simons P , et al
      . Barriers and facilitators related to the implementation of surgical safety checklists: a systematic review of the qualitative evidence. BMJ Qual Saf 2015;24:776–86.doi:10.1136/bmjqs-2015-004021
      OpenUrlAbstract/FREE Full Text
    37. 37.↵
      2. De Bie AJR ,
      3. Nan S ,
      4. Vermeulen LRE , et al
      . Intelligent dynamic clinical checklists improved checklist compliance in the intensive care unit. Br J Anaesth 2017;119:231–8.doi:10.1093/bja/aex129
      OpenUrl
    38. 38.↵
      2. Gillespie BM ,
      3. Marshall AP ,
      4. Gardiner T , et al
      . Impact of workflow on the use of the Surgical Safety Checklist: a qualitative study. ANZ J Surg 2016;86:864–7.doi:10.1111/ans.13433
      OpenUrl
    39. 39.↵
      2. Lynn MR
      . Determination and quantification of content validity. Nurs Res 1986;35:382–5.doi:10.1097/00006199-198611000-00017
      OpenUrlCrossRefPubMedWeb of Science
    40. 40.↵
      2. Polit DF ,
      3. Beck CT
      . Nursing Research: generating and assessing evidence for nursing practice. Philadelphia: Wolters Kluwer, 2017.

    Footnotes

    • Patient consent for publication Not required.

    • Contributors AS, ASH, MAB and ES contributed with the conception and design of the study. AS, ASH, HVW, RM and ES: acquisition of data. AS, ASH, HVW, MWN, MAB and ES: analysis and interpretation of data. AS drafted the article and designed the figures. ASH, MWN and ES supervised the project. AS is the overall content guarantor for the manuscript.

    • Funding AS was supported by a research grant from the Western Norway Regional Health Authority Trust, the Patient Safety Program (grant number HV1173) and Norwegian Nurses Organisation (grant number 15/0023). ASH was supported by a postdoctoral fellow research grant from the Western Norway Regional Health Authority Trust, the Patient Safety Program (grant number HV1172). HVW was supported by a research grant from the Western Norway Regional Health Authority Trust, the Patient Safety Program (grant number HV1174).

    • Disclaimer The funders have not been involved in the study design, data collection, analysis, interpretation of data, writing of the report or to the decision to submit the paper for submission.

    • Competing interests None declared.

    • Ethics approval The study was approved by the Western Norway Regional Ethical Research Committee (2012/560/REK West).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.