Article Text
PDF
Abstract
Background Systematically observing clinical performance in the operating room (OR) to support patient safety initiatives faces numerous logistical and methodological challenges. These may be solved by new audio-video recording technologies like the OR Black Box, which is a tool similar to black boxes in aviation. This study aimed to identify barriers and enablers that may influence patients’, clinicians’ and senior leadership team members’ support of the OR Black Box in order to guide its future implementation.
Methods Patients, clinicians and senior leadership team members were recruited to participate in semistructured interviews informed by the theoretical domains framework (TDF) to identify factors relevant to planning OR Black Box implementation. Deidentified interview transcripts were analysed in duplicate following a TDF coding structure.
Results Data saturation was achieved at 15 patients, 17 clinicians and 9 senior leadership team members. Seven domains were relevant for patients, nine for clinicians and four for senior leadership. Knowledge and Beliefs about consequences were barriers and enablers for all three groups. Memory, attention and decision processes and Social influences were enablers for both clinicians and senior leadership. Environmental context and resources, Emotion and Behavioural regulation were found to be barriers and enablers for both clinicians and patients. Social/professional role and identity and Reinforcement were enablers for patients only and Optimism and Intentions were barriers and enablers to clinicians.
Conclusions While most stakeholders were supportive of the OR Black Box, we identified many key areas that need to be addressed during its implementation. It is critical to ensure all stakeholders have adequate and accurate information about the OR Black Box system and research goals, and that the OR Black Box is positioned as a patient safety initiative for learning from and improving practice.
- patient safety
- healthcare quality improvement
- implementation science
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Despite advances in anaesthetic and surgical care, there has not been any substantial reduction in patient safety events in recent years.1 2 Retrospective analyses of patient safety incidents are subject to recall bias, unable to detect near misses and limited in their ability to precisely determine important precursors of adverse events.3–10 Prospective observation in the operating room (OR), however, would allow us to identify and learn from errors and near misses, monitor underlying trends and patterns in everyday practice and proactively detect and prevent future incidents of patient harm.
The OR Black Box was developed to collect comprehensive and accurate OR data without requiring human observers to be physically present in the room at the time of the surgery.11 Similar to black boxes used in aviation, the OR Black Box captures audio, video, physiological and environmental data from the clinical environment. It provides synchronised data capture and allows for robust analysis by expert raters and software-based algorithms. These data are included in a ‘surgical timeline’, which displays individual, team and systems-level information as well as intraoperative errors, events and near misses in a single layout. This makes it possible to analyse the succession of events leading to a near miss or complications. It also allows us to determine actionable safety threats (eg, distractions, workflow issues), opportunities for standardisation (eg, handover protocol) and positive practices or resilience supports (eg, coaching, in situ training, use of the surgical time out).12 Accordingly, interventions such as practice guidelines (eg, noise control at specific surgical phases) or feedback to teams can be developed that are both holistic and tailored to the local context.
Since the OR Black Box is a new technology, our institution was among the first to implement it, leaving us little opportunity to learn from many other centres on how to optimise implementation. The OR Black Box may naturally raise many questions related to ethics, privacy, legal concerns and patient safety, making it important to empirically identify key issues among stakeholders. This information can then be used to guide the planning stage of implementation according to a ‘plan, do, check, act’ improvement model.13–16In this study, we aimed to explore the perceptions of hospital stakeholders towards the OR Black Box to inform future implementation at our centre and provide critical insights for other hospitals planning to adopt this technology.
Methods
This manuscript is reported according to the Standards for Reporting Qualitative Research checklist.17
Study design and interview guide
We used a theoretically informed qualitative research design. Semistructured interviews were conducted with surgical patients, perioperative clinicians (OR nurses, anaesthesiologists, surgeons) and hospital senior leadership team members. All interviews were audio recorded and conducted in English by a trained researcher. Interviews were then transcribed verbatim and deidentified during this process for analysis. Strict anonymity of all participants was maintained at all times.
The interview guide was based on the theoretical domains framework (TDF). The TDF is an integrative framework which comprised 14 domains that can be used to provide a comprehensive theoretical assessment of implementation problems in healthcare settings.18–20 The domains include: Knowledge; Skills; Social/professional role and identity; Beliefs about capabilities; Beliefs about consequences; Intention; Goals; Optimism; Reinforcement; Memory, attention and decision processes; Environmental context and resources; Social influences; Emotion; and Behavioural regulation. Investigation of each of the 14 domains was represented by two to three questions. The definitions of each domain as applied within the context of this study are listed in table 1. The interview guide was designed based on recommendations in the literature19 and advice from experts in qualitative research and behaviour change on our coinvestigator team (AMP, JP, JMG). Methods used in this study were consistent with a recently published guide21 and are briefly described below.
Definitions of theoretical domains framework (TDF) domains within the context of OR Black Box implementation
Context
The study took place across all three campuses at The Ottawa Hospital, a large tertiary care centre in Ottawa, Ontario, Canada. Each campus serves unique patient populations, specialising in different surgical services.
Participants
All patients scheduled to undergo any form of surgery were eligible for participation. Patients were recruited and interviewed at the preoperative assessment clinic at each of the three hospital campuses. Clinicians were eligible to participate if they worked in the OR and belonged to one of the four core OR team professional groups (ie, scrub nurse, circulating nurse, anaesthesiologist, surgeon). All senior leadership team members were eligible to participate, regardless of their specific role. Clinicians and senior leadership team members were recruited via email and interviewed at a location and time of their convenience, either in person or over the phone.
Outcomes
We were primarily interested in perceived barriers and facilitators concerning whether participants would support implementing OR Black Box.
Researcher characteristics and reflexivity
Interviews were conducted by research assistants without clinical backgrounds to avoid any potential reluctance for participants to participate based on perceived impacts on their care or professional standing or reluctance to express non-favourable views of the OR Black Box.
Sampling strategy
Sample size was determined using the concept of data saturation, whereby no new data were identified in three successive interviews.22 We anticipated that 8–15 interviews per stakeholder group (ie, patients, clinicians, senior leadership) would be conducted, based on previous TDF studies in clinical settings.23–25
Analysis
Deidentified transcripts were analysed in NVivo (V.11, QSR International, Doncaster, Australia) using a theory-informed, qualitative approach. Two researchers (NE, AU) first coded a pilot interview together to develop a coding scheme. A second pilot interview was coded independently to establish reliability and an understanding of the coding scheme. Preliminary insights were discussed, followed by development of a coding structure based on the TDF. The TDF coding structure was used by each researcher to independently code all interviews. Discrepancies in coding were resolved through discussion and consensus. If agreement about single domain allocation could not be reached, the response was allocated to both identified domains.
Interview data were subsequently coded into common themes/specific beliefs within each TDF domain. The belief statements were generated by one coder (NE) and confirmed by a second coder (AU). Belief statements were phrased in a way to capture common meaning across participants, representing related participant responses.19 Representative quotations were then selected to demonstrate the nuances of each belief statement.
Consensus discussion was used by the two coders to identify key domains affecting whether individuals would participate in or support OR Black Box research. Domains were then confirmed by members of the research team with expertise in using the TDF (AMP, JP). Together, these methods ensured trustworthiness of the data.26 Three factors were considered concurrently to establish domain relevance (ie, whether that domain should be addressed in OR Black Box implementation): (1) perceived strength of the beliefs impacting the behaviour; (2) presence of conflicting beliefs; and (3) frequency of the beliefs across interviews.24 25 I
Patient and public involvement
Patients were involved as participants in this study.
Results
Participants
Fifteen patients (5 per campus; 3 female, 12 male), 17 clinicians (7 female; 10 male) and 9 senior leadership team members (1 female; 8 male) were interviewed. Patients were scheduled to undergo a range of surgical procedures, including hepatectomy, ileostomy, hip replacement, cholecystectomy, knee replacement, shoulder replacement, sleeve gastrectomy and laryngectomy. Of the clinicians, four were nurses, four were anaesthesiologists and nine were surgeons. Participating surgical specialties included thoracic (n=2), general (n=2), orthopaedics (n=1), urology (n=2), obstetrics and gynaecology (n=1) and cardiac (n=1).
Data characteristics
The mean (SD) interview duration was 29 min (SD=1 min), and data saturation was reached after 15 patient interviews, 17 clinician interviews and 9 senior leadership interviews. From the 41 interviews, 1210 utterances were coded into the 14 TDF domains.
Main results
Relevant enablers and barriers to guide future OR Black Box implementation are summarised in table 2. Specific results for each stakeholder group are discussed below.
Barriers and enablers to OR Black Box implementation identified using the theoretical domains framework (TDF)
Patients
Relevant TDF domains likely to inform whether patients would participate in OR Black Box research are summarised in online supplementary appendix 1. Seven theoretical domains were identified as influencing patients’ decision to participate in OR Black Box research: Knowledge; Social/professional role and identity; Beliefs about consequences; Reinforcement; Environmental context and resources; Emotions; and Behavioural regulation.
Supplemental material
Most patients expressed a general idea of how they thought the OR Black Box would be used, with many referencing the purpose of black boxes in aviation (Knowledge). Patients largely focused on the use of the OR Black Box for reconstructing events, providing objective data and promoting safety. Several were under the impression that the OR Black Box was already in use at the hospital, while few indicated that they did not know or were not sure about how the OR Black Box would work (Knowledge).
Some patients reported that they had a role to play in their safety and that supporting implementation of the OR Black Box simply be part of the process of being a patient (Social/professional role and identity). However, other patients reported that they did not view the OR Black Box as falling within their patient role and that it was more a tool for healthcare professionals. Several patients indicated that because of their professional role (eg, insurance broker, engineer), they viewed the OR Black Box as valuable and would therefore participate.
Patients commonly reported that the OR Black Box would improve care in the future and would make surgery safer now (Beliefs about consequences). Knowing that their participation might improve care for others was also reported to be an incentive for patients (Reinforcement). Beyond this, they reported wanting additional information about the research, confidentiality, privacy and impact of the OR Black Box before making their decision (Behavioural regulation). They did not, however, desire monetary compensation (Reinforcement).
Patients generally expected healthcare providers to perform better in ORs equipped with an OR Black Box, although some acknowledged it could potentially make their surgeon more nervous (Beliefs about consequences). Others thought that because clinicians are professionals, they should perform the same regardless of whether or not the OR Black Box was present (Beliefs about consequences).
Approximately half of the patients interviewed reported the hospital environment would have no effect on their decision to participate, while the other half noted that their trust in the hospital, its standard practices and its reputation as a learning environment played a role in their decision (Environmental context/resources). Most patients did not report having any concerns about the use of the OR Black Box at the hospital, while a few acknowledged it might make them feel a little nervous (Emotions). Those who expressed some concerns indicated that it would not necessary result in a decision not to participate, but rather, that they would require more information first (Behavioural regulation).
Seven domains appeared to be less relevant (Skills; Beliefs about capabilities; Optimism; Intentions; Goals; Memory, attention and decision processes; Social influences). Most patients, for example, indicated that to them, the OR Black Box was just another piece of equipment in the OR and would not create problems for them to proceed with their planned surgery. The majority of patients reported that the decision to participate in OR Black Box research would be easy to make, and many noted that they would not even think about it at all going into an OR (Memory, attention and decision-making). Patients largely appeared to favour the implementation of the OR Black Box at the hospital and identified no competing goals (Goals). They also noted that moving forward with technology was a good thing for the hospital and that the OR Black Box would one day become the norm for hospitals (Optimism).
Clinicians
Nine domains were identified as relevant for influencing clinicians’ participation in OR Black Box research: Knowledge; Optimism; Beliefs about consequences; Intentions; Memory, attention and decision processes; Environmental context and resources; Social influences; Emotion; and Behavioural regulation (online supplementary appendix 2). Domains identified as relevant were generally consistent across OR professions (nursing, anaesthesia, surgery), although there were some thematic differences between nurses and surgeons, which are discussed below.
Supplemental material
Some clinicians reported a general understanding of the concept of a black box and had some sense of what it might capture within the OR, but many more had questions about its purpose and how it would actually work (Knowledge). Questions were raised concerning the consent process, the security of the information collected and whether technical equipment would interfere with the surgery.
There were mixed responses regarding how the OR Black Box would impact clinical practice (Beliefs about consequences). Some clinicians indicated that they would simply perform as they do every day but would then use the OR Black Box reports to improve their practice, while others did not foresee use of a black box changing their practice (Beliefs about consequences). Some reported concerns about the OR Black Box being a distraction in the OR, which could potentially lead to practice errors (Beliefs about consequences).
Many clinicians expressed a belief that the OR Black Box would gain momentum and support over time, with the potential to create positive change for patient safety, though a few others expressed doubts about its role (Optimism). Those who were less optimistic about whether the OR Black Box would lead to any significant changes were primarily surgeons. One participant appeared to have a particularly negative view on the use of the OR Black Box, suggesting that it implied a lack of trust in staff at the hospital (Beliefs about consequences). Overall, clinicians appeared to have mixed feelings about the OR Black Box (Emotions). While some reported feeling excited, happy, unconcerned or comfortable, others reported feeling worried, scared, threatened or concerned about its implementation.
Most of the clinicians interviewed indicated that the OR Black Box would positively impact clinical practice as it would lead to such outcomes as learning opportunities, improved team dynamics, identification of systemic errors and elimination of unnecessary care practices (Beliefs about consequences). Still, others (primarily surgeons) expressed that the OR Black Box would delay cases, increase pressure in the OR and could result in punitive actions against staff or medicolegal problems (Beliefs about consequences). Many surgeons commented that the OR Black Box would negatively change the OR environment as people may interact differently with each other or might be less relaxed. All nurses interviewed, however, expressed that they expected the OR Black Box to improve professionalism in the OR (Beliefs about consequences). It was also noted by several clinicians that not implementing the OR Black Box would actually impede progress at the hospital.
In order to encourage them to participate in OR Black Box research, most clinicians expressed that they would either need more information about the research itself and/or the clinical impact of having the technology in the room (Knowledge/Behaviour regulation). Regarding their motivation to participate, most clinicians were either had the intention to participate or were undecided (Intentions). The decision to be recorded was anticipated to be easy by many participants, although some indicated that they would need to take time to consider whether they would participate (Memory, attention and decision processes).
Many clinician participants reported that the time involved to participate would influence their decision, given that the OR environment is already quite time pressured with an emphasis on efficiency and cost-effectiveness (Environmental context and resources). Some participants also reported that the OR Black Box aligned with the institutional values of the hospital as a learning organisation, while others saw no relationship (Environmental context and resources). Many participants noted that because they knew the individual leading the project, they trusted it was worthwhile, and would thus participate (Social influences). The willingness of other clinical team members to participate or having a supportive team in general also appeared to be important for clinicians. Still, about one-third of participants said that no other individual would influence their decision to participate.
Five domains were not relevant for clinicians: Skills; Social/professional role and identity; Beliefs about capabilities; Reinforcement; and Goals. The majority of participants reported that the OR Black Box fit within their role as a healthcare professional to improve processes of care and patient safety (Social/professional role and identity). This belief appeared to be strongest among nurses. Some clinicians also indicated that they would participate given their role as an academic physician and felt a responsibility to support research. Most indicated that the potential benefits to their practice or patients were incentive enough for them (Reinforcement). Many healthcare providers also reported that the OR Black Box aligned with their own personal or professional goals in terms of what they want for the OR and from the OR Black Box (Goals).
Senior leadership
Four domains were relevant to senior leadership team members: Knowledge; Beliefs about consequences; Memory, attention and decision processes; and social influences (online supplementary appendix 3).
Supplemental material
All senior leadership team members interviewed were familiar with the OR Black Box to some extent, although a few had questions about how it would impact staff (Knowledge/Beliefs about consequences). While most participants expected the OR Black Box to lead to improvements in patient safety culture, processes of care and patient outcomes, some expressed concerns that it could potentially have a harmful impact if not introduced appropriately (Beliefs about consequences). For example, participants expressed the OR Black Box would not be successful if used punitively. Most feelings of concern involved potential reactions by colleagues, including staff (Beliefs about consequences). Many senior leadership team members also stated that there would be negative consequences if the OR Black Box were not implemented at the hospital given it represents a critical opportunity to be a leading healthcare organisation and to improve practice.
For most senior leadership team members, the decision to support OR Black Box research was a ‘no-brainer’ (Memory, attention and decision processes). For some team members, however, a cost-benefit analysis was performed or the use of the technology at other centres factored into the decision-making processes. Most also reported supporting the OR Black Box because it is largely supported by their colleagues, although one person expressed concern about being judged by colleagues for not supporting the initiative (Social influences).
The remaining 10 domains were not relevant for senior leadership team members. For example, participants mostly indicated the OR Black Box within their professional role as leaders and/or researchers (Social/professional role and identity). The majority of team members also reported that the OR Black Box fit within the context of the hospital as an academic centre with a ‘just culture’ (Environmental context and resources) and that they were highly motivated to support its use (Intentions).
Summary of results
Seven domains were relevant for patients, nine for clinicians and four for senior leadership. Knowledge and Beliefs about consequences were relevant for all three groups. Memory, attention and decision processes and Social influences were relevant for both clinicians and senior leadership while Environmental context and resources, Emotion and Behavioural regulation were relevant for both clinicians and patients. Social/professional role and identity and Reinforcement were relevant to patients only and Optimism and Intentions were unique to clinicians. Based on our results, key factors that may improve uptake and future implementation of the OR Black Box are summarised in box 1.
Key factors to improve uptake and future implementation of the Operating Room (OR) Black Box
Be transparent with stakeholders: provide clear information on how the technology works, privacy/confidentiality, legal and ethical considerations, research goals, resource requirements and implications for clinical practice.
Situate the OR Black Box within a learning context: opportunity to learn from everyday practice and fit with teaching hospital goals.
Build support within front-line professional groups and identify ‘champions’.
Promote patient engagement.
Highlight the potential of the OR Black Box to improve practice and patient safety.
Discussion
Key results
This study identified factors relevant to supporting OR Black Box implementation for patients, clinicians and senior leadership team members using the TDF. We identified specific barriers to implementation as well as areas that can be used to promote the OR Black Box. Below, we highlight key themes that may be targeted in the planning stage of implementation to improve uptake of the OR Black Box.
First, it is noteworthy that for all three stakeholder groups, both Knowledge and Beliefs about consequences were relevant. For patients, clinicians and senior leadership it was clear that the concept of the OR Black Box was easy to grasp, but questions about how the system works (including security), the goals of the research and the implications for clinical practice were pertinent. Despite these questions, the majority of patients, clinicians and senior leadership team members indicated that using the OR Black Box for research would improve surgical patient care and represented an opportunity for progress. Moving forward with OR Black Box implementation, it will be important to provide clear and comprehensive information, including legal, ethical and privacy considerations, to all stakeholders and to emphasise the potential to create positive change in surgical safety. This may be especially critical for clinicians, who appeared to have more potential concerns than the other two groups, particularly regarding how the OR Black Box could impact practice, professional standing and interpersonal dynamics in the OR. Providing information about the OR Black Box and planned research appeared to be a necessary condition of participation for both clinicians and patients (as found in the Behavioural regulation domain). Information could also be key to managing the anxieties or uneasiness expressed by many clinicians and some patients in the Emotions domain. In addition, it may be important to situate the OR Black Box within a learning context and once again provide information concerning its resource requirements (as found in the Environmental context and resources domain).
For both senior leadership and clinicians, the domains Social influences and Memory, attention and decision processes were relevant. Each group appeared to value the opinions of their colleagues on the issue and personally knowing and trusting the principal investigator was also an important factor. Within both groups, however, some participants expressed they would need time to think about implementation or to weigh the pros and cons. These domains highlight the importance of building support within professional groups, including identifying OR Black Box ‘champions’ within each group.
This study has also revealed potential professional nuances that may need to be considered in OR Black Box implementation. For example, nurses tended to see immediate value in using the OR Black Box and identified a strong link between the OR Black Box and their professional role of providing care to patients. Conversely, surgeons had more questions concerning privacy and medicolegal implications, which may reflect that surgeons/physicians commonly perceive this legal risk as more prominent than nurses. Overall, it is important to be aware when implementing the OR Black Box that different professions have different concerns and implementation should be tailored accordingly.
It is interesting that the Social/professional role and identity domain was relevant for patients only. This may speak to the importance of patients playing an active role in their care and providing opportunities for patient engagement within the OR Black Box research programme. There is currently a knowledge gap regarding engaging patients in surgical safety research27 and OR Black Box implementation may represent a unique opportunity to fill this gap. Since clinicians play a central role in patient safety, one may have expected the clinician group to have also considered that it was their responsibility to support implementing the OR Black Box to contribute to improving patient safety initiatives. However, it may be that perceived challenges and threats outbalanced possible perceived patient safety benefits overall.
From our analysis, there are some universal messages that are relevant across patients, clinicians and senior leadership and some messages that must be tailored to the unique barriers/enablers identified within each group. These findings were used by our group to directly inform our OR Black Box implementation strategy at our centre. We incorporated key messages identified here into our hospital-wide information campaign targeting each stakeholder group. Multichannel strategies for dissemination of messages related to each target area might include print (eg, posters, pamphlets) and electronic material (eg, email, website) as well as presentations at grand rounds or staff meetings. The identified theoretical domains can be used by other centres to inform practical strategies for implementing the OR Black Box.
Limitations
This study involved one hospital only and is the first to formally explore stakeholder perceptions around implementation of the OR Black Box. As such, the extent to which the barriers and enablers identified here apply to stakeholders at other centres is unknown. It is recommended that centres planning to implement the OR Black Box follow our approach to identify relevant factors for their local context. Determining the generalisability of these findings to other contexts was beyond the scope of this study. Still, our approach can be used to inform implementation at other centres, demonstrating the importance of consulting with local stakeholders.
It should also be noted that most senior leadership team members were supportive of the OR Black Box and this may be the result of being engaged in the implementation process from the beginning in addition to having invested resources. Nevertheless, their positive perceptions may be encouraging for other centres planning to implement the OR Black Box.
Interpretation
While most stakeholders are supportive of the OR Black Box and would agree to be recorded, there are many key areas that need to be anticipated and addressed in the planning phase to successfully implement the OR Black Box. It is critical to ensure all stakeholders have adequate and accurate information about the OR Black Box system and research goals, and that the OR Black Box is positioned as a patient safety initiative for learning from and improving everyday practice.
Conclusion
There are many important barriers and enablers for patients, clinicians and senior leadership team members regarding implementation of the OR Black Box. For optimal success, it may be essential to continually align use of the OR Black Box with messages identified by key stakeholders.
Supplemental material
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Ethics approval was obtained from the Ottawa Health Science Network Research Ethics Board (Protocol No 20170117-01H).
References
Footnotes
Correction notice This article has been corrected since it was published. Author name 'Cole Etherington' has been updated.
Contributors All authors meet each of the following criteria: Substantial contribution to conception and design, acquisition of data, or analysis and interpretation of data: NE, AU, AMP, CT, APB, JP, JMG, SB. Drafting the article or revising it critically for important intellectual content: NE, AU, AMP, CT, APB, JP, JMG, SB. Final approval of the version to be published: NE, AU, AMP, CT, APB, JP, JMG, SB. Agreement to be accountable for all aspects of the work thereby ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved: NE, AU, AMP, CT, APB, JP, JMG, SB.
Funding This study was funded by The Ottawa Hospital Academic Medical Organization (TOHAMO). Dr. Boet was suported by The Ottawa Hospital Anaesthesia Alternate Funds Association.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.