Article Text
PDF
Abstract
Introduction Comprehensive data capture systems such as the Operating Room Black Box (OR Black Box) are becoming more widely implemented to access quality data in the complex environment of the OR. Prior to installing an OR Black Box, we assessed perceptions on safety attitudes, impostor phenomenon and privacy concerns around digital information sharing among healthcare professionals in the OR. A parallel survey was conducted in Canada, hence, this study also discusses cultural and international differences when implementing new technology in healthcare.
Methods A cross-sectional survey using three previously validated questionnaires (Safety Attitudes Questionnaire (SAQ), Clance Impostor Phenomenon Scale, Dispositional Privacy Concern) was distributed through Research Electronic Data Capture to 145 healthcare professionals from the OR (July to December 2019). Analysis of variance and analysis of covariance were used to test for differences.
Results 124 responded (86%): 100 completed the survey (69%) (38 nurses, 10 anaesthesiologists, 36 obstetricians/gynaecologists, 16 residents). Significant variability in all six SAQ domains, safety climate and teamwork being the lowest ranked and job satisfaction ranked highest for all groups. The SAQ varied in all domains in Canada. Moderate to frequent impostor phenomenon was experienced by 71% predominantly among residents (p=0.003). 72% in the Canadian study. Residents were most comfortable with digital information sharing (p<0.001), only 13% of all healthcare professionals were concerned/heavy concerned compared with 45% in Canada.
Conclusions The different healthcare professional groups had diverse perceptions about safety culture, but were mainly concerned about safety climate and teamwork in the OR. Impostor phenomenon decreased with age. All groups were unconcerned about digital information sharing. The Canadian study had similar findings in terms of impostor phenomenon, but a variety within the SAQ and were more concerned about data safety, which could be due to medical litigation per se and is not widespread in Scandinavia compared with North America.
- Anaesthesia
- Attitudes
- Continuous quality improvement
- Healthcare quality improvement
- Obstetrics and gynecology
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Anaesthesia
- Attitudes
- Continuous quality improvement
- Healthcare quality improvement
- Obstetrics and gynecology
WHAT IS ALREADY KNOWN ON THIS TOPIC
New data-driven technology which captures the complexity of the operation room with video and audio to access quality data is an emerging research area. However, little is known about how these initiatives impact healthcare professionals.
WHAT THIS STUDY ADDS
Insight and knowledge about factors, barriers and mechanisms that influence the implementation process prior to installation of such new technology. Additionally, discussions of differences and similarities among surgical healthcare staff working on different continents.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Our findings will feed into development of sustainable implementation strategies when implementing data-driven technology in the operation room.
Introduction/background
In the operating room (OR), a high-risk environment, up to 50% of adverse events are potentially preventable.1–4 Increased transparency in the surgical field is needed to enhance patient safety. In the surgical community this has resulted in initiatives like the Operating Room Black Box (OR Black Box), a platform that continuously captures multiple intraoperative data feeds, including audiovisual data from the OR and physiological parameters (among others), allowing comprehensive analysis of efficiency and safety.5 6 These data are analysed to understand the complexity of the OR and to identify events preceding adverse events. The OR Black Box can also be used to provide constructive feedback and to identify the need for continuing education for healthcare professionals.5 6
When collecting comprehensive data, especially audiovisual recordings, various barriers and enablers may arise in the healthcare professionals involved, both on a personal and organisational level.7–11 Concerns have been raised about privacy, risk for medical litigation and whether data can be used for disciplinary action.12–14 This emphasises the challenge of balancing data collection with individual concerns when implementing systematic audiovisual recording.10
Healthcare professionals may have differing perspectives on these types of initiatives.11 Reluctance may be a result of privacy concerns or related to impostor phenomenon,15 which is a chronic feeling of self-doubt and the fear of being discovered as an intellectual fraud is known in the medical world but not well studied in the operation room.7 8 13 15–18 Imposter phenomenon, or syndrome as some refer to it, occurs when high-achieving individuals have a pervasive sense of self-doubt combined with fear of being exposed as a fraud, this despite objective measures of success. A hypothesis is that individuals with imposter traits may feel worried or threatened by initiatives like the OR Black Box. If handled with care, systematic audiovisual recording can lay the foundation for interprofessional bridge building and provide an excellent opportunity to achieve greater transparency in healthcare.7
Prior to implementation of a comprehensive data capture system like the OR Black Box in the gynaecological surgery suites we conducted a single-site cross-sectional survey of healthcare professionals emphasising existing patient safety attitudes, occurrence of impostor phenomenon and perceptions about privacy concerns. The aim was to gather knowledge on perceptions to support a more sustainable implementation. We have compared results with a recent parallel survey conducted in Toronto by our Canadian research partners.19
Materials and methods
A single-site cross-sectional survey was designed using three previously validated questionnaires and a set of OR Black Box-related questions. There had been two 30 min educational sessions about the OR Black Box platform before the survey and a short informative description in the survey.
Survey design
We devised a survey containing 83 questions, divided into five categories: (1) demographics—8 questions, (2) Safety Attitudes Questionnaire (SAQ)20 21—34 questions, (3) Clance Impostor Phenomenon Scale (CIPS)22 23—20 questions, (4) Dispositional Privacy Concern (DPC)15—9 questions and (5) the OR Black Box—12 questions. The questions measuring participant beliefs and opinions towards the OR Black Box were developed in collaboration with our research partners from the University of Toronto and St Michael’s Hospital, Toronto, Canada.19 A 5-point Likert scale (strongly disagree, slightly disagree, neutral, slightly agree, strongly agree) was used for the entire survey. A score of 3 is considered neutral, while scores below 3 represent a negative opinion of the OR Black Box and scores above 3 represent a positive opinion. The entire survey was in Danish and participants were not given any financial incentives.
The Research Electronic Data Capture, a secure web application, was used to send emails with the survey to participants, each assigned a unique identifier to anonymise the data. Data collection started in July 2019 and ended in December 2019 with a total of four reminder mails sent out in the beginning of the month.
Survey measures and interpretation of scores
Safety Attitudes Questionnaire
Previously validated in a Danish setting,21 the SAQ covers six domains: (1) teamwork climate, (2) safety climate, (3) job satisfaction, (4) stress recognition, (5) perception of management and (6) working conditions. An item score of 4 or 5 indicates a positive response. The total score can range from 0 to 100, a score of >75 indicating a positive attitude towards overall institutional safety culture.
Clance Impostor Phenomenon Scale
CIPS measures self-perception of intellectual and professional fraudulence using an average score ranging from 5 to 100. A high score (>80) indicates occurrence of more frequent and intense impostor phenomenon (≤40: few signs of occurrence, 41–60: moderate, 61–80: frequent and intense).
Dispositional Privacy Concern
DPC measures inherent privacy concern and explores individuals’ desire for privacy and their concerns about the privacy behaviours of organisations and governments. The DPC consists of six questions and each item is rated using a 5-point Likert scale. A high score indicates increased comfort with digital information sharing.
OR Black Box-related questions
Questions exploring litigation, changes in communication and collaboration and willingness to receive feedback. These 12 questions were self-constructed in collaboration with our Canadian colleague and identical to their questions.19 Each question uses a 5-point Likert scale with anchoring text. Higher value represents a more positive attitude towards the OR Black Box and its implications. One question about litigation relates to operative data capture that uses reverse scoring. SAQ and DPC items were scored from 1 to 5 and then reverse scored for negative questions.15 20 21
Setting and participants
Copenhagen University Hospital–Rigshospitalet, Denmark, a quaternary care facility with over 2500 operations yearly. All healthcare professionals: OR/anaesthesia nurses (n=69), anaesthesiologists (n=15), obstetricians/gynaecologists (n=42), obstetrics/gynaecology and anaesthesiology residents (n=19); a total of 145 employed in the gynaecology and anaesthesiology OR, Copenhagen University Hospital–Rigshospitalet were invited to participate. Patients were not involved in this study.
Data analysis and statistics
Descriptive statistics were performed for baseline demographics using median and IQR. Means and proportions are reported with 95% CIs (exact CI for proportions). The association between categorical variables was explored using the Fisher’s test.
Cronbach’s alpha (α) was used to determine if there are non-random associations between two categorical variables (reliability) of the items for each section of the survey as well as for the sum scores of the survey. A score of 0.7 was defined as adequate internal consistency.
Analysis of variance was used to compare the mean of sum scores between healthcare professionals. Due to age differences between the healthcare professionals, adjusted analyses were also performed (analysis of covariance).
Analyses were performed using R V.4.1.1 (R Project for Statistical Computing, Vienna, Austria, www.r-project.org). P values below 0.05 were considered significant.
Results
Of the 145 healthcare professionals invited to participate, 124 responded (response rate: 86%): n=100 completed the entire survey (69%), n=9 partially (6%). Fifteen declined through the survey platform (10%) distributed on nurses=8, anaesthesiologists=1, obstetricians and gynaecologists=6 and residents=0. The non-response rate was n=21 (14%), distributed on nurses=12, anaesthesiologists=3, obstetricians and gynaecologists=3 and residents=3. Partially completed surveys were excluded. This resulted in 100 fully completed survey answers for analysis divided between four different groups: n=38 OR/anaesthesia nurses, n=10 anaesthesiologists, n=36 obstetricians/gynaecologists and n=16 residents. Table 1 provides an overview of the demographics.
Demographics of participants
Internal validity
Cronbach’s alpha, used to measure internal validity, was performed for each questionnaire: SAQ (α=0.70, 95% CI 0.56 to 0.77), CIPS (α=0.93, 95% CI 0.90 to 0.95) and DPC (α=0.82, 95% CI 0.74 to 0.87).
Safety Attitudes Questionnaire
Table 2 shows the percentages on SAQ for four different groups of healthcare professionals. Positive safety attitudes were defined as a subscale score of >75%, which corresponds to the Likert scale responses ‘agree’ and ‘strongly agree’. Positive attitude between 1%–60% indicates that action should be taken; between 61%–80% indicates that intervention should be considered; andif a high positive attitude is found, between 81%–100% the current level should be maintained
Mean scores with 95% CI for the healthcare professionals who completed the entire survey (n=100) for Safety Attitudes Questionnaire and the percentage of respondents with positive safety attitudes
Clance Impostor Phenomenon Scale
The lowest ranked item was domain 2 (safety climate), which was below 42% for all groups. The highest rated item was domain 3 (job satisfaction), which ranged from 75% to 100%. The item with the most discrepancy according to profession was domain 6 (working conditions), with 90% positive in anaesthesiologists and only 50% for nurses and 56% for residents. For domain 5 (perceptions of unit management), less than 60% were positive for all healthcare professionals, except for residents at 75%. There were no differences in SAQ responses when adjusting for age (p=0.081)
Moderate to frequent impostor phenomenon was experienced by 71 out of 100 (71%) participants (table 3). Impostor phenomenon occurred but decreased with age (p=0.039). There was a significant difference (p=0.03) in the mean scores of the four groups of healthcare professionals, predominantly higher scores among residents. However, we found no association between imposter phenomenon and groups of healthcare professionals when signs of impostor phenomenon were divided into few, moderate and intense (p=0.15). Few (5%) reported experiencing impostor phenomenon as intense and, at the other end of the scale, 21% reported experiencing few signs. We did not find any correlation in CIPS scores, neither positive nor negative, regarding implementation of the OR Black Box.
Mean and 95% CI for Clance Impostor Phenomenon Scale score for each healthcare professional group
Dispositional Privacy Concern
The overall mean for 100 respondents was 3.1 (95% CI 3.03 to 3.27), with significant difference between the four groups (p<0.001) (table 4). There were no differences in DPC responses when adjusting for age (p=0.081). Of all healthcare professionals, 35% were totally unconcerned about information sharing, 47% expressing little concern, 7% concern, 6% heavy concern and 5% extremely concerned.
Mean and 95% CI for Clance Impostor Phenomenon Scale score for each healthcare professional group
Beliefs about and opinions towards the OR Black Box
Eight (8%) of the 100 respondents reported that they had not previously heard about the OR Black Box. Table 5 shows the opinions the four groups had on the set of 12 questions designed by the team on the OR Black Box. In general, the respondents showed favourable opinions towards the OR Black Box, with scores above 3. Five out of 100 respondents scored high (4 or 5 on the Likert scale) in the question of favourable opinions of the OR Black Box and we could therefore not find an association between favourable opinions towards the OR Black Box and a high or low DPC score.
Participants’ opinions of the OR Black Box represented by mean responses and SD: 3 is a neutral score, <3 negative and >3 positive
Discussion
This study describes the perceptions towards patient safety and implementation of an OR Black Box based on a single-site cross-sectional survey of 100 healthcare professionals from anaesthesia, OR nursing and gynaecological surgeons. The impetus for this study was the implementation of a comprehensive data capture system, called the OR Black Box,6 at our institution. Prior to its introduction, we conducted an extensive survey on safety attitudes, impostor phenomenon, privacy concerns and knowledge and attitudes towards the OR Black Box to gain knowledge about facilitators and barriers. Patient safety is always a top priority and transparency in the healthcare system is rapidly being embraced, including live recording initiatives.24–26 Prospectively capturing all activities in the OR can help ensure unbiased performance evaluation and the assessment of trends and patterns during surgery. The idea of being evaluated while working, however, can raise concerns among healthcare professionals. A successful implementation plan includes understanding and identifying these factors since acceptance of and willingness to use health information technology are major determinants of the success. Implementation of this kind of technology in the OR is still an emerging research area and universal guidelines are limited.
The overall response rate was high compared with other surveys involving healthcare professionals,27 notably for the physician group. The survey comprised four different focus areas, which we will discuss separately. We also compare our findings with our team’s international collaborators from Toronto, Canada, who conducted an identical survey.19
Safety Attitudes Questionnaire
Internationally, SAQ is a validated instrument frequently used in various hospital settings20 21 28 29 to meet increasing demands for a strong clinical safety climate (often referred to as safety culture). Extracting exact data from respondents with different professional backgrounds can be difficult, but the knowledge obtained on the attitudes of individuals within the organisation can provide insight into how systems function and how they can be improved. Moreover, safety culture is considered fundamental to the delivery of safe care, and healthcare organisations must maintain a commitment to safety on all levels to achieve consistently safe operations.30
SAQ consists of six domains but we especially focused on two: safety climate and teamwork climate, since they are most often affected by interventions.31 This makes them indicative of the current status prior to implementation of a comprehensive data capture system like the OR Black Box. These two domains reflect how healthcare professionals experience their collaboration with coworkers, general risk management and clinical patient safety, with safety climate most often related to clinical outcomes.32 In our study, safety climate was perceived as positive by less than half of the participants (39%). Previous studies suggest a minimum per cent-positive threshold of 60%, and an ideal score between 80% and 100%. If the lower threshold is not met, an intervention is recommended.33 The link established between patient safety culture and patient outcomes provides the underlying basis for implementing safety initiatives like the OR Black Box. This result varies only slightly in the subgroups; however, experienced healthcare professionals perceived safety climate as positive more than inexperienced ones.
Teamwork climate embraces the perceptions of hospital healthcare professionals about their collaboration within a specific clinical area to provide safe care for patients.28 The operation room is a very complex setting and there is a growing literature on the critical relationship between teamwork.34 At 60%, nurses rated this domain lower than all the other groups did. We put OR and anaesthesia nurses into one group but are aware that the teamwork climate can be perceived differently depending on the nurses’ area of expertise. Anaesthesiologists were the most positive (80%), which could reflect the fact that teamwork is an important part of their training. Lingard et al mention that a key reason for miscommunication in the operation room arises from the power relationships that exist in healthcare as a result of different professional groups with traditionally different status.34 Understanding the associative path between team communication practices, system processes and health outcomes is a critical goal, and it is our belief that the OR Black Box can assist this complex task.
Impostor phenomenon
CIPS is a well-validated, widely used instrument for examining impostor phenomenon,17 35 but there are two different ways to interpret CIPS scores. They can be divided into: no (≤40), mild (41–60), moderate (61–80) and severe impostorism (>80), or using a score of 62 to distinguish impostors from non-impostors is recommended.36 Because the approaches differ, the prevalence of the impostor phenomenon occurs in 22%–60% of healthcare professionals.35 Moreover, this may be why the impostor phenomenon might be subject to publication bias, that is, the tendency of journals to publish studies with positive rather than negative findings.36 Little is known about how to deal with the inability to internalise success and the presence of persistent self-doubt. Due to the impostor phenomenon’s association with increased rates of burnout and less career planning and the motivation to lead, it represents an important area for further exploration.37
In our study, CIPS showed that over three-quarters of the healthcare professionals reported experiencing impostor phenomenon traits; however, if the line is drawn at a score of 62, then the impostor phenomenon was absent in all four groups, though, with a mean score of 61, residents came close. This aligns with our finding that there was a significant drop in impostor phenomenon with age, which could reflect the fact that greater experience leads to greater self-confidence. A recent review on impostor phenomenon, however, indicated that results diverge in terms of the age effect.36
Educational initiatives such as mentorships and support programmes that counteract a punitive culture are necessary as a non-punitive culture has been shown to alleviate impostor phenomenon.38 39 Data capture systems like the OR Black Box can add positively to this culture by providing anonymised objective feedback to individuals and teams about their strengths and positive behaviours in the OR and by helping to create a culture where mistakes are not interpreted as failure. We are also aware that discomfort among staff might occur when a data capture device is used to evaluate performance.
Digital Privacy Concern (DPC)
DPC explores the desire of the individual for privacy and the concerns they have about the privacy behaviours of organisations and governments.15 In our study, 80% of respondents were not concerned or minimally concerned with digital information sharing. There was no association between increase in age and discomfort; however, there was a significant difference between the four groups (anaesthesiologists, obstetricians/gynaecologists, nurses and residents), with residents having the highest score, indicating that they are the most comfortable with digital information sharing. van Dalen et al12 examined the legal aspect of using audiovisual recordings and concluded that they generally support the surgeon’s case.
Although not previously used in healthcare research, the DPC scale was chosen since it assesses the inherent privacy concerns of individuals across contexts, which means it delves deeply into the topic of data capture in the OR, a situation that may easily lead to feeling a loss of privacy. We encourage further research in this area.
International comparison
An identical survey was conducted at St Michael’s Hospital in Toronto, Canada as part of our research collaboration and with the intention to explore heterogeneity—or the opposite—to create an understanding of potential cultural and national differences when implementing a comprehensive data capture device such as the OR Black Box.19
Demographics were relatable; however, the Canadian study consisted of only 43 and the gynaecologists (9) spent on average 6 days/month in the OR compared with the Danish cohort where the gynaecologists/obstetricians spent 3, somewhat contaminated by the fact that obstetricians only perform planned caesarean sections once or twice a month.
Regarding SAQ, we found some differences between the two groups of respondents. Overall, five out of six SAQ domains were less than 60% positive in the Canadian group compared with only two in the Danish cohort. Globally, the Nordic countries have a high level of social trust, with patient safety culture being a subset of this.40 As a result, social trust may represent an underlying factor and provide some explanation for the differences in our findings.
In terms of impostor phenomenon, our findings were almost identical to the Canadian ones; no statistical differences were found (data not shown). This is plausible since the phenomenon, which is considered a personality trait, is found internationally and across a range of professions.36 SAQ, in contrast, is more highly linked to working conditions and not personality traits.
Unlike our Canadian colleagues, we did not find any concerns about data safety. The European General Data Protection Regulation, which is the toughest privacy and security law in the world, was implemented in May 2018 and may have positively affected the perspectives of healthcare professionals on data security, signalling a stronger belief in data privacy and security. Medical litigation is per se not widespread in Scandinavia compared with North America. Half of the Canadians expressed concerns about an increase in the number of lawsuits resulting from implementation of a comprehensive data capture system in the OR. This was not the case in the Danish participants; however, it is a highly important matter that any institution considering the implementation of audiovisual recordings should address. In a similar study from Canada, Etherington et al11 found that surgeons, and not nurses, mainly believed that audiovisual devices like the OR Black Box may result in punitive actions and medicolegal issues that negatively affect healthcare professionals. To date, however, there has been no test of discoverability of operative videos in court, and all OR Black Box raw captures are deleted at 30 days from the time of surgery. These differences emphasise the importance of determining perceptions across institutions and countries to explore cultural variations.
Strengths and limitations
One of the strengths of this study is that all the various types of staff were invited to participate and that a high response rate was achieved in all groups. Another strength is that three validated questionnaires were used in our survey. This study also has some limitations. Despite the high response rate, the results may not be generalisable due to sample size and being a single-centre study. The large number of questions in our survey may be one reason why 6% only partially completed the survey and 10% declined. The group comprising nurses may also have had a lower response rate since they rarely had administrative time to complete the survey. We conducted the study over a 5-month period, which may have caused recall bias. Non-responders may be the most unfavourably disposed towards the use of the OR Black Box, which could misrepresent the data in terms of non-response bias.
A postintervention survey would be ideal for remeasuring attitudes towards the OR Black Box since its implementation. The aim is to conduct this study once more OR Black Boxes are put into use. Surveys must be seen as a snapshot that can change over time, just as they only measure what is asked about, which means that some of the personal perceptions of the healthcare professionals in this study may not be described. A next step would be to perform a qualitative study to achieve a deeper understanding of the views of healthcare professionals working in an OR where their work is recorded and analysed.
General reflections
To ensure the best possible safety culture and the delivery of safe care, new standards for objective, real-time assessment tools like the OR Black Box are needed. Implementing a comprehensive data capture platform requires a cultural shift, therefore a well-designed socialisation process is of great importance. This study and the comparisons made with the Canadian study provide insights into existing enablers and can be used to ensure a successful implementation. Our findings will be used to draw up an engagement plan that takes safety climate awareness, occurrence of impostor phenomenon and supporting data sharing issues into consideration.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the Danish Data Protection Agency (file number: VD-2019-65). After reviewing the study in September 2019, the Danish National Committee on Health Research Ethics concluded that specific ethical approval was unnecessary because the study did not contain biological material and would not influence patient treatment. The survey was voluntary, and data were obtained pseudoanonymised and reported fully anonymised. Participants gave informed consent to participate in the study before taking part.
References
Footnotes
Twitter @jeanettstrandb1
Contributors JS, JLS, ES and LG conceived the idea for this study. JS and ND contributed to practical and logistical aspects of the study creating a database and retrieval of data. SR and JS performed the statistical analysis. JS wrote the draft manuscript and is responsible for the overall content as the guarantor, all authors contributed to the final manuscript. JLS, BO and TG (all professors) provided profound academic supervision.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests TG is the founder and Director of Surgical Safety Technologies and equity holder of Surgical Safety Technologies, Inc., Toronto, Canada.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.