Article Text
PDF
Abstract
Objectives This study aimed to investigate staff’s perceptions of patient safety culture (PSC) in two Danish hospitals before and after an in situ simulation intervention.
Design A repeated cross-sectional intervention study.
Setting Two Danish hospitals. Hospital 1 performs emergency functions, whereas hospital 2 performs elective functions.
Participants A total of 967 healthcare professionals were invited to participate in this study. 516 were employed in hospital 1 and 451 in hospital 2. Of these, 39 were trained as simulation instructors.
Intervention A 4-day simulation instructor course was applied. Emphasis was put on team training, communication and leadership. After the course, instructors performed simulation in the hospital environment. No systematic simulation was performed prior to the intervention.
Main outcome measures The Safety Attitude Questionnaire investigating PSC was applied prior to the intervention and again 4 and 8 weeks after intervention. The proportion of participants with a positive attitude and mean scale scores were measured as main outcomes.
Results The response rate varied from 63.6% to 72.0% across surveys and hospitals. Baseline scores were generally lower for hospital 1. The proportion of staff with positive attitudes in hospital 1 improved by ≥5% in five of six safety culture dimensions, whereas only two dimensions improved by ≥5% in hospital 2. The mean scale scores improved significantly in five of six safety culture dimensions in hospital 1, while only one dimension improved significantly in hospital 2.
Conclusions Safety attitude outcomes indicate an improvement in PSC from before to after the in situ simulation intervention period. However, it is possible that an effect is more profound in an acute care hospital versus an elective setting.
- safety culture
- patient safety
- medical education
- team training
- simulation
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction and objective
Improving patient safety is known as one of the biggest challenges in healthcare today.1–3 Numerous approaches to improve patient safety have been initiated and creating a culture of safety is considered a key approach.2 4 5
A culture of safety can be defined as an integrated pattern of individual and organisational behaviour, based on shared beliefs and values that continuously seek to minimise patient harm.3 6 Patient safety culture (PSC) captures a broad and rather steady phenomenon, encompassing the underlying values that shape behavioural patterns and processes related to the safety of the patients in a particular setting. PSC is a multidimensional and multilevel construct,7 which is a deeply rooted aspect of the safety climate. Thus, climate is considered the measurable and improvable aspect of culture.8 9
A growing evidence documents an association between safety culture and patient outcomes, such as reduced readmissions, length of stay and medication errors.8 10 A positive development in PSC and safety outcomes is in the interest of the patients. Further, a review examining the relationship between organisational and workplace cultures and patient outcomes (n=60 studies) found that over 70% of studies reported exclusively positive associations (48.4%), or a mixture of positive associations and no associations (25.8%), between culture and patient outcomes.8
A widely used questionnaire measuring PSC within hospital care is the Safety Attitudes Questionnaire (SAQ).11 12 It measures the proportion of participants with a positive attitude towards multidimensional aspects of patient safety. The dimensions include teamwork climate, safety climate, job satisfaction, stress recognition, working conditions and leadership support for patient safety.13 A positive attitude is defined for each individual by a mean scale score of at least 75 of 100 possible points.13–15 In studies where PSC has been observed over time, a 5 per cent point of improvement has been considered clinically relevant.14 16 17
Slightly different methods for observing change in PSC at the group level have been applied. When conducting intervention studies, PSC is usually measured and compared at two time points.
However, this may contribute to potential bias, since staff members constitute a dynamic group of people. A proportion of the employees will resign their positions and leave the unit, while new staff members will be assigned. A previous Danish study has shown that safety attitudes of newcomers, stable staff and staff leaving the unit differ. Thus, it is ideal to measure PSC in the group of stable staff members, since this allows an insight into the attitudes of the culture bearers alias stable group of staff exposed to the intervention over time.14
The strongest evidence for improvement of the PSC has been found in studies consisting of multifaceted interventions, supported by the leadership and aimed at enhancing communication, coordination and teamwork in safe clinical practices.3 6 14 These are all qualities naturally build into simulation training.
Simulation is a learning strategy that imitates a scenario in a clinical setting. A particular set of clinical conditions are created or replicated to mirror authentic situations.18 In situ simulation occurs in the actual clinical environment as part of the health professionals’ daily life.19 In situ simulation allows teams to review and reinforce their skills and to problem-solve in a simulated clinical environment. At its core, it provides an approach to support and develop teamwork, leadership and communication competencies.19 20 In situ simulation has been described as an important strategy in identifying and overcoming threats to patient safety, although it is a limited researched area.21 Simulation-based training programmes have been shown to be positively associated with clinical risk management practices mirrored in the safety climate.6 10 In a Danish pre-post simulation intervention study across two settings, PSC did not improve over time in either setting.22 The authors suggested that future research should focus on continuous simulation sessions and longer follow-up periods.23
Generally, knowledge about the change in PSC associated with in situ simulation is sparse, and more research is needed.22 Therefore, the present study aimed to investigate changes in PSC from before to after a time period with in situ simulation at two affiliated hospital sites. Our particular interest was to observe changes in dimensional PSC over time in a hospital setting characterised by acute care functions and a hospital characterised by elective functions, while also looking at stable groups of staff who reported the PSC both before and after intervention. In this regard, the research questions were as follows:
Do the proportions of healthcare staff with positive attitudes towards six PSC dimensions improve by more than 5% from before to after the in situ intervention?
Do the mean scale scores of the six PSC dimensions improve from before to after the in situ intervention?
Methods
Study design
A repeated cross-sectional experimental study design was applied across two affiliated hospitals. An in situ intervention programme was initiated after collecting the first assessment of the PSC and lasted until the end of the study period. Two further PSC surveys were carried out during the intervention period at 4 and 8 months, respectively, after first assessment.
Variables of interest: measurement of PSC and simulation sessions
To measure participants’ perceptions of PSC, a validated Danish version of the Safety Attitudes Questionnaire (SAQ-DK) was used.24 The psychometric properties of SAQ-DK were tested in a cross-sectional study including 925 multiprofessional healthcare workers. The test showed good construct validity and reliability, and confirmed a hypothesised six-factor model.13SAQ-DK comprises 32 items covering six dimensions of PSC, namely, teamwork climate, safety climate, job satisfaction, stress recognition, perception of management and working conditions. Answers are given on a 5-point Likert scale (1=Disagree strongly, 2=Disagree slightly, 3=Neutral, 4=Agree slightly, and 5=Agree strongly). Further, demographic background information on profession, gender, age group, organisational role, affiliation and work experience was collected.13 24
The in situ simulation sessions were documented using a standardised log note. The following information was logged: date, hospital, specialty, number of participants, time spent including briefing and debriefing, and the learning objectives of the session. It was also noted if the session was mock-up or planned.
Setting
The study took place from April 2017 to December 2018 across two hospital sites belonging to the same regional somatic hospital in the Region of Southern Denmark. Hospital 1 had a capacity of 333 beds, while the capacity of hospital 2 was 220 beds.
Participating staff from hospital 1 belonged to 11 different groups mainly handling acute care. The groups formed naturally around their common daily clinical tasks and likewise in the situ simulation setting, which can be seen further in online supplemental appendix 1. Likewise, 12 groups formed naturally in hospital 2, where mostly elective functions were carried out. In neither of the two hospitals, in situ simulation was facilitated systematically prior to the study.
Supplemental material
Material and data collection
Inclusion criteria in the first assessment of the PSC consisted of full-time or part-time clinical staff with patient contact. Only nurses, doctors, midwives or radiographers were included. Staff members assigned after baseline measurement were not invited to participate in the second and the third data collection of SAQ-DK as we mainly took an interest in the attitudes of the group of staff which participated throughout the study period. Each participant was assigned a unique personal identifier that remained across the three SAQ-DK surveys. In total, 967 (516 from hospital 1 and 451 from hospital 2) full-time and part-time staff with patient contact qualified for inclusion in the first assessment of the PSC.
Collection of SAQ-DK data was staggered across the two hospital sites with three intervention periods and three data collection periods (figure 1). SAQ-DK data were collected from April 2017 onwards in hospital 1 and from September onwards in hospital 2. In situ simulation started with the first SAQ-data collection (first survey), which was pretraining of the instructors. Further SAQ-data (second and third survey) were collected approximately 4 and 8 months after the intervention was applied.
Study timeline Safety Attitudes Questionnaire (SAQ)—data collection and intervention periods across the two hospital sites. 1Data were collected on 25 April–19 May 2017. 2Data were collected on 23 August–14 September 2017. 3Data were collected on 10 January–5 February 2018. 4Data were collected on 8 May–12 June 2018.
SAQ-DK was distributed via a unique link in emails to all included staff. The answers were not anonymous; however, only the person in charge of distribution of questionnaires had access to the individual responses. Reminders were emailed after 2 weeks to all staff that had not answered. To boost participation, nudging through contact was performed by the research group visiting departments taking part in the study. Furthermore, weekly emails to management and key persons with current response rates and collective appreciation at, for example, morning conferences were conducted. To account for vacation periods and public holidays, some data collection periods were slightly longer than others.
The in situ simulation intervention
In total, 21 staff members from hospital 1 and 18 staff members from hospital 2 were educated as simulation instructors. The hospital management selected the instructors with no set selection criteria at the individual level, apart from motivation. A minimum of one staff member from each of the 23 groups listed in online supplemental appendix 1 was trained as a simulation instructor. All instructors participated in a 4-day course.
The instructors’ training course emphasised non-technical skills focusing on team training, communication and leadership, of which all instructors had to complete a curriculum.23 Both the curriculum and a need analysis is illustrated in online supplemental appendix 2.
Supplemental material
Subsequently, the instructors performed in situ simulation in their respective groups/units, aiming to improve handling of specific clinical situations and thus the safety and quality of care provided. The sessions were initiated, planned, led and documented in log notes by the simulation instructors. The instructors started the in situ simulation the day after ending their training, and performed simulation prospectively.
Three staff members from MidtSim (the regional centre of simulation training in Central Denmark Region Denmark) facilitated the training of the instructors at hospitals 1 and 2.
Patient and public involvement
No patient or public involvement.
Ethics
The study was registered with the Danish Data Protection Agency (Journal number 17/1942) and was approved by the hospital management. Survey invitees were informed that participation was voluntary and that all answers would be treated with confidentiality. Survey IDs were used at the individual level across data collection times. Participants were granted anonymity in the publication of results at the clinical level and scientifically.
Statistical analyses
Data in this study are reported for all participants and likewise separately for responders replying to all three SAQ-DK surveys (the ‘stable group’). To be included in the analysis, participants had to reply to a minimum of one item in each dimension.
SAQ-DK data were reported as (1) the percent of respondents with a positive attitude (% positive, defined by an individual mean scale scores ≥75), and (2) mean scale scores and SD. Both PSC outcomes range from 0 to 100. Between surveys, a change in % positive describes the change in the proportion of staff with positive attitudes, whereas a change in the mean scale score describes the change in the perceived level of PSC among surveyed staff. In accordance with earlier studies, an improvement of minimum 5% over time was targeted and regarded as clinically relevant.14 16 17
For the purpose of calculating the two PSC outcomes, individual SAQ-DK item scores were converted to a 0–100 points scale, where 1=0, 2=25, 3=50, 4=75 and 5=100. Items 2 and 11 were reverse scored so that their valence matched the positively worded items. Individual mean scale scores were calculated by the average score of the scaled items.14 SAQ-DK mean scale scores were calculated for each dimension by the average score of the scaled items.
To answer our research questions, SAQ-DK outcomes were reported for hospital 1, characterised by acute care functions, and hospital 2, characterised by elective functions.
Percent positive was calculated and compared across data collection periods and subgroups (hospital 1 and hospital 2) by using logistic regression. McNemar’s tests were used to compare % positive across survey times for the stable group. Mean scale scores were compared using independent samples t-tests for subgroup analysis and paired sample t-tests for the stable group.
All analyses were performed using SAS V.9.4.
Results
Invitees and participants in the SAQ-DK surveys
The total number of invitees, participants and included staff in the three SAQ-DK surveys is listed in table 1. Across the collection of the six surveys at the two hospitals, the response rate varied from 63.6% to 72.0% (table 1). Additionally, a total of (36.6% of the invited) included participants from hospital 1 responded to all three surveys and were characterised as the stable group. In parallel, 144 (31.9% of the invited) included participants from hospital 2 were characterised as the stable group. Participants in the two hospitals were similar across sex and age groups.
Participant characteristics across the three SAQ-DK surveys for hospitals 1 and 2
In situ simulations performed
Characteristics of the simulation sessions performed are illustrated by hospital and study period in table 2. During the first and the second intervention period, 54 simulation sessions were performed in hospital 1, whereas 61 sessions were performed in hospital 2. After the second intervention period, the in situ simulation continued.
Characteristics of the in situ simulations performed during the first and the second study period at the two hospitals
SAQ-DK results over time for hospital 1 and hospital 2
PSC was surveyed three times at each hospital during the study period (six SAQ surveys). Across the six surveys, SAQ-DK scores between 1 and 5 were observed for all items, and the rate of not applicable answers at the item level ranged from 1.6% to 5.0%.
To answer research question 1, the proportion of staff with a positive perception of the PSC (% positive) is listed per hospital, survey and PSC dimension in table 3. The improvements ranged from 7.0 per cent points for job satisfaction to 13.6 per cent points for perception of management. In hospital 1, which handles acute care functions, % positive (the proportion of staff with positive attitudes) improved by more than the anticipated 5 percent point in all dimensions except for stress recognition, where a negative change was observed. Apart from job satisfaction, all improvements were statistically significant. For hospital 2, which mainly handled elective care, only the safety climate improved statistically significantly between the first and the third survey, where the improvement was >5 per cent point.
Proportions of participants with positive attitudes (% positive) per SAQ-DK dimension in the three surveys and the two hospitals
Likewise, improvements >5 per cent point for the stable group was found at hospital 1 in all dimensions, except for stress recognition. Apart from teamwork climate and job satisfaction, all improvements were statistically significant (p<0.05).
Regarding mean scale scores included in research question 2, table 4 illustrates statistically significant improvements (p<0.05) for hospital 1 in all dimensions, except for stress recognition. The improvements were observed between the first and the third PSC survey, as well as for the stable group. Such a pattern was not observed for hospital 2.
Mean scale results for SAQ-DK in the three surveys and the two hospitals
Discussion
This is the first in situ intervention study within Danish healthcare to report before and after intervention measures of PSC for acute versus elective care. Furthermore, the study is focusing on the perceptions of the stable group of staff who have reported the PSC both before and after an in situ intervention.
The main findings document an improvement in PSC in hospital 1. Both groups, consisting of ‘all participants’ and the ‘stable group’, improved >5 per cent point in five of six dimensions after 8 months. In both groups, the mean scales scores improved significantly in five of the six dimensions. For hospital 2, in which the staff is mainly handling elective tasks, an improvement >5 per cent point was observed in two of six dimensions for ‘all participants’ and in none of the six dimensions in the ‘stable group’. Furthermore, the mean scale scores improved significantly in one of six dimensions for all participants, whereas no dimensions improved in the stable group.
Strength and weaknesses of the study
Across hospitals and measurements, the response rate varied from 63.6% to 72.0%, which is considered to be a rather high response rate, compared with other studies investigating change in PSC.3 6 Furthermore, both the rate of non-applicable answers and the internal instrument reliability were favourable.13 24 The study design can be considered a strength, as the repeated measurements during an 8-month period allow for observations of change in PSC over time. Moreover, the fairly high number of invited participants minimises the risk of type I and II errors.25 This study used unique personal identifiers, which enabled analyses of PSC in the stable groups. An intense 4-day training course of the instructors created the basis for an effective and focused intervention, based on the same values, skills and pedagogical aims. Content and learning objectives in the course targeting the participants’ needs were based on a need analysis (online supplemental appendix 2).
Regarding the study design, there is a lack of a control group, which increases a possibility of unknown simultaneous initiatives affecting PSC during the 8 months’ observation period. To our knowledge, no other major initiatives were implemented during the intervention period.
Registration of participation in the in situ simulations using the individual study ID would have opened for analysis of SAQ results for participants exposed to the intervention versus non-exposed participants. Instead, we relied on the spread of the effect in everyday work. The SAQ-DK was used to measure participants’ self-reported perceptions of PSC. The measurements could have caused bias in terms of social desirability bias and information bias, which could influence the results in either directions and possibly even counter act. Other studies have found a positive change in PSC over time without an intervention, which suggests that administering a measurement alone might have an impact on PSC.26 Likewise, participants could be influenced by a sense of being observed, also called the Hawthorne effect. This could potentially lead to a type II error. However, we consider the Hawthorne effect insignificant in our study; this evaluation is based on previous findings among American nurses and doctors in a questionnaire survey.27 Furthermore, a previous Danish study found that leaders, instructors and simulation participants experience the in situ simulation as relevant and profitable.28 These findings support the improvement we found in the PSC. One could speculate to minimise the potential Hawthorne effect, through anonymous responses, however this approach does not enable the strong design, we have applied using personal identifiers and investigating PSC pre and post intervention among stable staff. We consider this approach imperative, and the Hawthorne effect minimal.
Based on the potential other bias mentioned, it is difficult to conclude in which direction the PSC results may have been influenced. Considering the good compliance and comparable participants’ characteristics across hospital 1 and hospital 2 though, potential biases are expected to influence the results in the same way in the two hospitals and would therefore only affect comparable results to a limited extent.
Given the potential bias, the results can be transferred to other settings with caution. Complex social organisational elements such as management support, extent of implementation of the intervention and the staff facilitating the intervention should be accounted for in terms of generalisability.6
Comparison to other studies
Numerous strategies have been conducted aiming to explore factors influencing PSC.3 6 Overall, these studies were characterised by poor study design and biases, including low response rate, incomplete reporting and non-significant findings. Reviews concluded limited evidence documenting change in PSC throughout interventions.3 6 Compared with these studies, our study contains a large number of invitees, a high response rate and several significant findings.
Furthermore, it is argued that an efficient strategy to improve PSC is selective and flexible, and is adaptable in the very context and climate in which they are implemented.29 30 The simulation instructor programme implemented in the current study may be characterised as such a strategy, since in situ simulation is selectively targeting healthcare professionals simulating in the actual clinical context (local culture). Moreover, in situ simulation is flexible in the sense that both technical and non-technical skills are trained, while also evaluating organisational competences as well as identifying latent conditions that predispose to medical errors.21 Additionally, Weaver et al outlines criteria for an effective strategy improving PSC and states that best evidence includes team training and mechanisms to support communication.3 The 4-day in situ simulation instructor course emphasised team training and communication, which are objectives consistent with the main purpose of simulation.19–21 Thus, the in situ simulation intervention implemented is characterised as applicable and feasible in this setting.
PSC is a local phenomenon closely linked to everyday practices in patient safety, which is regulated at the local, regional and national level. To our knowledge, only one prior Danish study has investigated in situ simulation in relation to PSC over time.22 Sørensen et al examined PSC in two settings; an in situ simulation setting versus an offsite simulation setting, although no significant improvements were found in either setting. However, this study differs from ours. First, participants replied to the SAQ 1 month before and 1 month after participating in the in situ simulation programme, hence PSC was measured over a significantly shorter period, when compared with our study. It is mentioned that degradation of non-used skills appears after 3–18 months,22 which calls for the longer period of measurements that was applied in our study. We see an improvement perspective in gathering the instructors after each intervention period and sharing experiences aiming for mutual inspiration, alignment and an enhanced impact of the intervention. Developing and improving the culture in an organisation have proven to be challenging, since knowledge of the influence of initiatives over time is limited.6 No other studies have investigated if in situ simulation improves PSC over a longer period of time. Therefore, our results contribute to new and needed knowledge. Second, instructors in our study continued to facilitate simulation sessions in their daily setting. This suggests that to improve PSC, initiatives need to be implemented during a longer period of time for the effect to fully merge into daily practice.
Generally, both % positive and mean scale scores improved in hospital 1 but not in hospital 2. However, hospital 1 had a lower score before the intervention was applied. These findings suggest that hospitals and departments handling more acute tasks may be characterised by a lower PSC score, which enables the possibility for a greater improvement. A previous study found that hospital units with <60% positive responders have the most to gain in PSC from efficient initiatives.31 These results call for future interventions and strategies focusing on staff handling more acute tasks.
Study implications
Conducting in situ simulation seems to have a significant impact on PSC in the clinical encounter. Results may have an important impact on future research and interventions. When applying similar in situ simulation interventions in other settings, policy makers should consider attendees, since staff handling more acute functions seems to gain most regarding PSC.
Conclusions
In this repeated cross-sectional intervention study, PSC was observed over time in an acute care and an elective care setting, in which in situ simulation was facilitated during the observation period. Results indicate that in situ simulation can be used as a tool to improve PSC over time, particularly in the acute setting.
Acknowledgments
The authors would like to acknowledge the study participants for their effort, Head of Departments and the Hospital management for their support and the employees at Clinical Simulation Department, among others including Pernille Ibsgaard and Pernille Lennert at Lillebaelt Hospital for their engagement in the project.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors All authors were part of the research team. CP, SK and HIJ contributed substantially to the conception and design of the study. AS, GK-A and HIJ participated in data collection. SK and AS drafted the paper. KBC, SK and AS performed the data analysis. All authors critically revised the paper for important intellectual content and approved the final version prior to publishing.
Funding This work was supported by Lillebaelt Hospital, Region of Southern Denmark and Corporate HR, MidtSim, Central Region Denmark.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study was approved by the hospital management. Registration number from the Danish Data Protection Agency: Journal number 17/1942.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data available on request from the authors.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.