Original research•

Evaluation of a new patient safety educational programme to reduce adverse events by encouraging staff to speak up: application of the trigger tool methodology

Abstract

Background Poor communication contributes to adverse events (AEs). In our hospital, following an experience of a fatal incident in 2014, we developed an educational programme aimed at improving communication for better teamwork that led to a reduction in AEs.

Methods We developed and implemented an intervention bundle comprising external investigation committee reviews, the establishment of a working group (WG), standards and emergency response guidelines, as well as educational programmes and tools. To determine the effectiveness of the educational programmes, we measured communication abilities among doctors and nurses by administering psychological scales focused on their confidence in speaking up. Furthermore, we applied the trigger tool methodology in a retrospective study to determine if our interventions had reduced AEs.

Results The nurses’ scores for ‘perceived barriers to speaking up’ and ‘negative attitude toward voicing opinions in the healthcare team’ decreased significantly after the training from 3.20 to 3.00 and from 2.47 to 2.29 points, respectively. The junior doctors’ scores for the same items also decreased significantly after the training from 3.34 to 2.51 and from 2.42 to 2.11 points, respectively. The number of AEs was 32.1 (median) before the WG, 39.9 (median) before the general training, 22.2 (median) after the general training and 18.4 (median) after implementing the leadership educational programmes. During the intervention period the hospital’s incident reports per employee kept increasing.

Conclusion Our new educational programmes improved junior doctors and nurses’ perceptions of speaking up. We speculated that our intervention may have improved staff communication, which in turn may have led to a reduction in AEs and a sustained increase in incident reports per employee.

What is already known on this topic

  • Adverse events (AEs) are a leading cause of death in hospitals, often due to communication issues such as a failure to speak up. Incidents are reported more frequently in environments where staff feel safe to speak up.

What this study adds

  • Following the educational intervention, nurses and junior doctors perceived fewer barriers to speaking up, and they held more positive attitudes towards voicing opinions in the healthcare team, as measured by a psychological scale. This result suggests that the intervention promoted staff willingness to speak up, which might lead to improved communication and decrease in AEs. Continuous increases in annual incident reports per employee following the interventions support these results.

How this study might affect research, practice or policy

  • Our study suggests that the educational programme improved scores on the speaking-up scale, and therefore improved communication. This improvement led to a reduction in AEs. However, direct observation is necessary to verify the improvement in staff communication skills. These results suggest that the creation of an environment in which the staff feels safe to speak up could ultimately reduce AEs.

Introduction

Communication failures such as incorrect or missing information are a common cause of adverse events (AEs)1 worldwide, including Japan. Communication errors may be due to a lack of training and teamwork, insufficient equipment or procedural design, conflicting organisational goals and defective organisational systems or management decisions.2

Good communication requires voluntary information sharing, including speaking up. In a medical setting, speaking up means alerting team members about unsafe situations and patient safety threats.3 Failure to speak up is a communication error.4 5 Research shows that AEs (eg, response delays) often occur when healthcare providers, who are aware of potential threats to patient safety, fail to speak up.6 7 Thus, improving healthcare professionals’ confidence to speak up can prevent AEs.8

In 2014, our hospital experienced a fatal AE caused by poor communication between the supervisors, junior doctors and nurses responsible for a patient’s care. The junior staff recognised that the patient was experiencing the same symptoms repeatedly and attempted to solve the problem independently without reporting it or asking for support from the senior doctors. Following this incident, we developed an intervention bundle, including a new educational programme for communication training. Speaking-up attitudes were monitored using a psychological scale, prior to conducting the intervention and annually thereafter. Furthermore, the occurrence of AEs was retrospectively calculated using a trigger tool. In this study, we discussed the improvement of speaking-up attitudes in doctors and nurses and its impact on reducing the occurrence of AEs.

Methods

Target hospital

Our hospital is a large medical centre affiliated with a school of medicine in Japan. It has 900–1000 beds and approximately 800 doctors and 1000 nurses out of around 2500 employees. Electronic medical records were implemented in 2007. We developed the interventions in response to a fatal AE in 2014. Subsequently, a department in charge of patient safety was established with seven full-time general patient safety officers.

Details of the triggered incident

As mentioned in the introduction, our hospital began to intervene following a severe AE that occurred in 2014. The incident was none of the staff—including four junior doctors, one junior nurse and a mid-career nurse—reported or communicated the problem to senior doctors or nurses in 15 hours. Although subsequent treatment restored the patient’s heartbeat, the patient suffered hypoxic-ischaemic encephalopathy and died. The outline of the AE is shown in the following timeline of events:

|

The morning after the incident, a ward nurse reported this incident to general patient safety officers. The patient safety committee and the hospital president established a committee to investigate this incident.

Analysis and proposal by the investigation committee

An investigation committee, including experts from other institutions, reviewed the triggered incident and found that the organisation had no system for junior doctors and nurses to promptly report concerns about a patient’s condition or consult with a department’s senior doctors and nurses. The committee interviewed the junior doctors and nurses involved in the incident and found that their inability to consult with senior team members was due to one or more of the following causes: concerns that asking for help would make them appear less professional; prior negative experiences when questioning or consulting senior members; lack of confidence in consulting senior members; lack of knowledge of when, how and what to report; low awareness of the necessity to report patients’ safety issues and consult senior members. The investigation committee concluded that improving the organisational system required the hospital to educate junior doctors and nurses, encouraging them to report patients’ safety issues and consult senior members about them.

Intervention design

Establishment of a working group

In 2015, a working group (WG) was established to develop and implement an intervention bundle based on the analysis and proposals of the investigation committee. The WG comprises 12 people: a doctor, nurses and pharmacists serving as general patient safety officers; critical care doctors; a cardiologist; and nurses in emergency ward. The WG has continued to meet three or four times a year since its inception in 2015.

The WG determined the purpose, goal and schedule of the intervention. The WG also developed the standards, a guideline describing the standards, training methods and tools. Finally, it implemented the intervention bundle. The activities of the WG were reported at departmental safety officers’ meetings and the patient safety management committees of the hospital.

Developing new standard for working effectively as a team

The WG developed new standards in the case of patients’ acute deterioration for the junior and senior members to work effectively as a team. The WG refers to the Care of the Critically Ill Surgical Patients guidelines, which aim to teach the necessary practical skills and knowledge to assess and manage critical situations.9

The WG also referred to TeamSTEPPS10 and incorporated four tools into our standards: ‘SBAR (Situation, Background, Assessment, and Recommendation)’, ‘Check back’, the ‘Two-challenge rule’ and ‘CUS (‘I am Concerned,’ ‘I am Uncomfortable,’ and ‘This is a Safety issue’)’. Originally, TeamSTEPPS was designed to be introduced as a system, but reports indicate that even if only one of its tools is introduced, communication and teamwork skills are improved.11 This was the reason for selective inclusion of four of the TeamSTEPPS tools in the new standard.

Next, we created a flow chart based on the standards: recognising the signs of severe illness and unsafe situations, sharing information and consulting with senior team members or the intensive care department about patients’ prolonged or recurring symptoms. According to the flow chart, doctors and nurses follow the first step criteria to decide whether a patient’s condition requires a prompt response and use the clearly defined second step criteria to seek assistance from senior members. Teams learn the importance of reporting even minor symptoms and the benefit of sharing patients’ information rapidly according to the flow chart.

We posted the flow chart in all the wards and outpatient clinics so that the teams can follow the standards and act swiftly and efficiently.

Developing a new educational programme and tools

In 2015, the WG developed a new educational programme and tools based on our new standards and named it Early Awareness & Rapid Response Training in Hospitals (EARRTH).12 The EARRTH educational programme comprises two types of training: general and practical training. The general training targets all employees, whereas the practical training is an additional programme that targets junior doctors and junior nurses (defined as having 5 years’ experience or less in their roles). In other words, junior doctors and junior nurses, such as those involved in the 2014 incident, received both general and practical training. The general training initially adopted a classroom style, and the practical training was in the form of a role-play workshop. Both training styles communicated EARRTH’s background, philosophy and objectives and explained effective communication and mutual support.

The WG developed an animated movie to explain EARRTH’s philosophy,13 a flow chart, lecture slides and role-play scenarios as educational tools based on past cases of severe AEs in our hospital. In the general training, participants watch an animated movie and are presented with a short lecture. In the practical training, facilitators of the role-play workshop teach junior doctors and nurses how to share patients’ information quickly and request prompt support from senior staff. They use the EARRTH flow chart and role play scenarios in the workshops.

In 2017, we also developed an interactive leadership training programme for departmental safety officers and leaders (ie, managers, directors and executives) comprising an animated movie, applied theatre exercises and group discussions.14

Implementation

We started the general training in June 2016. We also developed an e-learning version of the general training and offered it to staff who missed the in-person training. Since 2017, we have provided this e-learning version to newly employed staff.

Since November 2016, we started the practical training and have held multiple training sessions during the year for junior doctors and nurses, with two to three senior doctors or nurses guiding two to three junior participants. The practical training is implemented efficiently with the cooperation of the departmental safety officers. We did not offer follow-up training after the initial training but since 2019, some doctors who participated in the initial training began participating again as facilitators.

Furthermore, we initiated the annual interactive leadership training programme in September 2017 (online supplemental table 1).

Study and measures

Evaluation of understanding and applicability to clinical settings in general training

We administered questionnaire surveys during the first year (2016) after each general training session. To determine the participants’ level of understanding of the training content we asked them to rate items on a 5-point scale (1: not at all, 5: very well) and calculated the percentage of positive responses (online supplemental table 2). We also asked them to describe their applicability to the clinical settings using a 5-point scale (1: not at all, 5: very much) for questions Q2–Q7 and calculated the percentage of positive responses (online supplemental table 2).

We verified the number of participants in the general training and checked the response rates to ensure the completeness of the questionnaire survey data for analysis.

Measuring speaking-up attitudes

We used a psychological scale, the ‘Japanese version of the Speaking Up Attitude Scale’ developed by Okuyama et al,14 to measure the team’s communication competencies. The following three factors were determined through 11 questions: ‘perceived barriers to speaking up’ (four questions), ‘perceived response to speaking up’ (acceptance of their opinions) (four questions) and ‘negative attitudes toward voicing opinions in the healthcare team’ (three questions).

Since 2016, we conduct this survey annually with doctors and nurses, excluding chiefs, managers and executives. We administered the questionnaires prior to the general training in 2016 and annually since the intervention’s commencement in 2017.

Monitoring and feedback

To monitor awareness of the EARRTH flow chart posted in each ward, we periodically visit the wards and randomly ask staff related questions. We have continued this flow chart awareness survey periodically since 2017. We also conduct a questionnaire survey after the practical training to ask participants about the suitability of the training in terms of length, ease of understanding and usefulness. Based on this feedback, we improve the programme every year.

We report the results of our activities annually at the patient safety committee, including the questionnaire feedback and roster of junior doctors/nurses as participants and senior doctors/nurses as facilitators. We express our gratitude to participants, facilitators and departmental heads for their support. Additionally, we request support from departmental heads to motivate the participation of junior staff.

Analysis

AE detection

To investigate the annual changes in incidences of AEs from 2013 to 2018 retrospectively, we applied the trigger tool methodology. We used the Institute for Healthcare Improvement (IHI) Global Trigger Tool (GTT), which was originally used to monitor changes in AEs prospectively.15 The GTT includes 53 types of AEs, including transfusions, use of blood products and readmission within 30 days. Twenty randomly selected cases were reviewed each month, and among the 53 types, the number of AEs at incident category E which indicate temporary harm to the patient and required intervention15 or higher was counted and converted to a number per 1000 person-days.16–18

We randomly selected patients for the study using Microsoft Excel’s (Redmond, Washington, USA: Microsoft) RANDBETWEEN randomising function on data from 2013 to 2018, with 10 cases in the first half of every 2 months and 10 cases in the second half of each year (table 1). This process yielded 720 cases for our medical records review to calculate the number of AEs per 1000 person-days every 2 months. Because this study was conducted as a retrospective review of 6 years of data (2013–2018) to examine the intervention effects, we modified the original method of the IHI-GTT into the following procedure: the first author, a pharmacist, reviewed the electronic medical records, then the second author, a physician, reviewed and corrected the first author’s results.

Table 1
|
Characteristics of all inpatients and eligible patients whose medical records were reviewed using GTT between 2013 and 2018

Calculation of annual incident reports per employee

We retrospectively examined the annual incident reports per employee from 2013 to 2018. We divided the total number of reports for each year by the total number of employees for that year. In this study, we examined the annual changes in incident reporting activity for all employees who have undergone general training. In other words, we attempted to monitor the tendency of the reporting culture of the entire organisation.

Statistical analysis

We conducted our statistical analysis using IBM SPSS for Windows V.28 (IBM) and applied the Kruskal-Wallis test for the speaking-up ability score.

Ethical considerations

To validate the incidence of AEs, we used anonymised data for research purposes, as instructed by the ethics committee, and posted an opt-out option on the home page. The incident report data contained no personal patient information.

Patient and public involvement

Patients and the public were not involved in this study.

Results

The Gantt chart in figure 1 summarises the interventions and measures.

Figure 1
Request permissionrequest permission icon
Figure 1

Gantt chart of interventions and measures.

The level of understanding and applicability to clinical settings in general training

During the first year, about 2000 out of 2500 employees received general training. The results of the questionnaire survey showed that we obtained a high percentage of positive responses (88.9–94.6%) regarding employees’ understanding of the training content (online supplemental table 2). The positive response rate that was related to prompt information sharing and the two-challenge rule differed significantly across professions. Compared with other professions, doctors, especially junior doctors, had the lowest percentage (43.7%) for the two-challenge rule. Compared with all nurses, junior nurses had the lowest percentage on all questions.

Speaking-up attitudes in doctors and nurses

The average response rate of annual questionnaire surveys from 2016 to 2018 was 50.1% (47.9–52.2%) for doctors and 88.0% (84.1–94.1%) for nurses. The score of ‘perceived barriers to speaking up’ for doctors and nurses decreased significantly after the training programme from 2.91 to 2.60 (p<0.01) and from 3.20 to 3.00 (p<0.01), respectively (figure 2A,B). The score of ‘Negative attitude toward voicing opinions in the healthcare team’ was not reduced among all doctors, but junior doctors and nurses did show a significant reduction (figure 2C,D).

Figure 2
Request permissionrequest permission icon
Figure 2

(A) All doctors and junior doctors: changes in scores of ‘perceived response for speaking up’ used to measure attitudes towards speaking up before the training (2016), during the training (2017) and the year the training was established (2018). (B) All nurses and junior nurses: changes in scores of ‘perceived response for speaking up’ used to measure attitudes towards speaking up before the training (2016), during the training (2017) and the year the training was established (2018). (C) All doctors and junior doctors: changes in scores of ‘negative attitudes toward voicing opinions in the healthcare team’, used to measure attitudes towards speaking up before the training (2016), during the training (2017) and the year the training was established (2018). (D) All nurses and junior nurses: changes in scores of ‘negative attitudes toward voicing opinions in the healthcare team’, used to measure attitudes towards speaking up before the training (2016), during the training (2017) and the year the training was established (2018).

Monitoring awareness of the flow chart

The results from monitoring the awareness of the EARRTH flow chart posted in each ward by the flow chart awareness survey were as follows: in 2017, 69% of doctors and 85% of nurses, while in 2018, 58% of doctors and 92% of nurses were aware.

Number of AEs and the annual incident reports per employee

The median of AE occurrences per 1000 person-days for incidents in categories E–I was 32.1 before the intervention, 39.9 after starting the activity of the WG, 22.2 after conducting the general and practical training programmes and 18.4 after starting the leadership training (figure 3).

Figure 3
Request permissionrequest permission icon
Figure 3

The number of adverse events (AEs) per 1000 person-days in category E–I incidents using Global Trigger Tool (GTT) between 2013 and 2018.

Figure 4 shows the increase in annual incident reports per employee during the intervention period from 2.89 in 2013 to 3.55 in 2018.

Figure 4
Request permissionrequest permission icon
Figure 4

Changes in annual incident reports per employee from 2013 to 2018. Annual incident reports per employee were calculated by dividing the total number of reports for each year by the total number of employees for that year.

Discussion

Summary of the main outcomes

The main outcomes of our intervention bundle are, first, an improvement in speaking-up attitudes measured by the psychological scale and, second, its possible effect on staff communication behaviours, which may have led to a decrease in AEs. The decreased resistance to raising problems suggests that the promotion of speaking-up attitudes led staff to adequately cope with the patient risks and, consequently, it could have led to a decrease in AEs.

Status of clinical settings in the first year of training

The questionnaire survey following the implementation of practical training in 2016 showed that doctors and nurses held few positive opinions regarding ‘prompt information sharing’ and ‘two-challenge rule’, although they understood the training purpose quite well. Junior doctors and nurses showed low positive responses to ‘prompt information sharing’, ‘two-challenge rule’, ‘SBAR’, ‘monitoring and consultation’, ‘check back’ and ‘attitudes toward listening’. These results indicate that the authority hierarchy led to communication issues, especially for junior staff, in our hospital in 2016.19

Training improves speaking-up attitude scores among junior doctors and nurses

Figure 2 shows a decrease in the scores for two factors: ‘perceived barriers to speaking up’ and ‘negative attitudes toward voicing opinions in the healthcare team’. Figure 2A,C show that the score of the former factor did not decrease for all doctors but decreased significantly for junior doctors. Contrastingly, figure 2B,D show that all nurses, including the junior group, showed significantly decreased scores for both factors. These results suggest that the training helped improve the perception of speaking-up attitudes for all nurses and junior doctors. One reason for improving speaking-up attitudes among nurses and junior doctors is that we incorporated TeamSTEPPS tools to our training programme. TeamSTEPPS is renowned as non-technical skills training programme20 that improves the ability to speak up.21

The other reason could be explained by the concept, psychological safety, which is the ‘shared belief held by team members that the team is safe for interpersonal risk-taking’ (p 350).22 In psychologically safe workplaces, people can admit mistakes or ask questions without fear of blame or reprisals.23 24 Organisations must ensure psychological safety and guarantee that staff will not be punished for reporting threats to patient safety.25 From the perspective of psychological safety, we suggest that the interactive leadership training introduced in 2017 might have enhanced the effect of the training for junior doctors and nurses. Owing to the leadership training, supervisors, departmental chiefs and executives might be trying to build a psychologically safe environment.

According to Edmondson’s study, the Japanese culture has difficulty building psychological safety, except for some Japanese companies such as Toyota.26 Our hospital is no exception; thus, it is necessary to continue the leadership training and monitor the entire organisation climate using Edmondson’s psychological safety scales.21

Improved speaking-up attitude promotes decrease in AEs and increase in incident reports

Figure 3 shows that AEs decreased gradually following the interventions, including establishing the WG and starting the general and practical training (EARRTH) and leadership training, even though the median age of the patients did not change (table 1). Furthermore, figure 4 shows that the annual incident reports per employee had been increasing during the intervention. According to previous reports, rapid information sharing among healthcare professionals reduces AEs by prompting appropriate actions.8 Thus, improving speaking-up attitudes enhances prompt, appropriate actions to ensure patient safety. As mentioned, the improvement in speaking-up attitudes due to the training might have enhanced communication skills and led to prompt and appropriate actions to cope with patients’ safety issues. It is also suggested that the leadership training may have encouraged supervisors to create a psychologically safe workplace. These hypotheses are supported by the sustained increase in the number of incident reports per employee during the interventions. However, in the future, these hypotheses need to be assessed measuring communication skills directly and investigating the number of AEs caused by communication failure.

Strength of the intervention

The intervention is sustainable. Since 2016, we have continued EARRTH using standardised educational method and tools such as the flow chart, lecture slides, animated movie, role-play scenarios and interactive training.13 The WG evaluates and improves the programmes continuously, and each year new facilitators are trained.

Secondarily, the training programme is interactive. The programme using tools of TeamSTEPPS included group discussion and role-play after watching animated movie. The combination between an interactive programme and non-technical skills training is effective. For example, a comparison of role-play training on SBAR in TeamSTEPPS results in higher communication skills transfer than lecture-style training.27 As mentioned in previous studies, our new interactive educational programmes seem to have better learning effects on learners in understanding the importance of improving communication skills.12

Finally, the leadership training helps build psychological safety in the organisation because, as Edmondson21 mentioned, building psychological safety is the role of team leaders. Using theatre exercises for the leadership training programme may help leaders understand how to build a psychological safe workplace.

Limitations

Originally, IHI-GTT requires two individuals to screen data and one to review the results, every 2 weeks on a prospective basis. Due to the past 6-year survey period in this study, only one person screened the data. In the future, two individuals should screen the data every 2 weeks and monitor the occurrence of AEs prospectively to evaluate the interventions.

Second, we showed an increase in incident reports per employee during the intervention period. However, the number of incident reports had been on the rise even before the intervention began. Therefore, this result does not indicate the outcome of the intervention. We therefore consider that our intervention may have been one of the contributing factors to the continuous increase in incident reporting behaviours of employees.

Third, in the questionnaire survey regarding the speaking-up scale, doctors’ response rate was low at around 50%. Therefore, the data cannot be generalised to all doctors at the hospital. We will continue to make efforts to increase doctors’ response rate.

Finally, the intervention was conducted only at our institution. We are currently developing more generalised programmes and collaborating with other institutions. We will report these results in future.

Conclusions

Following a 2014 incident resulting from a communication failure, we developed and implemented an intervention bundle based on the proposal of the investigation committee. The intervention included a new educational programme and tools for communication training, which was implemented 2016 onwards. Using a psychological scale, the perception of speaking-up attitude improved after the training, especially among junior doctors and nurses. Additionally, the occurrence of AEs, calculated by the IHI-GTT, decreased after the intervention. It is speculated that improving the perception of speaking-up attitudes resulted in adequate sharing of information regarding patients’ safety issues. Continuous increase in the annual incident reports per employee could support this speculation. However, the direct measurement of staff behaviours and communication skills will be required in the future to support these hypotheses.

Article metrics
Altmetric data not available for this article.
Dimensionsopen-url