Article Text
PDF
Abstract
Background Root cause analysis (RCA) is a structured investigation methodology aimed at identifying systems factors to prevent recurrence of incidents. To enhance staff’s knowledge and skills, a hybrid RCA training course was conducted in February 2021. Overseas instructors conducted training online and local participants attended the training together physically with onsite facilitator support. This study aimed at understanding the experiences of trainees who have undergone the training, evaluated its effectiveness and identified opportunities to enhance RCA training quality in the future.
Methods A qualitative study using virtual synchronous focus group interviews was conducted. Purposive sampling was adopted to invite all trainees from the RCA training course to join. A semistructured interview was used to guide the study participants to share their experiences. All groups were audio-recorded, transcribed verbatim and anonymised for data analysis.
Results Overall, 6 focus groups with 19 participants were held between July and November 2021. Five key themes were identified including: (1) training contents, (2) perceptions of RCA, (3) challenges in RCA, (4) hybrid training and (5) future perspectives. Participants felt the RCA training was useful and broadened their understanding in incident investigation. More in-depth training in interviewing skills, report writing with practical sessions could further enhance their competencies in RCA. Participants accepted the use of hybrid online–offline training well. Most participants would welcome an independent organisation to conduct RCA as findings would be more objective and recommendations more effective.
Conclusions This study provided an evaluation on the effectiveness of a hybrid RCA training course. Healthcare and training organisations can consider this training mode as it could reduce the cost of training and enhance flexibility in course arrangement while preserving quality and effectiveness. Virtual focus groups to interview participants were found to be convenient as it minimised travelling time and onsite arrangement while maintaining the quality of discussion.
- Root cause analysis
- Focus Groups
- Education
- Human error
- Qualitative research
Data availability statement
Data are available upon reasonable request. Focus group data will be shared on reasonable request to the corresponding author.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Team members of root cause analysis (RCA) who have undergone dedicated training could achieve better quality of RCA. This study aimed at understanding the experiences of trainees who had undergone a 2-day training course in a hybrid mode.
WHAT THIS STUDY ADDS
This study showed that participants were satisfied about conducting RCA training course in hybrid mode. The experience in preparing a hybrid RCA training was illustrated. Virtual focus group to interview research participants was also found to be convenient while maintaining the quality of discussion.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This study can be used as a basis for developing a better curriculum on RCA training and also proved the effectiveness of virtual focus group in conducting qualitative research.
Introduction
Root cause analysis (RCA) is a structured investigation methodology aimed at identifying the latent factors in the work system contributing to underperformance, variation and design failures and supporting the identification of improvement actions to prevent future occurrence of similar incidents.1 2 However, various studies criticised that many RCA recommendations were weak.3–6 The reasons why RCA cannot effectively identify stronger recommendations include lack of time and resources, inadequate training, lack of expertise in human factors, incomprehensive panel composition and complicated processes in carrying out large-scale improvements.2–8
The process of RCA involves multiple steps. The formation of an RCA team is crucial to determine whether an RCA is successful.9 10 The team should include representatives from various disciplines and the members should be specialised in their respective clinical practices or safety science, understand how to conduct an RCA, be equipped with knowledge and skills in systems thinking and human factors and be experienced in dealing with emotions and blame.3 6 8 9 To achieve better quality of RCA, team members should undergo dedicated RCA training to ensure they are equipped with the required knowledge and skills.6 8 9
The duration of commercial RCA training courses commonly ranges between 1 and 3 days.11–14 With the emergent impact of COVID-19, many learning courses have migrated from face-to-face to online or hybrid (comprising both online and face-to-face format) learning.15–18 Online learning definitely promotes social distancing and minimise infection risks; however, its drawbacks should not be underestimated.15 Students may feel unmotivated, more stressful and distracted and experience less interpersonal interaction and immediate feedback.16 19 The sudden change of teaching approach also cause stress and anxiety to instructors.15 18
In Hong Kong, the public hospital service is governed by the hospital authority (HA). According to the HA Sentinel and Serious Untoward Events Policy, an RCA team has to be set up for investigation and proposing recommendations. The RCA team consists of representatives from the involved hospitals and HA Head Office, depending on the nature of the adverse events. An RCA report has to be submitted to the HA Head Office within 8 weeks. To enhance staff’s knowledge and skills, the HA RCA Review Workgroup conducted 2 identical classes of a 2-day RCA training course to 24 staff (each with 12 staff) in February 2021. These staff came from different public hospitals, were required to be familiar with their hospital’s quality and safety service and would normally participate in incident investigations.
The 2-day training course was conducted by a non-profit organisation from the UK. The course instructors were experienced in incident investigation and systems improvement. The course contents included how to prepare and conduct an RCA meeting and knowledge, skills and techniques in gathering facts, identifying and analysing problems, writing RCA reports and implementing solutions. The benefits of better engagement and support for the trainees, and infection risks were continuously evaluated by the programme coordinators of the RCA training course during course planning. Due to travel restrictions and the ongoing COVID-19 pandemic situation, the overseas instructors could not travel from the UK to Hong Kong to conduct the training. Therefore, the training was conducted via a hybrid mode, that is, the instructors conducted the training online (via Zoom) and local participants attended the training in a single training site in person. The programme coordinators and local information technology team supported the venue and equipment set-up at the training site. The programme coordinators, who were experienced in RCA investigations, facilitated the discussion between instructors and trainees. The trainees were divided into small groups to encourage discussion and ensure social distancing. Pretraining materials were distributed to the trainees while a pretraining survey to understand trainees’ perception and attitudes to RCA and its training and a post-training survey to gather course feedback were conducted.
This study aimed to understand the experiences of trainees who have undergone a hybrid RCA training, evaluated the training’s effectiveness and identified opportunities to enhance training quality in the future.
Methods
Study design
A qualitative study using focus group interviews was conducted. Purposive sampling was adopted by inviting all trainees who had participated in the RCA training course to join the focus groups. Due to the COVID-19 pandemic situation, virtual synchronous (via Zoom) instead of face-to-face focus groups were conducted. The researchers and participants (the term ‘participants’ in this report is used to indicate trainees who joined the focus group study) joined the virtual meeting rooms (required password to join to ensure privacy and confidentiality) at their own working sites. The researchers AM and YTK acted as the focus group facilitators. Both the researchers and participants were required to turn on their cameras so everyone could interact auditorily and visually. All participants were informed of the arrangement in the invitation and consent forms were collected before the interview.
The researchers AM and YTK were members of the HA RCA Review Workgroup and programme coordinators of the RCA training course. YTK was an HA staff while AM was an ex-HA staff. The researchers worked/had worked in the HA but did not have any direct relationship established with the participants prior to the study commencement.
Data collection
All trainees were invited to join one of the scheduled focus group sessions. To ensure smooth running of the focus groups, the number of participants had been limited to at most six persons in each group (fewer than traditional face-to-face format).20 21 Trainees who refused to join or had left the HA were excluded from the study, that is, not counted as ‘participants’. They were also excluded if they could not join despite multiple attempts to arrange. The researchers AM and YTK facilitated the focus groups by using a semistructured questionnaire (table 1) to guide the participants to share their experiences. The questionnaire was developed after reviewing the results of the pre-training and post-training surveys to obtain a more in-depth understanding of the participants’ training experiences. Each focus group lasted 1–1.5 hours depending on the number of participants in each group. All groups were audio-recorded, transcribed verbatim and anonymised for data analysis.
Focus group questions
Data analysis
A unique number was assigned to each participant randomly to enhance confidentiality. The focus group data were coded and categorised using NVivo V.12. Thematic analysis was conducted by researcher YTK and agreed by researcher AM. The codes and themes were revised until mutual agreement was reached between the researchers.
Patient and public involvement
The research was done without patient or public involvement.
Results
Overall, 19 out of 24 trainees (ie, 19 participants) had joined the focus group study. Six focus groups were held between July and November 2021, each with two to six participants. Of the 19 participants who joined the focus groups, 58% (n=11) were male and 42% (n=8) were female. Overall, 42% (n=8), 53% (n=10) and 5% (n=1) were doctors, nurses and administrator, respectively. Of the five participants who did not join, three could not join due to unavailable schedule, one refused to join due to personal reason and one had left the HA.
Five key themes were identified and summarised below. The themes included: (1) training content, (2) perception of RCA, (3) challenges in RCA, (4) hybrid training mode and (5) future perspectives.
Theme 1: training content
All participants appreciated the organisation of the training course. They were impressed by the comprehensive training content. The course also provided useful tools for them to apply in incident investigation.
The experience was very good because I think it was the first time for me to join such training. It was quite informative. [Participant 6]
The most impressed in the training was the tool introduced showing us how to obtain details and find the contributing factors. [Participant 13]
Some participants felt interviewing staff was challenging. They believed a session in teaching interviewing skills would be beneficial. Some participants also wished to learn more about writing balanced but effective RCA reports that promotes change, have some practical sessions and have experience sharing from RCA experts.
I think it is very important to let the panel members know how to conduct the interview, so as to have the chance to get more details of the incident. [Participant 16]
It is also very good to have hands-on practice on interviewing techniques … because the person may have emotions and guilty feelings and also they may be suffering from stress during the interview. [Participant 14]
We expect to have more practical sessions especially on day two. It’s more useful especially when we try to use the newly introduced tools to analyze the case. [Participant 18]
I think ‘how to develop recommendations’ can be added in the next training because it is very essential for us in writing a report. [Participant 15]
We probably need to invite local experts to come and share their experiences. We really need to learn from them like how to handle sensitive cases. [Participant 17]
Theme 2: perception of RCA
Participants felt the RCA training had changed their perception of incident investigation. They felt their mindset had changed from being person focused to system focused.
The training makes us approach things more systematically and structurally. [Participant 2]
I like the idea how they look into the root causes because they're not only looking at something ‘very human’. They mainly look at the system and how we can modify the behavior by modifying the system. [Participant 10]
I think the training has broadened my perspective into thinking about more root causes and analyzing the recommendations to know whether they’re good or bad, or strong or weak. [Participant 17]
The confidence of participants in teaching or conducting RCA varied. Although some participants had already conducted a few RCAs after the training, they commented that they needed more real case experience before having the confidence to teach.
I feel more confident in asking the interviewees questions in an RCA meeting, because I have prepared some questions beforehand, and so I'll use them to ask the participants to clarify the fact. [Participant 7]
I'm not so experienced in using the new tools, and I did have bad experience using it in real situation. If I have some successful cases that I can share with my colleagues, maybe I will be more confident to be a trainer in RCA workshops. [Participant 18]
I think my confidence of conducting RCA is much improved after the RCA training …I think I need to do more RCAs, then I’ll feel much more comfortable to do RCA. [Participant 4]
Theme 3: challenges in conducting an RCA
Participants identified several challenges that affect the development of an effective RCA. These include the extensive time required in conducting an RCA, limited knowledge of the RCA investigators, biased perception of other panel members and the reluctance of involved departments to follow RCA recommendations.
If someone may already have their conclusion in their mind, it’s quite easy for them to interfere the results, and even sometimes it’s difficult for us to ask questions, because the atmosphere is not really good. [Participant 4]
Sometimes I encountered that the local RCA panel members may have biases especially if the interviewees came from their department. [Participant 18]
They (the involved departments) may think that your recommendations do not focus on solving the problem indeed. [Participant 8]
I needed to spend more than 10 hours in each case, not including writing the report. I need to spend a lot of time meeting a lot of people and sit down to understand the logistics. I also need to conduct site visit to look at the environment in detail. … It’s quite time consuming. [Participant 6]
I think the difficulty is the deadlines in finding the root causes … Sometimes the root causes may be sensitive and panel members do not find it appropriate to document them in the final RCA report. [Participant 17]
Theme 4: hybrid training
Most participants understood the reason why the RCA training course had to be conducted in a hybrid mode. Some of them felt it was acceptable while others preferred in-person training as they could have better interactions with the trainer.
I think the online experience was okay, because in the COVID situation, I have already been attending many Zoom meetings. [Participant 14]
I would like to attend in-person training rather than the online one. However, because of the restrictions, it’s better than none actually. [Participant 2]
If the speaker is physically present, we can communicate more directly and there is more interaction, so there will be more benefits for us. [Participant 1]
The hybrid training mode might also have limited the efficacy of a practical training session especially for staff interview techniques.
I can see why interviewing skills were not conducted for this virtual training because I need to look at your eyes, your body posture, etc. It’s very difficult for virtual teaching to teach the interviewing skills. [Participant 11]
I think it’s because of this time the teachers are online with zoom teaching, so it makes them difficult to have an instant interview training or more interactive way. [Participant 2]
Theme 5: future perspectives
Participants shared different perceptions if patients and their relatives were invited for an interview during the investigation. Some participants felt there might be difficulty in handling patient’s emotions.
I think it’s good to have our patients or relatives joining the interview because we may only use our professional view to look into incidents. With relatives or patients involved in the interview, we can know from their perspective how the incident happened and how to improve or prevent the errors from happening again. [Participant 3]
Sometimes it would become a ‘complaining’ meeting. They can find a time to complain and keep asking us why. It is quite challenging to the RCA panel. [Participant 4]
Participants were also asked if patient group representatives should be invited into the RCA panel. Some commented that the patient group representatives could help provide a broader perspective but others were concerned that they could be biased towards the patient.
If we have a trained person who is from the patient association to join as a RCA panel member, it’s helpful. [Participant 18]
I think if a lay member is involved and understands what we have recommended from the findings, the society or public will have higher acceptance to our findings. [Participant 19]
They usually have their perspective, from their side making an accusation on us, reasonable or unreasonable, so it will make things more complicated. [Participant 2]
Almost all participants were enthusiastic in having an independent organisation to conduct RCA. They agreed that it allowed RCA findings to be more objective, minimised their workload as they might not need to participate in an investigation, reduced stress and supported more effective recommendations.
External agency or organization can do the work better because they have no conflicts of interest, and they see the event as outsiders to make fair recommendations. [Participant 1]
Well, I love it. I don't want to be the bad guy in the department. [Participant 10]
The independent party or special RCA team needs much more time to be familiar with our staff and make our staff comfortable. [Participant 15]
If there is an independent external body to carry this out, I hope this allows it to have more authority to implement changes. [Participant 2]
They may not actually know the situation or limitations of the HA or the departments, so the recommendations or advice given by the external parties may not be totally practicable in the system … maybe we can improve the proportion of the internal and external members so that the recommendations will be more practical and more independent. [Participant 3]
Discussion
This study showed that the participants felt the RCA training was useful and broadened their understanding in incident investigation. They perceived the course as suitable for people who did not have much experience in RCA. More in-depth training in interviewing skills, report writing with practical sessions could further enhance their competencies. The participants also shared their perceived challenges in RCA.
Although the hybrid training mode was new to the trainer and participants, the course was well accepted by the participants. Based on the experience of our RCA training course, a few areas were identified that need consideration during course organisation to ensure the hybrid training format was successful. First, the presence of facilitators at the local training site is crucial to facilitate interaction between the instructors and trainees, provide on-site support such as material distribution, prevent overrunning of the training and handle any unexpected problems such as unstable internet connection. The presence of facilitators at the instructors’ location might not be able to handle such matters effectively. Second, on-site testing of internet connection with the instructors prior to training can identify potential internet connection problems. Third, the set-up of training site should facilitate interactions between the instructors and trainees, for examples, in our RCA training course, projector screens were set up to show the instructors’ faces and training PowerPoint slides, a training site camera was set up (connected to the Zoom meeting room) to show a wider view of the training site to allow the instructors to have an overview of the training site and understand the actual situation, computers with built-in camera and microphone were set up on the table of each trainees’ group to allow the instructors to facilitate discussion and have a closer look of trainees’ facial expressions. The venue set-up of our RCA training course is illustrated in figure 1.
Illustration of venue set-up of root cause analysis (RCA) training course.
The participants also shared their perspectives on the future of RCA. It is believed that RCA training courses have to include emotion and conflict management if patients, relatives and patient group representatives are involved in an RCA. Despite training have been given, most participants welcomed an independent organisation to conduct RCA as they believed the findings and recommendations would be more objective and effective, respectively.
Training is essential before a staff participates in an RCA. Training needs should be collected to ensure contents of the training course fulfil the actual requirement in an RCA. The challenges identified in this study were similar to those from other studies.1 22 Leadership from the hospital management is also instrumental to successfully introducing RCA concepts and systems thinking, and spreading improvements derived from an RCA to a wider context beyond the local department.7–9
Virtual focus groups have become popular and have many advantages.20 21 23 In our study, the participants worked in different hospitals and the geographical constraints were minimised. This allowed them and the researchers to better schedule and arrange the interviews. There were also fewer technological problems as the participants were used to virtual meetings. Although virtual focus groups may hinder interactions among the participants, this effect was minimised because the participants could see each other’s facial expressions through the camera. A virtual focus group will be successful when the following actions are considered: (1) remind participants to find a quiet and secure room to ensure privacy, (2) ensure adequate amount of cameras are set up so the online interviewer can have a better overview of the physical venue and closer look of the participants during discussion, (3) test the cameras and audio devices before interview and (4) explain the ground rules including speaking one by one and muting the microphone when not speaking.24
This study was one of the pioneers in evaluating the effectiveness of an RCA training course especially via an online or hybrid mode. Healthcare and training organisations can consider this hybrid training mode as it could save expenses of inviting overseas trainers and enhance flexibility in course arrangement while preserving the quality of the course. More research on training effectiveness could also help develop a better curriculum on RCA training.
This study had some limitations. It was the first time for the HA to conduct an RCA training course via a hybrid mode. There was no other local study evaluating the effectiveness of such training mode for comparison. Also, the focus group study was conducted 6 months after the RCA training. Most participants did not have much experience in participating in an RCA. Their perception may be different when they have more RCA experience.
Conclusions
This study provided an evaluation on the effectiveness of a hybrid RCA training course. Participants were satisfied with the course. They believed more in-depth training in interviewing skills and report writing with practical sessions could further enhance their competency. Most participants agreed that an independent organisation to conduct RCA would be beneficial. Also, virtual focus groups to interview research participants was found to be convenient as it minimised travelling time and onsite arrangements while maintain the quality of discussion.
Data availability statement
Data are available upon reasonable request. Focus group data will be shared on reasonable request to the corresponding author.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants. Approval from the HA Clinical Research Ethics Committee (Kowloon Central/Kowloon East) (reference number: KC/KE-21-0049/ER-3) was obtained. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors would like to thank members of HA RCA Review Workgroup, especially Dr Gladys Kwan and Dr Sara Ho, for their support in conducting this study.
Footnotes
Contributors YTK and AM contributed to the design, focus groups, NVivo coding and manuscript. Both authors read and approved the final manuscript. YTK was the guarantor of the study.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.