You are here

Article Text

Article menu
Original research
Generic analysis method to learn from serious adverse events in Dutch hospitals: a human factors perspective
  1. Mees Casper Baartmans1,
  2. Steffie Marijke Van Schoten2,
  3. Cordula Wagner3
  1. 1Department of Organisation and Quality of Care, Netherlands Institute for Health Services Research (Nivel), Utrecht, The Netherlands
  2. 2Department of Public and Occupational Health, Amsterdam UMC Locatie VUmc, Amsterdam, Noord-Holland, The Netherlands
  3. 3Netherlands Institute for Health Services Research (Nivel), Utrecht, The Netherlands
  1. Correspondence to Mees Casper Baartmans; m.baartmans{at}nivel.nl

Abstract

Background Hospitals in various countries such as the Netherlands investigate and analyse serious adverse events (SAEs) to learn from previous events and attempt to prevent recurrence. However, current methods for SAE analysis do not address the complexity of healthcare and investigations typically focus on single events on the hospital level. This hampers hospitals in their ambition to learn from SAEs. Integrating human factors thinking and using a holistic and more consistent method could improve learning from SAEs.

Aim This study aims to develop a novel generic analysis method (GAM) to: (1) facilitate a holistic event analysis using a human factors perspective and (2) ease aggregate analysis of events across hospitals.

Methods Multiple steps of carefully evaluating, testing and continuously refining prototypes of the method were performed. Various Dutch stakeholders in the field of patient safety were involved in each step. Theoretical experts were consulted, and the prototype was pretested using information-rich SAE reports from Dutch hospitals. Expert panels, engaging quality and safety experts and medical specialists from various hospitals were consulted for face and content validity evaluation. User test sessions concluded the development of the method.

Results The final version of the GAM consists of a framework and affiliated questionnaire. GAM combines elements of three methods for SAE analysis currently practised by Dutch hospitals. It is structured according to the Systems Engineering Initiative for Patient Safety model, which incorporates a human factors perspective into the analysis. These eases aggregated analysis of SAEs across hospitals and helps to consider the complexity of healthcare work systems.

Conclusion The GAM is a valuable new tool for hospitals to learn from SAEs. The method can facilitate a holistic aggregate analysis of SAEs across hospitals using a human factors perspective, and is now ready for further extensive testing.

  • human factors
  • adverse events
  • epidemiology and detection
  • safety management
  • root cause analysis
  • patient safety

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study. Not applicable.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2021-001637

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Adverse events in healthcare represent a major source of morbidity and mortality and result in substantial societal costs.1–4 Hospitals in numerous countries, such as the Netherlands, the USA, Canada, Australia and the UK, aim to learn from previous events by thoroughly investigating these. In particular, hospitals invest in studying the events that have caused temporary or permanent disability, death or prolonged hospital stay, also known as sentinel events or serious adverse events (SAEs).5–10 After hospitals investigate a SAE, they write a detailed report. This report starts with a comprehensive reconstruction of the event and concludes with a root cause analysis (RCA), in which hospitals search for root causes and formulate recommendations to prevent recurrence.11 SAE investigations and the associated reports offer great potential for hospitals to learn from each other and improve patient safety. However, SAE investigations and analyses are being criticised for oversimplification,12 resulting in weak solutions,10 11 13 and overall ineffectiveness.11 13–15 Two important shortcomings impede hospitals in particular in their ambition to learn from SAEs.

    First, SAE investigations often neglect the complexity of healthcare.13 15 Current methods applied for SAE analysis, focus on finding one linear root cause, even though SAEs are more likely to arise from interactions and combinations of causes and contributing factors in the complex healthcare system.10 13 14 SAE analysis thus needs a shift from focusing on finding the ‘one’ linear root cause towards searching for the combination of interrelated contributing factors and causes. The incorporation of human factors thinking in SAE analysis is assumed to induce this shift.16–20 Human factors is concerned with the understanding of interactions and interdependencies between humans and other elements of a work system.21–23 It searches for opportunities to design healthcare systems that have a greater tolerance of faults and thus might improve the resilience of the system.24–26 One model that integrated principles of human factors into the healthcare domain is the Systems Engineering Initiative for Patient Safety (SEIPS).27–32 This theoretical model is used to study patient safety hazards and adverse events in various healthcare settings.16 30–35 SEIPS is based on the well-known structure–process–outcome model for healthcare quality.36 It therefore is assumed to be familiar to audiences working in healthcare quality and patient safety.31 Although the potential of SEIPS to improve patient safety is proven,31 adoption of the model in retrospective SAE analysis is still limited and should be refined and accelerated.16 30

    A second shortcoming of current SAE investigations is that hospitals typically focus on single events within their own organisation,13 14 even though attempts to learn from aggregate analysis of multiple events across hospitals, as already performed in Australia37 and the USA,38 39 are believed to improve learning.38–41 It helps to discover combinations of recurring, underlying, patterns of causes and contributing factors42 and may improve the formulation of more effective, system-aimed recommendations.14 38 However, methods for analysing SAEs vary substantially,10 which complicates aggregate cross-hospitals analysis of SAEs. For example, in the Netherlands, where aggregate analysis of SAEs is scarce, hospitals use three differing methods for analysing SAEs41 43 44: Prevention and Recovery Information System for Monitoring and Analysis (PRISMA-medical),45 Tripod Beta46 and Systemic Incident Reconstruction and Evaluation (SIRE).47 These methods are useful for a structured analysis of SAEs, yet based on their description in scientific articles,41 44 48 49 anecdotal evidence43 46 47 and experience, each method has its own approach, focus and limitations. PRISMA-medical, for example, is highly analytical and categorises latent (technical and organisational) as well as active (human) failures.45 However, the absence of organisational or technical barriers that could prevent SAEs are not considered a root cause in this method.50 Tripod Beta, however, thoroughly investigates system failures and organisational barriers that could prevent SAEs. However, human errors are also explained as organisational or system failures,46 49 even though evidence indicates that flaws in the system and cognitive factors play an important role in the emergence of human error.51–53 SIRE, in turn, emphasises particularly on the primary process and provides a comprehensive narrative description of the event.47 54 To ease aggregate cross-hospital analysis of SAEs, Smits et al43 made a primary attempt to integrate the structures and foci of PRISMA-medical, SIRE and Tripod Beta into a generic framework. Anecdotal evidence of the benefits of this framework to perform an aggregate cross-hospital analysis was presented, but the study emphasised the importance of further development and evaluation.43

    Addressing the complexity of healthcare13 and using a consistent method10 41 could thus improve learning from SAEs. This could help to formulate more effective recommendations to enhance patient safety. An alternative approach to study SAEs is therefore necessary. This study aims to develop a novel generic analysis method (GAM) that integrates the SEIPS model31 32 and the PRISMA-medical, SIRE and Tripod Beta framework43 to: (1) facilitate a more holistic analysis using a human factors perspective and (2) ease aggregate analysis across hospitals.

    Methods

    We followed a structured process (figure 1) of developing, evaluating, testing and refining prototypes of a GAM.

    Figure 1
    Figure 1

    Overview of the steps taken in developing the Generic Analysis Method. PRISMA, Prevention and Recovery Information System for Monitoring and Analysis; SIRE, Systemic Incident Reconstruction and Evaluation; SEIPS, Systems Engineering Initiative for Patient Safety.

    Description of the prototype

    A first prototype of the GAM was developed based on the theoretical SEIPS model31 32 and the framework43 combining PRISMA-medical, SIRE and Tripod Beta. This prototype included three main pillars. The first pillar addresses the gathering of basic SAE information. Second, each sociotechnical domain of the SEIPS model is evaluated. All relevant characteristics should be collected about the: (1) persons involved, (2) tasks performed, (3) technologies used, (4) organisational factors affecting the work, (5) the physical environment, and (6) the external environment. In this evaluation, we also adopted some important elements of PRISMA-medical, SIRE and Tripod Beta as accumulated by Smits et al43 related to these six domains. The third pillar focuses on the outcomes of the SAE for the involved patients and/or their family, healthcare professionals and the organisation. This part also includes the evaluation of the recommendations formulated in response to the event.

    Evaluating and testing the prototype

    We started with consulting two theoretical experts (A), both academics in the field of patient safety, to evaluate the first version of the prototype and ask for potential additional theoretical concepts that should be embedded.

    We then used the prototype for a pretest session (B), in which the author (MCB) analysed six SAE reports covering the variation of RCA methods used by Dutch hospitals (PRISMA-medical, SIRE and Tripod Beta). These reports were based on thorough SAE investigations performed by independent committees of various hospitals in 2018 or 2019. The multidisciplinary hospital committees investigating the event, typically consisted of a combination of clinicians and quality and safety officers. Committees combined multiple sources for studying the event. The most important source being in-depth interviews with involved healthcare personnel and patients and/or family. Additionally, the committees reviewed all relevant information such as patient records, test results (eg, medical images, laboratory findings, etc) and documentation (eg, guidelines, work instructions, etc). The combination of interviews with the persons involved and reviewing all relevant documentation resulted in an extensive and detailed reconstruction of the event. After the event description, the hospital committees performed an RCA and formulated recommendations to prevent recurrence. The comprehensive data in the SAE reports was suited for a retrospective (re)analysis of the event and, hence, a first test of the prototype’s content and face validity. By including a sample covering all three RCA methods, we could test whether our prototype was applicable irrespective of the initial RCA method used. This step contributed to improving the usability of the prototype, for example, the prototype could be aligned with the working methods of the hospitals.

    Next, a panel with experts from practice (C) was consulted for a thorough evaluation of the face and content validity of the prototype during a face-to-face meeting. This panel consisted of a mix of quality and safety officers (n=8) and medical specialists with direct experience in patient safety (n=5). All of them were currently practising in various Dutch general hospitals (n=12). Minutes were taken during the meeting, and the transcribed notes were sent for verification to the participants afterwards.

    Subsequently, two potential users, both quality and safety officers, tested the prototype (D). Just as in the pretest session, information-rich SAE reports were used. The users analysed two distinct reports, written by two general hospitals in 2018. This test session was meant to further evaluate content validity of the method as experienced by potential users. It also helped to find flaws and practical barriers and provided the possibility to observe users’ interactions with the prototype. Commentaries from the respondents were used to refine the usability. In addition, the author (MCB) analysed the same two reports. This allowed a rough assessment of the agreement in analyses between the potential users and the researcher. As the most important questions were open ended, a qualitative approach on assessing the agreement between raters was performed. Overall agreement on all questions was reviewed by eye assessment, and differences between the interpretation of raters for the important questions were reflected on (eg, by considering: did raters give different answers? Why did raters not answer that question the same way? Were there random outliers, or did it show a systematic difference in interpretation?).

    Patient and public involvement

    Since the method is specifically intended for use by healthcare professionals to learn from SAEs across hospitals, professionals (potential users) were consulted at multiple stages during the study. Patients were not involved in developing the method as they currently have no formal role in performing (aggregate) analysis of SAEs. Nevertheless, it is important to involve patients and their family in the SAE investigation by including their vision on the event through interviews and giving them insight into the findings.

    Results

    Evaluation and test results

    In the first step of evaluation, theoretical experts (A) stressed the importance to assess the element of human error in SAEs and therefore referred to the model of unsafe acts.52 This model helps to differentiate between various types of human errors (eg, violations, mistakes, slips and lapses). This could provide a more in-depth understanding of these errors. Another concept the experts recommended to include were the efficiency–thoroughness trade-off principles. Evaluation of the trade-offs between working efficiently and working thoroughly, in this context safely, might enhance our understanding of safety critical decisions being made in complex healthcare situations.55 As these trade-offs reflect situations in which there is a conflict between, for example, production pressure and providing safe and high quality care, they may raise awareness to more system-related issues.

    After pretesting the prototype (B), questions and answer categories were reconsidered and rewritten in order to make it more compatible with the information gathered in SAE investigations. Test cases also gave a useful overview of the SAEs and helped to identify relevant characteristics, contributing factors and root causes related to each of the sociotechnical work system domains. Examples of these findings are listed in table 1.

    View this table:
    Table 1

    Description of the six interacting domains of sociotechnical work system, adopted from Holden et al31 and Carayon et al32

    The discussion with experts from practice (C) resulted in a few additional topics that were particularly relevant and therefore needed to be explicitly adopted (eg, the identification and classification of fragility among elderly patients). Also, more practical issues were put forward such as the description and categorisation of clinical areas and age groups. The boundaries of external factors, such as financial restrictions or labour shortages, were argued and established as well.

    The final test phase by potential users (D) brought up some important modifications of the questionnaire (eg, textual revision of questions, adjustment or addition of answer categories and merging some of the questions). This test session also presented important misinterpretations of a few questions and answer categories. Overall agreement between the potential users and researcher was acceptable, with exemption of the misinterpreted questions. We decided better instructions were needed and composed an explanation file and example case in which the ambiguities were explained to help future users in interpreting the questions and answer categories as intended.

    Analysis of the event reconstructions using the prototype helped in systematically considering all aspects of the sociotechnical work system. This resulted in a holistic analysis of the event. Mapping contributing factors and causes according to the prototype assisted in evaluating interactions and interdependencies between factors of various domains, addressing the complexity.

    Description of the final GAM

    The described process of developing, evaluating, testing and refining prototypes resulted in a final version of the GAM, consisting of a framework (figure 2) and an affiliated questionnaire (online supplemental appendix 1). Both follow a basic structure for analysing SAEs.

    Supplemental material

    Figure 2
    Figure 2

    GAM framework, based on the work of Smits, Langelaan, and De Groot43 and the SEIPS models31 32

    The first step of the model only intents to gather basic information on what happened. A suggestion of the most important elements that could be included is presented in the upper part of figure 2.

    The method then evaluates the six domains of the work systems according to the SEIPS model. Each domain includes a description of relevant characteristics and potential contributory factors. Interactions and interdependencies between domains and factors can be mapped to address and visualise complexity. Online supplemental appendix 2 provides an example of a case for which the work system was analysed using the GAM. In the tasks element, we included a question to get insight in unsafe human acts52 contributing to the event. For the tasks and organisational elements, trade-offs are evaluated between efficiency and thoroughness. Table 1 specifies the six domains of the work system with a more detailed description and provides some examples, based on the test sessions and previous case studies.

    Supplemental material

    After analysing the work systems and processes, the consequences of the SAE must be considered. It is important to assess the effects on patients and family, as well as caregivers, other professionals and the healthcare facility. Outcomes of SAEs can be multidimensional and may affect the physical, psychological, juridical, social and financial situation of all those involved. When evaluating the impact of SAEs, both immediate as well as distant outcomes must be considered. Formulated recommendations are incorporated in this part as well, so evaluation of the type(s) of recommendations as suggested by the hospital in response to the SAE is possible.

    Discussion

    Previous studies highlighted issues that hamper hospitals in their ambition to learn from SAEs. Methods used for analysing events, for example, have a strong focus on finding a linear root cause, thereby neglecting the complexity of healthcare.13 14 Hospitals also often study single events within their organisation, instead of analysing multiple events across organisations.13 Substantial variety in methods applied might complicate attempts to perform such aggregate cross-hospital analysis. Although potential solutions such as integrating human factors thinking in SAE analysis and using a more consistent analysis method have been proposed,10 18 20 41 there remains a need for methods embracing these possible improvements. This study aims to develop a novel GAM that stimulates a holistic analysis using a human factors perspective and eases aggregate analysis across hospitals.

    A GAM prototype was developed based on a human factors model and integrating important elements of three currently practised methods for analysing SAEs. The paper describes the multiple steps of carefully evaluating, testing and refining prototypes. Suggestions of theoretical experts to complement the prototype were incorporated. Pretesting, evaluation by experts from practice and user test sessions, helped to improve the usability and face and content validity. User test sessions also allowed us to roughly assess inter-rater agreement and think of manners to improve future reliability. These steps resulted in a final version of the GAM, consisting of a framework and affiliated questionnaire. These assist in gathering structured information about the event, evaluate all domains of the sociotechnical work system, map interactions between factors from various domains and assess outcomes of the SAE.

    The SEIPS model on which the GAM is based helps to focus beyond hierarchical, linear and causal relations and stimulates evaluation of complexity. To successfully unravel the complexity of healthcare work systems, it is essential to understand the interactions and interdependencies between the persons involved and the other domains of the sociotechnical work system (eg, technology, tasks, organisation and environment). As described by Holden et al,31 any analysis of the work systems should, therefore, include three important concepts: (1) the sociotechnical work systems must be viewed as dynamic, hierarchical and interactive, (2) all individuals engaged in a process or work system should be mapped systematically to reveal social structures and (3) analysis must depict how dynamic systems evolve over time in intended and unintended ways. As exemplified in online supplemental appendix 2, the GAM helps to evaluate the dynamics and interactions within a work system. By mapping all domains of the work system including the persons involved in a structured manner, it might be possible to depict the complex properties of SAEs. Whenever more SAEs are systematically mapped this way, underlying patterns of recurring interactions and structures might emerge. The inclusion of the trade-offs between efficiency and thoroughness,55 as adopted in the organisational and task elements of the GAM, could provide an opening to compare actual (work as done) and expected work practices (work as imagined). This could help to understand how work systems have evolved over time in intended and unintended ways.

    In addition to incorporating human factors thinking, GAM combines the narrative approach of SIRE47 with the more analytical approach of PRISMA-medical45 and Tripod beta46 on both organisational barriers and human-related issues. Integrating important elements of these methods helps to perform a holistic analysis. At the same time, the combination of existing methods in one generic method might ease aggregate analysis of SAEs across hospitals.

    As GAM is structured according to the widely used SEIPS model30 31 56 and built on methods that are currently applied by hospitals, we believe GAM may be easily adopted. Other promising methods that aim to improve learning from RCA such as the Human Factors Analysis and Classification System,57 AcciMap58 and Functional Resonance Analysis Method59 were not specifically build on the well-known SEIPS model and existing methods. Other methods may also need a specific practice to investigate SAEs or tend to focus specifically on a small and delineated process. This might make them less easy to adopt for SAE analysis. Existing initiatives for aggregate analysis, such as those performed in the USA and Australia37–39 and the preliminary attempts in the Netherlands,41 43 lack the important human factors perspective to guide analysis. Aggregate analysis using the GAM might therefore lead to better understanding of SAEs and could result in more effective system-aimed solutions. Besides retrospective analysis of patient safety events, methods such as the Health Care Failure Mode and Effect Analysis60 61 and Tripod Delta HC62 intent to prospectively identify (latent) risks for patient safety. These methods remain important and can be used complementary to retrospective aggregate analysis of SAEs using the GAM. Outcomes of retrospective GAM analysis might help to identify risks for such prospective analysis.

    In its most basic application, GAM can help to perform a structured, holistic analysis of SAE reports from various hospitals, irrespective of the initial RCA method used. The method is thus suited to study the existing ‘treasure’ of SAE reports that is stored in the archives of hospitals and unleash its potential to jointly learn from SAEs. Considering GAM was developed in the Netherlands and integrates methods Dutch hospitals use for SAE analysis, the generic method is particularly suited for application in the Dutch context. However, PRISMA-medical, SIRE and Tripod are not solely used by Dutch hospitals. In particular PRISMA-medical is applied by hospitals in various other countries as it has been accepted by the World Alliance for Patient Safety of the WHO.63–65 At the same time, we believe GAM is also appropriate to stimulate and improve aggregate analysis of SAEs for hospitals currently using other methods, such as cause and effect diagrams, five whys and fishbone diagrams. These methods are after all also based on the principle of investigating single SAEs and a search for a root cause. Application of the GAM might in this case be more time consuming though.

    Limitations

    A limitation of the study was the modest user test session, as it was limited to two cases. The process of testing was highly time consuming for potential users. Although we performed other testing and evaluation steps, including a pilot phase in which six more reports were analysed by the author, the number of cases in the user test session can be considered as a weakness. More extensive testing is therefore needed to further assess reliability and validity. Further testing might also provide leads to lean the method and improve its usability.

    Another issue we should address is that we tested the GAM by applying it to reanalyse information gathered in SAE reports. Although the reports provided a thorough description of the event, the investigation did not specifically study the event according to the GAM. This might have made it difficult to subtract new or other relevant factors from the reports than those already found in the initial RCA. The test sessions, however, indicated that adopting the GAM perspective to reanalyse information as included in the event reconstruction of the SAE reports can still lead to new insights. Directly studying an event using the GAM enables investigators to ask specific additional questions during the interviews with those involved on subjects that are only limitedly covered in traditional SAE investigations and analyses. Based on this study, we cannot draw strong conclusions on the usability of GAM for directly studying a SAE, but given its value in the analysis of SAE reports, the method seems promising for this purpose as well.

    Conclusion

    A novel GAM was carefully developed. On balance, the method is a promising tool for taking a next step in learning from SAEs. It facilitates a holistic analysis of the SAE and addresses the complex sociotechnical work system in which SAEs in healthcare originate. GAM could also ease aggregate analysis of SAE across hospitals. Using GAM might empower hospitals in their ambition to learn from SAEs and could help to formulate more effective system-aimed recommendations to prevent recurrence of events. The method can now be used to analyse a large sample of events to further explore its potential to improve patient safety.

    Data availability statement

    Data sharing not applicable as no datasets generated and/or analysed for this study. Not applicable.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Evaluated by the Medical Ethics Review Committee of VU University Medical Center (FWA00017598) and considered as not subject to the Medical Research Involving Human Subjects Act (WMO).

    Acknowledgments

    The authors would like to thank the professionals that participated in evaluating and testing the prototype. We would also like to thank the hospitals that shared their anonymised serious adverse event reports for test sessions.

    References

    1. ↵
      1. de Vries EN,
      2. Ramrattan MA,
      3. Smorenburg SM, et al
      . The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17:216–23.doi:10.1136/qshc.2007.023622pmid:http://www.ncbi.nlm.nih.gov/pubmed/18519629
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Jha AK,
      2. Larizgoitia I,
      3. Audera-Lopez C, et al
      . The global burden of unsafe medical care: Analytic modelling of observational studies. BMJ Qual Saf 2013;22:809–15.doi:10.1136/bmjqs-2012-001748pmid:http://www.ncbi.nlm.nih.gov/pubmed/24048616
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. David G,
      2. Gunnarsson CL,
      3. Waters HC, et al
      . Economic measurement of medical errors using a hospital claims database. Value Health 2013;16:305–10.doi:10.1016/j.jval.2012.11.010pmid:http://www.ncbi.nlm.nih.gov/pubmed/23538182
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Goodman JC,
      2. Villarreal P,
      3. Jones B
      . The social cost of adverse medical events, and what we can do about it. Health Aff 2011;30:590–5.doi:10.1377/hlthaff.2010.1256pmid:http://www.ncbi.nlm.nih.gov/pubmed/21471477
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Joint Commission Resources
      . Root cause analysis in health care : tools and techniques. Sixth edition. Oak Brook, IL: Joint Commission Resources, 2017.
    6. ↵
      1. Charles R,
      2. Hood B,
      3. Derosier JM, et al
      . How to perform a root cause analysis for workup and future prevention of medical errors: a review. Patient Saf Surg 2016;10: :20. doi:10.1186/s13037-016-0107-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/27688807
      OpenUrlPubMed
    7. ↵
      1. Williams PM
      . Techniques for root cause analysis. Baylor University Medical Center Proceedings 2001;14:154–7.doi:10.1080/08998280.2001.11927753pmid:http://www.ncbi.nlm.nih.gov/pubmed/16369607
      OpenUrlPubMed
    8. ↵
      1. Harel Z,
      2. Silver SA,
      3. McQuillan RF, et al
      . How to diagnose solutions to a quality of care problem. Clin J Am Soc Nephrol 2016;11:901–7.doi:10.2215/CJN.11481015pmid:http://www.ncbi.nlm.nih.gov/pubmed/27016495
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Nicolini D,
      2. Waring J,
      3. Mengis J
      . Policy and practice in the use of root cause analysis to investigate clinical adverse events: mind the gap. Soc Sci Med 2011;73:217–25.doi:10.1016/j.socscimed.2011.05.010pmid:http://www.ncbi.nlm.nih.gov/pubmed/21683494
      OpenUrlCrossRefPubMed
    10. ↵
      1. Karkhanis AJ,
      2. Thompson JM
      . Improving the effectiveness of root cause analysis in hospitals. Hosp Top 2021;99:1–14.doi:10.1080/00185868.2020.1824137pmid:http://www.ncbi.nlm.nih.gov/pubmed/32969765
      OpenUrlPubMed
    11. ↵
      1. Kellogg KM,
      2. Hettinger Z,
      3. Shah M, et al
      . Our current approach to root cause analysis: is it contributing to our failure to improve patient safety? BMJ Qual Saf 2017;26:381–7.doi:10.1136/bmjqs-2016-005991pmid:http://www.ncbi.nlm.nih.gov/pubmed/27940638
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Hollnagel E
      . Safety-I and safety–II: the past and future of safety management. Ashgate Publishing, Ltd, 2014.
    13. ↵
      1. Peerally MF,
      2. Carr S,
      3. Waring J, et al
      . The problem with root cause analysis. BMJ Qual Saf 2017;26:417–22.doi:10.1136/bmjqs-2016-005511pmid:http://www.ncbi.nlm.nih.gov/pubmed/27340202
      OpenUrlFREE Full Text
    14. ↵
      1. Wu AW,
      2. Lipshutz AKM,
      3. Pronovost PJ
      . Effectiveness and efficiency of root cause analysis in medicine. JAMA 2008;299:685–7.doi:10.1001/jama.299.6.685pmid:http://www.ncbi.nlm.nih.gov/pubmed/18270357
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Percarpio KB,
      2. Watts BV,
      3. Weeks WB
      . The effectiveness of root cause analysis: what does the literature tell us? Jt Comm J Qual Patient Saf 2008;34:391–8.doi:10.1016/S1553-7250(08)34049-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/18677870
      OpenUrlPubMed
    16. ↵
      1. Khunlertkit A,
      2. Paine L
      . A human factors approach for root cause analysis: a case of duplicate medical record number. Proceedings of the International Symposium on Human Factors and Ergonomics in HealthCare, 2015:156–61.
    17. ↵
      1. Henriksen K, et al
      . Understanding adverse events: a human factors framework, 2008.
    18. ↵
      1. Gosbee J,
      2. Anderson T
      . Human factors engineering design demonstrations can enlighten your RCA team. Qual Saf Health Care 2003;12:119–21.doi:10.1136/qhc.12.2.119pmid:http://www.ncbi.nlm.nih.gov/pubmed/12679508
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Canham A,
      2. Thomas Jun G,
      3. Waterson P, et al
      . Integrating systemic accident analysis into patient safety incident investigation practices. Appl Ergon 2018;72:1–9.doi:10.1016/j.apergo.2018.04.012pmid:http://www.ncbi.nlm.nih.gov/pubmed/29885719
      OpenUrlCrossRefPubMed
    20. ↵
      1. Gurses AP,
      2. Ozok AA,
      3. Pronovost PJ
      . Time to accelerate integration of human factors and ergonomics in patient safety. BMJ Qual Saf 2012;21:347–51.doi:10.1136/bmjqs-2011-000421pmid:http://www.ncbi.nlm.nih.gov/pubmed/22129929
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Dul J,
      2. Bruder R,
      3. Buckle P, et al
      . A strategy for human factors/ergonomics: developing the discipline and profession. Ergonomics 2012;55:377–95.doi:10.1080/00140139.2012.661087pmid:http://www.ncbi.nlm.nih.gov/pubmed/22332611
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Carayon P,
      2. Xie A,
      3. Kianfar S
      . Human factors and ergonomics as a patient safety practice. BMJ Qual Saf 2014;23:196–205.doi:10.1136/bmjqs-2013-001812pmid:http://www.ncbi.nlm.nih.gov/pubmed/23813211
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Association, I.E
      . Definition and Domains of Ergonomics. 22-01-2020]. Available: https://www.iea.cc/whats/
    24. ↵
      1. Siewert B,
      2. Hochman MG
      . Improving safety through human factors engineering. Radiographics 2015;35:1694–705.doi:10.1148/rg.2015150107pmid:http://www.ncbi.nlm.nih.gov/pubmed/26466179
      OpenUrlCrossRefPubMed
    25. ↵
      1. Borsci S,
      2. Uchegbu I,
      3. Buckle P, et al
      . Designing medical technology for resilience: integrating health economics and human factors approaches. Expert Rev Med Devices 2018;15:15–26.doi:10.1080/17434440.2018.1418661pmid:http://www.ncbi.nlm.nih.gov/pubmed/29243500
      OpenUrlPubMed
    26. ↵
      1. Xie A,
      2. Carayon P
      . A systematic review of human factors and ergonomics (HFE)-based healthcare system redesign for quality of care and patient safety. Ergonomics 2015;58:33–49.doi:10.1080/00140139.2014.959070pmid:http://www.ncbi.nlm.nih.gov/pubmed/25323570
      OpenUrlCrossRefPubMed
    27. ↵
      1. Kolodzey L,
      2. Trbovich P,
      3. Kashfi A, et al
      . System factors affecting intraoperative risk and resilience: applying a novel integrated approach to study surgical performance and patient safety. Ann Surg 2020;272:1164–70.doi:10.1097/SLA.0000000000003280pmid:http://www.ncbi.nlm.nih.gov/pubmed/30946083
      OpenUrlPubMed
    28. ↵
      1. Steele ML,
      2. Talley B,
      3. Frith KH
      . Application of the SEIPS model to analyze medication safety in a crisis residential center. Arch Psychiatr Nurs 2018;32:7–11.doi:10.1016/j.apnu.2017.09.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/29413076
      OpenUrlPubMed
    29. ↵
      1. Frith KH
      . Medication errors in the intensive care unit: literature review using the SEIPS model. AACN Adv Crit Care 2013;24:389–404.doi:10.1097/NCI.0b013e3182a8b516pmid:http://www.ncbi.nlm.nih.gov/pubmed/24153217
      OpenUrlAbstract/FREE Full Text
    30. ↵
      1. Carayon P,
      2. Schoofs Hundt A,
      3. Karsh B-T, et al
      . Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006;15 Suppl 1:i50–8.doi:10.1136/qshc.2005.015842pmid:http://www.ncbi.nlm.nih.gov/pubmed/17142610
      OpenUrlCrossRefPubMed
    31. ↵
      1. Holden RJ,
      2. Carayon P,
      3. Gurses AP, et al
      . SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013;56:1669–86.doi:10.1080/00140139.2013.838643pmid:http://www.ncbi.nlm.nih.gov/pubmed/24088063
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      1. Carayon P,
      2. Wooldridge A,
      3. Hoonakker P, et al
      . SEIPS 3.0: Human-centered design of the patient journey for patient safety. Appl Ergon 2020;84:103033. doi:10.1016/j.apergo.2019.103033pmid:http://www.ncbi.nlm.nih.gov/pubmed/31987516
      OpenUrlPubMed
    33. ↵
      1. Al-Jumaili AA,
      2. Doucette WR
      . A systems approach to identify factors influencing adverse drug events in nursing homes. J Am Geriatr Soc 2018;66:1420–7.doi:10.1111/jgs.15389pmid:http://www.ncbi.nlm.nih.gov/pubmed/29691843
      OpenUrlPubMed
    34. ↵
      1. Rivera AJ,
      2. Karsh B-T
      . Human factors and systems engineering approach to patient safety for radiotherapy. Int J Radiat Oncol Biol Phys 2008;71:S174–7.doi:10.1016/j.ijrobp.2007.06.088pmid:http://www.ncbi.nlm.nih.gov/pubmed/18406921
      OpenUrlCrossRefPubMed
    35. ↵
      1. Gurses AP,
      2. Kim G,
      3. Martinez EA, et al
      . Identifying and categorising patient safety hazards in cardiovascular operating rooms using an interdisciplinary approach: a multisite study. BMJ Qual Saf 2012;21:810–8.doi:10.1136/bmjqs-2011-000625pmid:http://www.ncbi.nlm.nih.gov/pubmed/22562873
      OpenUrlAbstract/FREE Full Text
    36. ↵
      1. Donabedian A
      . The quality of care. How can it be assessed? JAMA 1988;260:1743–8.doi:10.1001/jama.260.12.1743pmid:http://www.ncbi.nlm.nih.gov/pubmed/3045356
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Taitz J,
      2. Genn K,
      3. Brooks V, et al
      . System-Wide learning from root cause analysis: a report from the new South Wales root cause analysis review Committee. Qual Saf Health Care 2010;19:e63doi:10.1136/qshc.2008.032144pmid:20671073
      OpenUrlAbstract/FREE Full Text
    38. ↵
      1. Neily J,
      2. Ogrinc G,
      3. Mills P, et al
      . Using aggregate root cause analysis to improve patient safety. Jt Comm J Qual Saf 2003;29:434–9.doi:10.1016/S1549-3741(03)29052-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/12953608
      OpenUrlPubMed
    39. ↵
      1. Mills PD,
      2. Neily J,
      3. Luan D, et al
      . Using aggregate root cause analysis to reduce falls and related injuries. The Joint Commission Journal on Quality and Patient Safety 2005;31:21–31.doi:10.1016/S1553-7250(05)31004-X
      OpenUrl
    40. ↵
      1. Houssaini K, et al
      . Contributing factors to severe complications after liver resection: an aggregate root cause analysis in 105 consecutive patients. Patient Saf Surg 2020;14:1–15.
      OpenUrl
    41. ↵
      1. Hooker AB,
      2. Etman A,
      3. Westra M, et al
      . Aggregate analysis of sentinel events as a strategic tool in safety management can contribute to the improvement of healthcare safety. International Journal for Quality in Health Care 2019;31:110–6.doi:10.1093/intqhc/mzy116
      OpenUrl
    42. ↵
      1. Hagley GW,
      2. Mills PD,
      3. Shiner B, et al
      . An analysis of adverse events in the rehabilitation department: using the Veterans Affairs root cause analysis system. Phys Ther 2018;98:223–30.doi:10.1093/ptj/pzy003pmid:http://www.ncbi.nlm.nih.gov/pubmed/29325162
      OpenUrlPubMed
    43. ↵
      1. Smits M,
      2. Langelaan M,
      3. Groot Jd
      . Leren van calamiteiten in de ziekenhuiszorg: welke nieuwe inzichten komen naar boven bij Een ziekenhuisoverstijgende analyse van calamiteiten? 2018.
    44. ↵
      1. Bos K,
      2. Dongelmans DA,
      3. Greuters S, et al
      . The next step in learning from sentinel events in healthcare. BMJ Open Qual 2020;9: :e000739. doi:10.1136/bmjoq-2019-000739pmid:http://www.ncbi.nlm.nih.gov/pubmed/32098775
      OpenUrlAbstract/FREE Full Text
    45. ↵
      1. Van der Schaaf T,
      2. Habraken M
      . PRISMA-Medical: a brief description. Eindhoven University of technology, faculty of technology management, patient safety systems, 2005.
    46. ↵
      1. Tan H
      . Elk incident heeft Een context Het analyseren van Een incident heeft Het meeste effect ALS DIT de organisatie áchter Het voorval blootlegt. want menselijk falen heeft altijd Een context. Medisch Contact 2010;65:2290.
    47. ↵
      1. Leistikow IPDR,
      2. K.; De Vries B
      . Patiëntveiligheid: Systematische incident Reconstructie en Evaluatie. 2nd revised ed. Maarssen: Elsevier gezondheidszorg, 2009.
    48. ↵
      1. Snijders C,
      2. van der Schaaf TW,
      3. Klip H, et al
      . Feasibility and reliability of PRISMA-medical for specialty-based incident analysis. Qual Saf Health Care 2009;18:486–91.doi:10.1136/qshc.2008.028068pmid:http://www.ncbi.nlm.nih.gov/pubmed/19955462
      OpenUrlAbstract/FREE Full Text
    49. ↵
      1. Katsakiori P,
      2. Sakellaropoulos G,
      3. Manatakis E
      . Towards an evaluation of accident investigation methods in terms of their alignment with accident causation models. Saf Sci 2009;47:1007–15.doi:10.1016/j.ssci.2008.11.002
      OpenUrl
    50. ↵
      1. Wagner C,
      2. Merten H,
      3. Zwaan L, et al
      . Unit-based incident reporting and root cause analysis: variation at three hospital unit types. BMJ Open 2016;6:e011277. doi:10.1136/bmjopen-2016-011277pmid:http://www.ncbi.nlm.nih.gov/pubmed/27329443
      OpenUrlAbstract/FREE Full Text
    51. ↵
      1. Zwaan L,
      2. Thijs A,
      3. Wagner C, et al
      . Relating faults in diagnostic reasoning with diagnostic errors and patient harm. Acad Med 2012;87:149–56.doi:10.1097/ACM.0b013e31823f71e6pmid:http://www.ncbi.nlm.nih.gov/pubmed/22189886
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      1. Reason J
      . Human error: models and management. BMJ 2000;320:768–70.doi:10.1136/bmj.320.7237.768pmid:http://www.ncbi.nlm.nih.gov/pubmed/10720363
      OpenUrlFREE Full Text
    53. ↵
      1. Neale G,
      2. Woloshynowych M,
      3. Vincent C
      . Exploring the causes of adverse events in NHS hospital practice. J R Soc Med 2001;94:322–30.doi:10.1177/014107680109400702pmid:http://www.ncbi.nlm.nih.gov/pubmed/11418700
      OpenUrlCrossRefPubMedWeb of Science
    54. ↵
      Nederlandse Federatie van Universitair medische centra, N., Patiëntveiligheid, de handen ineen 2006:10–12.
    55. ↵
      1. Hollnagel E
      . The ETTO principle: Efficiency-Thoroughness trade-off. Londen: CRC press, 2009.
    56. ↵
      1. Holden RJ,
      2. Carayon P
      . SEIPS 101 and seven simple SEIPS tools. BMJ Quality & Safety 2021;30:901–10.doi:10.1136/bmjqs-2020-012538
      OpenUrl
    57. ↵
      1. Diller T,
      2. Helmrich G,
      3. Dunning S, et al
      . The human factors analysis classification system (HFACS) applied to health care. Am J Med Qual 2014;29:181–90.doi:10.1177/1062860613491623pmid:http://www.ncbi.nlm.nih.gov/pubmed/23814026
      OpenUrlCrossRefPubMed
    58. ↵
      1. Igene OO,
      2. Johnson CW,
      3. Long J
      . An evaluation of the formalised AcciMap approach for accident analysis in healthcare. Cognition, Technology & Work, 2021.
    59. ↵
      1. Patriarca R,
      2. Di Gravio G,
      3. Woltjer R, et al
      . Framing the FRAM: a literature review on the functional resonance analysis method. Saf Sci 2020;129:104827. doi:10.1016/j.ssci.2020.104827
    60. ↵
      1. Habraken MMP,
      2. Van der Schaaf TW,
      3. Leistikow IP, et al
      . Prospective risk analysis of health care processes: a systematic evaluation of the use of HFMEA in Dutch health care. Ergonomics 2009;52:809–19.doi:10.1080/00140130802578563pmid:http://www.ncbi.nlm.nih.gov/pubmed/19562591
      OpenUrlCrossRefPubMedWeb of Science
    61. ↵
      1. DeRosier J,
      2. Stalhandske E,
      3. Bagian JP, et al
      . Using health care failure mode and effect analysis: the Va national center for patient safety's prospective risk analysis system. Jt Comm J Qual Improv 2002;28:209:248–67. doi:10.1016/S1070-3241(02)28025-6pmid:http://www.ncbi.nlm.nih.gov/pubmed/12053459
      OpenUrlPubMed
    62. ↵
      1. van Schoten SM,
      2. Baines RJ,
      3. Spreeuwenberg P, et al
      . The ecometric properties of a measurement instrument for prospective risk analysis in hospital departments. BMC Health Serv Res 2014;14:103. doi:10.1186/1472-6963-14-103pmid:http://www.ncbi.nlm.nih.gov/pubmed/24589240
      OpenUrlPubMed
    63. ↵
      1. Fluitman KS,
      2. van Galen LS,
      3. Merten H, et al
      . Exploring the preventable causes of unplanned readmissions using root cause analysis: coordination of care is the weakest link. Eur J Intern Med 2016;30:18–24.doi:10.1016/j.ejim.2015.12.021pmid:http://www.ncbi.nlm.nih.gov/pubmed/26775179
      OpenUrlPubMed
    64. ↵
      1. Gaal S,
      2. Verstappen W,
      3. Wolters R, et al
      . Prevalence and consequences of patient safety incidents in general practice in the Netherlands: a retrospective medical record review study. Implement Sci 2011;6:37. doi:10.1186/1748-5908-6-37pmid:http://www.ncbi.nlm.nih.gov/pubmed/21470418
      OpenUrlCrossRefPubMed
    65. ↵
      1. Organization, W.H
      . World Alliance for patient safety: the conceptual framework of an international patient safety event classification. Copenhagen: WHO, 2006.

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors MCB, SMVS and CW jointly conceived the study concept and design. A first prototype of the analysis method was developed by MCB. SMVS and CW provided feedback on this prototype. MCB coordinated and conducted the data gathering process of evaluating, testing and refining prototypes, supported and supervised by SMVS and CW. MCB presented the prototype to the theoretical experts and performed the pretest session. During the meeting with the expert panel, MCB presented the prototype and discussed issues of face and content validity. CW participated in the discussion session. The double analysis in the user test session was performed by MCB, as well as the qualitative assessment of the inter-rater agreement. In all stages of the study, authors discussed findings and potential adaptations to the prototype. MCB wrote a first draft of the manuscript with input of SMVS. SMVS and CW gave feedback multiple times after critical revision of the manuscript. MCB is the guarantor and accepts responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.

    • Funding The study was funded by the Dutch Ministry of Health, Welfare and Sport. No grant/award number.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.