Article Text
PDF
Abstract
Background System factors in a regional Australian health district contributed to avoidable care deviations from invasive meningococcal disease (IMD) management guidelines. Traditional root cause analysis (RCA) is not well-suited to IMD, focusing on individual cases rather than system improvements. As IMD requires complex care across healthcare silos, it presents an opportunity to explore and address system-based patient safety issues.
Context Baseline assessment of IMD cases (2005–2006) identified inadequate triage, lack of senior clinician review, inconsistent vital sign recording and laboratory delays as common issues, resulting in antibiotic administration delays and inappropriate or premature discharge.
Methods Clinical governance, in partnership with clinical and public health services, established a multidisciplinary Meningococcal Reference Group (MRG) to routinely review management of all IMD cases. The MRG comprised representatives from primary care, acute care, public health, laboratory medicine and clinical governance. Baseline data were compared with two subsequent evaluation points (2011–2012 and 2013–2015).
Interventions Phase I involved multidisciplinary process mapping and development of a standardised audit tool from national IMD management guidelines. Phase II involved formalisation of group processes and advocacy for operational change. Phase III focused on dissemination of findings to clinicians and managers.
Results Greatest care improvements were observed in the final evaluation. Median antibiotic delay decreased from 72 to 42 min and proportion of cases triaged appropriately improved from 38% to 75% between 2013 and 2015. Increasing fatal outcomes were attributed to the emergence of more virulent meningococcal serotypes.
Conclusions The MRG was a key mechanism for identifying system gaps, advocating for change and enhancing communication and coordination across services. Employing IMD case review as a focus for district-level process reflection presents an innovative patient safety approach, combining the strengths of prospective hazard analysis with more traditional RCA methodologies.
- audit and feedback
- process mapping
- quality improvement
- root cause analysis
- governance
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Introduction
Problem description
Invasive meningococcal disease (IMD) is the most common infectious cause of childhood death in developed countries.1 Prompt recognition, timely administration of antibiotics and specialist care may be life-saving, however the early clinical signs of IMD may be subtle and easily missed.2 The management of IMD is therefore a unique marker of the quality and coordination of health services across clinical, laboratory and public health settings for potentially fatal systemic bacterial presentations.3 Failures in IMD management are an opportunity for prompting evaluation for system improvement in the management of critically ill patients, particularly sepsis.4
Available knowledge
In 2005, two high-profile deaths in children from IMD triggered a review by the New South Wales Clinical Excellence Commission (CEC).5 This review identified a number of system and administrative issues contributing to avoidable delays in the recognition and management of IMD. These issues included clinical handover and communication processes, triage processes in emergency departments, senior supervision and feedback of review findings to frontline staff.
Two key recommendations were: for fatal IMD cases to be subjected to root cause analysis (RCA) and for resourcing to be provided by New South Wales Department of Health to support the dissemination and implementation of best-practice guidelines and protocols for IMD management through Local Health District (LHD) Clinical Governance Units.5
RCA is a relatively recent entrant to the field of patient safety.6 It is generally triggered by a preventable healthcare-related event that leads to death or serious harm of a patient, such as surgery performed on a wrong limb. While the aspiration of identifying the underlying reasons for adverse events within the health system is noble, there are limitations in applying RCA as a patient safety tool. Recent criticisms of RCA include its potential to be affected by political sensitivities, the disaggregated nature of analysis precluding system-wide learning, lack of closure of feedback loops and failure to explore deep system problems due to time and resourcing constraints.7 8 Importantly, RCA is by nature reactive, triggered only after an incident has occurred. In meningococcal disease, the morbidity or mortality the patient suffers may not be considered preventable and therefore may not trigger an RCA investigation, with a lost opportunity for system learnings.5
In Australia, sentinel events are prospectively defined, require individual investigation and are nationally reportable on an annual basis.9 IMD fulfils many of the criteria of a sentinel event. It is an uncommon disease with significant associated morbidity and mortality and often generates significant community and media interest. In addition, the healthcare system required to manage IMD is complex. Potential alternative safety mitigation tools include prospective hazard analysis methodologies such as failure mode and effect analysis, key components of which include the assembly of a multidisciplinary team and the mapping of clinical processes to identify potential system failures, irrespective of the observed consequences.10
We describe the development of a systematic multidisciplinary approach to reviewing and reporting IMD guideline adherence at district level, reflecting a hybrid of the iterative process of prospective hazard analysis, informed by timely data from reviewing all cases of IMD as sentinel events. Our intervention commenced in 2007. The following report details the findings of evaluations undertaken in 2013 (Evaluation 1) and again in 2016 (Evaluation 2) compared with baseline findings.
Methods
Setting
This study took place at a large regional LHD in New South Wales, Australia. The LHD spans 131 785 square kilometres and services over 900 000 people, including over 51 000 Aboriginal and Torres Strait Islander people. It contains a major urban centre, several large regional centres and many smaller rural centres and remote communities in its jurisdiction, with 38 emergency departments ranging in delineation from level 1 (lowest complexity) to level 4 (major referral emergency department).11
Approximately 10 cases of IMD are reported per year. Despite the relatively low absolute numbers, this district experiences the highest age-standardised rates of IMD in the state, approximately 1.6 times higher than the state average.12 It is also home to the largest number of Aboriginal and Torres Strait Islander people in the state, a population identified to be at greater risk of developing IMD.13 14
Intervention
The Meningococcal Reference Group (MRG) developed from the work of a core group of senior clinicians from emergency medicine, public health and clinical governance in 2007. This initial group conducted a baseline audit of all cases of IMD notified to the LHD in 2005–2006 using a data collection form designed by emergency, paediatric, clinical microbiology and health protection specialists, in order to identify common or recurrent issues in management of cases.15 The baseline assessment found the following:
Triage level at emergency department presentation was a critical point determining the timeliness of patient management and review.
Lack of review by a senior clinician contributed to suboptimal management practices, for example, delays in antibiotic administration, inappropriate discharge.
The mean delay to administration of antibiotics was 58 min after health service presentation.
Inappropriate ordering of investigations (eg, serology with no blood culture or PCR) contributed to avoidable delays in diagnosing IMD.
Recording of patient vital signs was inconsistent and the adoption of a systematic approach may have improved recognition of early sepsis and prevented inappropriate discharge.
The suboptimal management practices observed contributed to a range of poor patient outcomes, including death in one case and permanent disability from multiple partial limb amputations in another.15
Based on these initial findings, the MRG was formalised in 2008, bringing together specialist representatives across the spectrum of IMD management (table 1). The group was formed under the mandate of clinical governance, reporting through its Director to the District Chief Executive. The primary aims of the group were to:
Meningococcal Reference Group membership
Regularly review the management of all cases of IMD against evidence-based state and national protocols and provide direct feedback to clinicians involved.
Develop, disseminate and implement local protocols and policies to support best practice in IMD care.
Phase I of the MRG involved the development and implementation of a structured approach to IMD case review, requiring consideration of the entire process of care. An evidence-based standardised template checklist was collaboratively designed in 2009 and further refined in 2013 (online supplementary appendix 1). This tool was developed with input from all relevant disciplines. In addition to basic demographic and clinical information, the tool mapped the entire episode of care, covering the patient journey from first presentation to outcome, including re-presentation data, triage category, initial vital signs, senior medical officer involvement, time from medical review to intravenous fluid and antibiotic administration, antibiotic choice, diagnostic testing and public health follow-up, including notification and management of close contacts.
In phase II, the group formalised its processes for case review and discussion. Original case notes for each IMD case were reviewed by a medical officer or public health officer to complete the standardised audit checklist. Audit checklists formed the basis of MRG review. The group met bimonthly via teleconference to determine, on a case-by-case basis, the appropriateness of documented management and identify any system or policy recommendations arising and which, if any, clinicians should receive feedback on the case. Where appropriate, specific clinician feedback, both positive and negative, was provided using a standardised letter template from the Clinical Governance Unit encouraging review of processes at the Ward or Unit level or commending excellence of care.
In phase III, strategies focused on improving clinician performance through advocacy for a number of operational changes to laboratory and ED processes, changes to standardised medical records and dissemination of findings through a peer-reviewed journal publication, in-service presentations to Emergency, Laboratory and Infectious Disease clinicians and continuing medical education for primary care clinicians.
The advocacy efforts of MRG members were also influential in the adoption by the LHD of two broader interventions in the recognition and management of undifferentiated sepsis. The first intervention was the ‘Between the Flags’ initiative, which was implemented by New South Wales Health in 2010, promoting the adoption of Standardised Observation Charts across adult, paediatric, maternity and emergency departments; the formation of Clinical Emergency Response Systems; education and training and the development of key performance indicators.16
The second intervention, ‘Sepsis Kills’, was a statewide initiative of the CEC which was implemented in the LHD during 2011–2012.17 The ‘Sepsis Kills’ intervention was a sepsis bundle of care, promoting improved recognition; appropriate testing (eg, collection of a point-of-care serum lactate and blood cultures), timely resuscitation with intravenous fluids and antibiotics; referral to appropriate senior clinicians and data collection into a centralised CEC Sepsis database.17
The MRG was evaluated twice during a 7-year period of operation from 2009 to 2015. The first evaluation (Evaluation 1), undertaken in 2013, reviewed the epidemiology and management of 24 IMD cases audited during 2011–2012 through descriptive analysis of quantitative data. Key informant interviews and document analysis were also conducted to identify strengths and possible areas for improvement of the review process.
A second evaluation (Evaluation 2) was conducted in 2016. This was a mixed-methods study, examining a case series of all notified IMD cases audited from 2013 to 2015 (n=29), comprising process review (documentation, meeting records, feedback letters, semistructured interviews with MRG members) and patient outcome review. Analysis was performed of key quantitative data from the 3-year dataset with a comparison dataset from the 2005–2006 baseline audit, using STATA V.14.
Results
From 2009 to 2015, the MRG met regularly on a bimonthly basis, adjusted for frequency of incident IMD cases. During this time, a total of 83 cases of IMD were notified. Care given to cases of IMD at services across the district was generally of acceptable quality by comparison with state clinical practice guidelines.18 Some aspects of IMD management significantly improved over time, both within the 2013–2015 period and across the three case series (baseline, Evaluation 1, Evaluation 2). The greatest improvements in documented care were observed from mid-2013 onwards.
Table 2 presents the findings for the 2013–2015 cohort. There were improvements in appropriateness of triage category at emergency department presentation, compliance with the sepsis bundle of care recommended actions, use of PCR testing to confirm the diagnosis and a decrease in the median time to antibiotics in minutes. However despite these improvements, the proportion of IMD cases meeting the benchmark of ‘antibiotic receipt within 60 min of medical review’ was just 4/8 (50%) in 2015. While the median laboratory turnaround time did not change over the 3 years, the maximum improved from 138 hours in 2013 to 57 hours in 2015, reflecting more consistent use of PCR and implementation of timelier laboratory testing.
Intracohort comparison of IMD management, cases notified to Local Health District, 2013–2015
Table 3 compares selected indicators over the three time periods spanning 9 years of intervention. There were improvements in the median time to antibiotics from medical review and in the proportion discharged and re-presenting to the emergency department prior to recognition. The proportion of clinicians notifying IMD cases to public health on clinical suspicion did not improve. However, nearly all cases were laboratory-notified to public health within 12 hours of laboratory confirmation.
Comparison of IMD management between baseline, first evaluation and second evaluation cohorts
The comparison of clinical outcomes over the extended period of time was confounded by an observable change in the epidemiology of IMD over the course of the evaluation period. Figure 1 shows the age distribution of notified cases of IMD, with a shift towards older age groups in the later cohorts, which is reflective of trends across New South Wales State as a whole.19 Table 3 shows that while the absolute number of IMD cases declined over time, the case fatality ratio was substantially higher in the final evaluation period. The responsible serogroup also changed, with a dramatic decline in Group C IMD (corresponding to the introduction of meningococcal C conjugate vaccine into the National Immunisation Program in 2003)20 and an increase in 2013–2015 in the proportion of IMD caused by Group Y and, in particular, W135 organisms.
IMD cases by age group over time. IMD, invasive meningococcal disease; LHD, Local Health District.
Serogroup was found to be a strong predictor of death in a pooled analysis of all LHD IMD cases from January 1998 to June 2016 (n=274) (table 4). The crude case fatality ratio (CFR) was higher for Group W135 and Group Y disease, both of which had an older age distribution. The observed increased OR for death in serogroup Y compared with serogroup B disease was not statistically significant and the effect measure was lost when adjusted for age category (0–4 years, 5–14 years and 15–49 years). The age-adjusted OR of death in serogroup W135 disease was 4.1 (P=0.06, 95% CI 0.9 to 17.8). This figure approached but did not achieve statistical significance, as the absolute case numbers were small.
Serogroup versus death, IMD cases notified in Hunter New England District, January 1998–July 2016
Comparison of clinical outcomes was limited by audit data incompleteness and inconsistent recording over time. The audit field capturing ‘disability on discharge’ was complete for only 11/29 cases, two of whom had disability recorded as ‘unknown’. Due to the striking effect of serogroup on mortality and the small number of cases for which detailed data were available, further quantitative analysis of the of effect of other potential determinants such as rurality, after-hours management, usage of the sepsis pathway and delay to antibiotics on patient outcomes was not attempted.
The collaborative and advocacy efforts of the MRG in reviewing cases led to a range of changes to local and statewide policy and operations, including:
Improvement of access to rapid meningococcal diagnostic tools, such as PCR testing around-the-clock (previously only available in-hours on certain days).
Adoption of standardised recording of vital signs and clinical escalation with deterioration as well as a wider suite of care related to sepsis recognition and early management into emergency department protocols and medical records.
Improvement of processes for accessing after-hours antibiotic chemoprophylaxis for the public health clearance treatment of close contacts of IMD cases.
Identification of conflicts between local and state protocols and harmonisation of pathways.
Creation of a secure, shared access computer drive for storing meeting records and checklists.
Dissemination of findings to key stakeholders through a peer-reviewed journal publication, in-service presentations to emergency department, laboratory and infectious disease clinicians and continuing medical education for primary care clinicians.
Participant satisfaction at both evaluation points was high. Reference Group members reported high levels of enthusiasm and engagement and a perception of the value of audit meetings. A common theme was the benefit of the meetings in increasing collaboration and networking and the uniqueness of IMD management as a framework for bringing together disciplines that would have little interaction in day-to-day work. Participants appreciated being exposed to the clinical process in its entirety and learning from the insights of colleagues in other disciplines. The seniority of participants and longevity of the process as well as the learning opportunities provided for more junior staff through participation were also cited as contributors to the value of the meetings. All participants expressed a desire for the group review process to continue.
External administrative support was considered a key enabler of the process and was essential for meeting and document coordination. While meeting by teleconference was viewed as convenient, this was also acknowledged to come at the expense of the engagement afforded by face-to-face meetings. When asked about potential improvements, multiple respondents reported that data completeness and analysis could have been improved by rationalisation of audit checklist fields to an essential set of indicators and the maintenance of a centralised database of cases.
Discussion
We found that MRG was an innovative and well-received method of prospective hazard analysis, using the systematic multidisciplinary audit of IMD cases as a nidus for iteratively reviewing clinical processes. The assembly of a multidisciplinary team of experts, a shared approach to identification and solving of potential problems with a systems focus and the dissemination of feedback both to individual clinicians and at district level were considered the most valuable aspects of the intervention. The systematic review of all IMD cases, irrespective of outcome, allowed for identification and monitoring of all aspects of care and its coordination across health services in the region, which are typically only scrutinised under circumstances of a patient safety investigation in other Australian settings.21 The repeated review of cases by a stable group of clinical experts produced rich and informative data that would not otherwise have been captured.
While there were high perceptions of value and satisfaction from participants, attendance and participation were frequently limited by time barriers and competing clinical duties. This is consistent with other evaluations of prospective hazard assessment approaches, which are typically time-intensive.10 22
The greatest observable improvements in IMD guideline compliance appeared from mid-2013 onwards. This corresponded with the embedding of more general sepsis pathways through integration into emergency department patient medical records, supported by improved access to point-of-care testing (eg, for serum lactate) across LHD facilities. The early signs of sepsis can be subtle, particularly in young people, and the use of standardised observation charts as the trigger for more timely or intensive review appeared to have contributed to greater involvement of senior clinicians and appropriateness of initial triage category over time in this series. Other measures more specific to IMD management, such as clinician notification of public health, were not embedded and did not improve. This supports a growing body of evidence that systems-based or technological changes are more effective than person-based approaches such as counselling or disciplinary action in mitigating patient safety.8 23 Importantly, the ‘Sepsis Kills’ programme stemmed from a cumulative review of RCAs at state level that recognised that a systematic patient safety programme was required, given the number of potentially preventable deaths.24
There were measurable changes over time in selected indicators of quality of care. Although the improvement in proportion receiving antibiotics within 60 min from 30% to 50% over 3 years may seem modest, it is comparable with the gains seen in similar studies of sepsis bundle introduction in Western Australia and on a national level.17 25 These studies showed improvements in antibiotic timeliness from 24% (95% CI 13% to 37%) to 44% (95% CI 30% to 60%) and from 29% to 52% (P<0.001).
Despite these improvements in care, a substantially higher case fatality ratio was observed in the second evaluation period than in the previous two case series. We attribute this to changes in the local epidemiological profile of IMD, with an increase in the proportion of cases caused by meningococcal serogroup W (MenW), affecting older age groups. The emergence of a hypervirulent strain of MenW, which is associated with less classical presentations (eg, gastroenteritis, septic arthritis) and rapidly progressive disease, has been observed in recent years in UK and South America26 27 and since 2013 in Australia.28 The changing epidemiology of IMD and the rise in atypical clinical presentations reinforces the importance of improving clinician recognition of undifferentiated sepsis in the early detection of IMD cases. Further work could be undertaken to extend sepsis initiatives to the Australian primary care sector, as is currently occurring in UK.29
Limitations
The primary limitation of our study was the small cohort size, precluding the use of non-parametric methods for statistical comparison of cohorts and the construction of a comprehensive statistical model that was predictive of outcome. However, it is encouraging that our findings of epidemiological trends and improvements in care are consistent with those of larger Australian and international studies.17 25 30
Another limitation is that the evaluations used an uncontrolled before-and-after design, which is prone to confounding and bias. This also led to other difficulties with comparison across cohorts, such as the definition of time zero for the calculation of antibiotic delay, which was inconsistently recorded on audit forms as time of patient arrival, time of medical officer review, and time of sepsis recognition. This has been identified as an issue in a recent study of paediatric sepsis initiatives30 and highlights the need for careful choice of consistent quality measures to enable comparability and meta-analysis of future evaluations. Another possible confounder in clinical outcome assessment was the changing age distribution of cases over time, driven both by vaccination policy (introduction of a government-funded Meningococcal C conjugate vaccine for infants since 2004, with a catch-up campaign for adolescents between 2005 and 2009)31 and the changing epidemiology of disease serogroups.
Finally, delayed presentation to health services is another factor which may affect clinical outcome of IMD. As the intervention was directed at improving quality of healthcare delivered, prehospital time to presentation was deemed to be out of scope and was not systematically captured on audit checklists. Public awareness of meningococcal disease, however, is likely to have remained relatively constant over the period of study and our finding of a higher case fatality ratio observed in cases of serogroup W is consistent with emerging genomic data suggesting that this is a hypervirulent strain.27 32 In May 2017, the New South Wales Government began offering a dose of 4-valent Meningococcal ACWY conjugate vaccine to older adolescents in its school vaccination programme.
Conclusion
MRG was a key governance mechanism for proactively identifying gaps in policy and operational processes, advocating for system change and enhancing communication and coordination across emergency, public health, laboratory and clinical governance services. Improved clinical compliance of IMD care with guidelines was most effectively achieved through embedding recommendations into clinical tools (eg, patient charts) and integrating these into a broader suite of measures for sepsis recognition and management. The methodology of using regular case review at district level as a focus for process reflection presents an innovative approach, combining the strengths of prospective hazard analysis with more traditional RCA methodologies as a means of improving patient safety.
References
Footnotes
Contributors KAT designed and conducted the evaluation, cleaned and analysed the data and drafted and revised the paper. CH, DND and TM initiated the project, monitored data collection and drafted and revised the paper. JF, NR and PM revised the draft paper.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.