You are here

  • Home
  • Archive
  • Volume 11, Issue 3
  • Impact of a standardised rapid response system on clinical outcomes of female patients: an interrupted time series approach

Article Text

Article menu
Original research
Impact of a standardised rapid response system on clinical outcomes of female patients: an interrupted time series approach
  1. Jack Chen1,2,
  2. Lixin Ou1,2,
  3. Ken Hillman1,2,
  4. Michael Parr3,4,
  5. Arthas Flabouris5,6,
  6. Malcolm Green7
  1. 1The Simpson Centre for Health Services Research, University of New South Wales Faculty of Medicine, Sydney, New South Wales, Australia
  2. 2Ingham Institute for Applied Medical Research, Liverpool, New South Wales, Australia
  3. 3Intensive Care Unit, Liverpool Hospital, University of New South Wales Faculty of Medicine, Sydney, New South Wales, Australia
  4. 4Intensive Care Unit, Maquaire University Hospital, Maquaire University, Sydney, New South Wales, Australia
  5. 5Intensive Care Unit, Royal Adelaide Hospital, Adelaide, South Australia, Australia
  6. 6Faculty of Health Sciences, School of Medicine, University of Adelaide, Adelaide, South Australia, Australia
  7. 7Clinical Excellence Commission, Sydney, New South Wales, Australia
  1. Correspondence to Dr Jack Chen; jackchen{at}unsw.edu.au

Abstract

Background This study aimed to assess the impact of a standardised rapid response systems (the Between the Flags (BTF)) implemented across New South Wales (NSW), Australia, among female patients.

Methods We conducted an interrupted time series (2007–2013) population-based linkage study including 5 114 170 female patient (≥18 years old) admissions in all 232 public hospitals in NSW. We studied changes in levels and trends of patient outcomes after BTF implementation among four age groups of female patients.

Results Before the BTF system introduction (2007–2009), for the female patients as a whole, there was a progressive decrease in rates of in-hospital cardiopulmonary arrest (IHCA), IHCA-related mortality and hospital mortality for female patients. However, there were no changes in deaths in low-mortality diagnostic-related groups (DLMDRGs), IHCA survival to discharge and 1-year post-discharge mortality after surviving an IHCA. Only the female patients aged 55 years and older showed the same results as the whole sample. After the BTF programme (2010–2013), the same trends (except for DLMDRG) continued for female patients as a whole and for those aged 55 years or older. There was a significant reduction in DLMDRG among female patients aged 35–54 years (p<0.001), those aged 75 years and over (p<0.05) and female patients as a whole (p<0.05). The decreasing secular trend of surviving an IHCA to hospital discharge before the BTF system (p<0.05) among patients aged 18–34 years old was reversed after the BTF implementation (p<0.01).

Conclusions For female patients the BTF programme introduction was associated with continued reductions in the rates of IHCA, IHCA-related mortality and hospital mortality, as well as a new reduction in DLMDRG for 35–54 years old patients and those aged 75 years and older, and increased survival for those aged 18–34 years who had suffered an IHCA.

  • Medical emergency team
  • Patient safety
  • Critical care
  • Health services research
  • Performance measures

Data availability statement

Data are available upon reasonable request. As stipulated by the ethical requirement for the confidentiality of the data, the individual patient data are not available but data are available in the aggregated form upon reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2021-001614

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • There is inconsistent evidence of the effectiveness of rapid response systems (RRS) on reducing patient mortality and other adverse events for general adult inpatient populations. There was no published effectiveness evidence of RRS among different age groups of female inpatient populations.

    WHAT THIS STUDY ADDS

    • This study provided comprehensive evidence of the impact of a standardised RRS on different age groups of female populations.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • This study showed significant impact of a standardised RRS in a large jurisdiction in improving female patient outcomes. Such results should be confirmed by other research in different jurisdictions with different RRSs. The targeted research on the impact of obstetrical and maternal RRS is urgently needed.

    Background

    Rapid response systems (RRSs) that aim to provide timely intervention for deteriorating patients have been implemented in hospitals across the world. Systematic reviews provide support for RRS effectiveness1 2 while acknowledging methodological limitations in observational before and after studies. A recent large study showed the positive effect of implementing a standardised RRS (ie, the Between the Flags (BTF) programme) in Australia in reducing the incidences of in-hospital cardiopulmonary arrests (IHCA), IHCA-related mortality and the deaths in low-mortality diagnostic-related groups (DLMDRG).3 While the concept of obstetrical RRS has been introduced,4 no studies have explored the potential effects of implementing RRSs on the female patient. Evidence suggests that female patients are more likely to have worse outcome for certain conditions such as complex endovascular aortic repair,5 acute coronary syndrome,6 acute myocardial infarction7 8 when compared with male patients. Moreover, female patients were reported having significant worse outcomes9–11 including when in their reproductive age (ie, 18–49)12 in out-of-hospital cardiac arrests, than male patients. However, a recent systematic review13 also reported that female patients had better prognostic outcomes after suffering from IHCA.

    In 2010, the Clinical Excellence Commission (CEC) in New South Wales, Australia, developed standardised policies and procedures for deteriorating patients and implemented the statewide RRS system (BTF programme) in all 232 state public hospitals. The implementation process received support from government, health administrators and clinicians. This was the first standardised RRS implemented across a large health jurisdiction in the world, at the time. One of the very unique features of the BTF programme is that it had developed specific standardised charts for paediatric, maternity and emergency department patients, respectively.14 15 Moreover, as physiological clinical observations such as vital signs are different in pregnant women compared with non-pregnant women as are abnormal thresholds,15 the CEC developed its own obstetrical observation chart. Despite that the BTF was shown to have an impact on patient outcomes for whole study population,3 significant interaction effects across gender and age groups have been identified in the subsequent analyses.

    Given the different physiology among different women age groups, the lack of effectiveness evidence on a specific type of BTF (BTF in obstetrical patients) and the existence of significant gender interaction effect on BTF outcomes, in this subgroup study, we set out to investigate the impact of implementing an RRS on the outcomes of adult female patients, of different age groups.

    Methods

    Development and implementation of the BTF programme

    The detailed development and implementation process have been described previously.14 15 In short, the CEC developed a five-component strategy including: (1) a standardised documentation and response observation chart to be used in every hospital with criteria for defining two groups of deteriorating patients: (a) an at-risk but less urgent (‘yellow zone’) group and (b) an (urgent ‘red zone’) group (Supplemental Digital Content—online supplemental appendix 1); (2) a standardised response to deteriorating patients (including minimum skills of the responder and a minimum response time); (3) a governance structure with the chief executive officer of each institution responsible for implementing the programme; (4) an educational programme aimed at all hospital staff; and (5) a minimum data set to track the effectiveness of the programme. Preparation began in 2009 and the programme was launched in January 2010. The BTF programme released the New South Wales (NSW) Standard Maternity Observation Chart (SMOC, for women over 20 weeks gestation) in 2012 and cut-off calling criteria through a thorough literature review and nationwide consultation with clinical expert groups. The chart and calling criteria were piloted in hospitals prior to implementation across all NSW hospitals. The observation chart includes respiratory rate, SpO2%, oxygen requirement, systolic blood pressure, diastolic blood pressure, heart rate, neurological status (ACVPU: A=Alert, C=new confusion/change in behaviour, V=Rousable by voice (collected through Glasgow Coma Score (GCS)), P=Rousable only by pain (collected through GCS), U=Unresponsive), temperature, pain, cumulative blood loss value, blood glucose level and urinalysis results (online supplemental appendix 1).15

    Supplemental material

    Study design and sample

    We adopted a modified interrupted time series design(ITS)/segmented regression modelling approach16 to assess the change in patient outcomes before and after the implementation of BTF among adult women patients (>18 years old). We included all individual patient admission data in the analyses (in contrary to the conventional data management strategy to aggregate the outcome according to the time units). Such an approach enables the adjustment for both individual patient’s and ecological confounding variables in the final analytical model and also includes explicit modelling of seasonal effects and avoids the necessity in adjusting for autocorrelation as required in the aggregated data setting. All other aspects of the approach still follows the analytical principle and interpretation of interrupted time series design.16 17 The total number of public hospitals in the state of NSW, Australia (population: 7.3 millions), between 1 January 2007 and 31 December 2013 was 232. Patient outcomes and other related variables were derived from the NSW Admitted Patient Data Collection (APDC) database, which includes demographic and diagnostic information for each public and private hospital admission episode. All admissions to the study hospitals were linked to the NSW Registry of Births, Deaths, and Marriages through the Centre of Health Record Linkage, NSW Ministry of Health. This linked data made it possible to derive 1-year post-discharge mortality of patients who suffered an IHCA. This outcome is to ascertain whether there are unintended consequences of increased post-discharge mortality among female patients who suffered an IHCA.

    Patient and public involvement statement

    The design and analysis were also benefitted from the collaborations with related policymaking institutes (such as Clinical Excellence Commission of NSW) and their associated patient and consumer groups.

    Study outcomes

    The primary study outcomes were: (1) IHCA rate: the number of IHCA divided by total number of admissions (including same-day admissions); (2) IHCA-related mortality rate: the number of deaths among those patients who suffered an IHCA divided by the total number of admissions; (3) Hospital mortality rate: the number of hospital deaths divided by total number of hospital admissions; (4) Survival of IHCA to discharge: percentage of those whom survived an IHCA to discharge; (5) DLMDRG: the incidence of death in low-mortality diagnostic-related groups (LMDRG) per 1000 LMDRG admissions. As previously reported,18 the LMDRGs were defined by combining all patients admitted under a DRG with a mortality <0.5% in any of the previous 3 years18; (6) 1-year post-discharge mortality after surviving an IHCA: percentage of the deaths within 1 year after discharge alive from hospital among patients who suffered from IHCA.

    Death was defined as a patient documented as ‘deceased’ within the APDC database.

    A cardiopulmonary arrest was identified from the International Classification of Disease, V.10, Australian Modification (ICD 10-AM, V.5.0–V.5.1) and defined as a state of pulselessness (I.46) and/or cessation of breathing (R09.2) which required cardiac massage, defibrillation or artificial ventilation. A patient coded as I.46 or R09.2 in any of the 52 non-principal diagnostic fields, but not coded for these as the principal diagnostic field, was defined as having had a cardiopulmonary arrest during hospitalisation. This process aimed to differentiate patients who suffered an IHCA from patients admitted after an out-of-hospital cardiopulmonary arrest. NSW implemented the ICD-10-AM system in 1998. Each NSW public hospital has accredited coders who code data based on the patient charts. There were no changes for relevant diagnostic definitions and coding during the study period.

    Statistical analysis

    To evaluate changes in baseline characteristics by calendar year (grouped as before (2007–2009), run-in (2010) and after (2011–2013) the implementation of the BTF), we applied the Rao-Scott χ2 test which takes into account the hospital cluster effect. To assess the possible intervention effect of introduction of BTF, we used segmentation regression to estimate the monthly outcome trends before the programme (T1)and after the programme (T2), the change in the trend (∆T) after the BTF intervention and the immediate level change in outcome after the intervention (Int).16 Previously, we found significant interaction effects between gender and post-BTF trend among the total study population (n=9 799 081 admissions) (eg, p<0.001 for gender main effect and p=0.022 for the interaction effect between gender and post-BTF trend for hospital deaths). We also found significant interaction effects for age groups of female patients (ie, 18–34 years; 35–54 years; 55–74 years; ≥75 years). Thus, in the current study, we studied 5 114 170 female adult patient (>18 years old) admissions in all 232 public hospitals in NSW. We studied changes in trends for annual rates of all study outcomes before and after its introduction. We also explored the differential impacts of the BTF on four age groups of female patients. To assess changes in each outcome over calendar year, we derived an adjusted trend for each outcome variable including calendar year as a categorical variable (with 2007 as the baseline reference year). The time unit for the pre-BTF and post-BTF trends was specified as the consecutive month since the event. We specified a Poisson distribution to directly estimate rate ratios instead of ORs in the models.19 A Huber/White Sandwich estimator was used to account for hospital cluster effect for all regression models.20 In the adjusted model, we included year, age groups, marital status, country of birth, socio-economic status (SES) (based on the Socio-Economic Indexes for Areas developed by Australian Bureau of Statistics21), geographical area of hospitals (urban vs rural), private health insurance status and major hospital peer groups. We also included calendar months as indicator variables to adjust for potential seasonal effect. We examined baseline risk groups with the Elixhauser method and patient comorbidities with the Charlson Index based on ICD-10 coding.22 We did not include baseline risk groups and the Charlson Index in the adjusted model given recent reporting of potential biases introduced by these methods.23 The cases with missing covariate values were excluded from the final modelling. The proportion of cases excluded from the modelling was 17.9% for 1-year post-discharge mortality after surviving an IHCA and ranged between 6.3% and 8.6% for other five-study outcomes. We also provided related predictive ITS curves for those results with significant after-BTF trend changes.24 A p value of 0.05 was used as indicative of statistical significance. All the analyses were conducted using Stata V.16.0 (StataCorp, 2019, College Station, Texas, USA). The authors had full access to the data used in the analyses.

    Given that the current study results showed significant impact of BTF on reducing DLMDRG among 35–54 years old, 75 and older female patients, and increased IHCA survival of discharge among 18–34 years old female patients, we estimated the lives potentially saved by BTF for these outcomes based on a counterfactual base. We only estimated the lives saved between 2011 and 2013 as 2010 was a running-in year for all hospitals. The methodological details were presented in online supplemental appendix 2. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement checklist of items, extended from the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), was presented in online supplemental appendix 3.

    Supplemental material

    Supplemental material

    Results

    Patient demographic characteristics before and after BTF

    Overall, the study sample included 5 114 170 female patient admissions aged 18 years or older between 2007 and 2013 (table 1). The average age increased from 56.6 years (SD: 21.8) in 2007/2008 to 57.3 years (SD: 21.7) in 2009/2010 and 57.8 years (SD: 21.6) in 2011/2013. A detailed distribution of four age groups across the years was presented in figure 1. There were increased proportions over the time of patients aged over 55 years, patients never married or divorced/separated, patients born in a non-English speaking country, patients with a more advantageous SES and patients living in metropolitan area (table 1).

    View this table:
    Table 1

    The demographic characteristics of female patients before and after the implementation of the Between the Flags programme

    Figure 1
    Figure 1

    Age distributions of female patients across the study years (2007–2013).

    Unadjusted patient outcomes stratified by age groups and the study years

    We present the crude rates of six patient outcomes, stratified by age groups and the study years in table 2. We found significant increased risks with increased age for IHCA, IHCA-related mortality, 1-year post-discharge mortality after surviving an IHCA, DLMDRG and hospital mortality. There was also a decreased chance of surviving an IHCA to discharge with increased age. These significant age effects were consistent across the study years (2007–2013). There was significant improvement of patient outcomes between 2013 and 2007 with the magnitude of the improvement having a clear age gradient effect, but no improvement on DLMDRG among those aged less than 55 years old. For example, the reduction of IHCA incidence was a 3.8% in those aged 18–34 years, 41.3% in 35–54 years; 45.1% in 55–74 years and 56.1% in those 75 years and older.

    View this table:
    Table 2

    Patient outcomes by age groups over the study years

    The patient outcomes trends and its changes after the introduction of BTF (results from the interrupted time series segmented regression analysis)

    We presented historical trend estimates and the changes of the trend and levels after the implementation of BTF based on segmented regression modelling results (tables 3 and 4). Before the BTF system (2007–2009), for the female patients as a whole, there was a progressive decrease in rates of IHCA, IHCA-related mortality and hospital mortality, but no changes in DLMDRGs, IHCA survival to discharge and 1-year post-discharge mortality after surviving an IHCA. The female patients aged 55 years and older and the female patients as a whole showed the same results patterns for above outcomes. After BTF programme (2010–2013), the same trends (except for DLMDRG) continued for female patients as a whole and for those aged 55 years or older. However, post BTF implementation, there was a new reduction in DLMDRG among patients aged 35–54 years (p<0.001), patients aged 75 years and over (p<0.05) and patients as a whole (p<0.05) after the BTF. The decreasing trend (p<0.05) of surviving an IHCA to hospital discharge before the BTF system among patients aged 18–34 years was reversed after the BTF implementation (p<0.01). An ITS graph (figure 2) of predicted trends based on the modelling results for three significant outcomes were also presented that included pre-BTF trend, post-BTF trend and counterfactual trend (ie, what would have happened if the pre-BTF trend had continued).

    Figure 2
    Figure 2

    Predicted ITS trends (pre-BTF and post-BTF and counterfactual trends) for three significant patient outcomes. BTF, Between the Flags; DLMDRG, deaths in low-mortality diagnostic-related group; IHCA, in-hospital cardiopulmonary arrest.

    View this table:
    Table 3

    Interrupted time series segmented regression results for clinical outcomes of female patients before and after the implementation of the BTF programme (T1: 2007–2009; T2: 2010–2013)

    View this table:
    Table 4

    The outcome trends before the BTF and the trend changes after the BTF

    Estimated lives saved between 2011 and 2013 due to reduced DLMDRG and increased IHCA survival to discharge after the BTF implementation

    Overall, there were 1 736 469 LMDRG admissions between 2007 and 2013. The crude numbers and denominators of DLMDRG across age groups were presented in two supplementary tables (online supplemental tables S1,S2). The numbers of IHCA across age groups were presented in online supplemental appendix table S3. Between 2011 and 2013, the extra lives saved were 327 (27% of 1197 deaths of DLMDRG in 75 years and older, online supplemental tables S1-S2), 44 (86% of 51 deaths of DLMDRG in 35–54 years old) and 14 (10% of 140 IHCA deaths in 18–34 years old, online supplemental appendix table S3), respectively. The total extra lives saved amounted to 385 between 2011 and 2013 for the participating hospitals.

    Discussion

    Key findings

    Our study, of female only hospital patients, found significant progressive decreasing rates of IHCA, IHCA-related mortality and hospital mortality and no change in rates of IHCA survival to discharge and 1-year mortality post discharge after surviving an IHCA over the study period. In contrast, there was a reduction of DLMDRG after the implementation of the BTF among the whole female patient sample, in particular those aged 35–54 years old and patients aged 75 years or older. The declining survival in IHCA to hospital discharge before the BTF system among patients aged 18–34 years old was reversed after the BTF implementation. In a conservative estimate, 385 lives were saved due to reduction in DLMDRG and increased IHCA survival to hospital discharge between 2011 and 2013 among participating hospitals.

    Comparison with previous studies

    Our study results are consistent with the previous study that evaluated the BTF effectiveness on all adult patients,3 that is, the previous significant improvement of patient outcomes such as the rates of IHCA, IHCA-related mortality and overall hospital mortality had continued after universally adopting the BTF system in 2010 with a background of voluntary uptake an RRS from 31.7% (in 2002) to 74.4% (in 2009)25 among all public hospitals in NSW. At the end of the study period, the incidence rate of IHCA (ie, 0.98 per 1000 admissions) among female patients were among the lowest reported (most reported between 1 and 6 per 1000 admissions26 27). The incidence rates of IHCA survival to discharge (39.8%) were higher than the reported one-quarter in Australia and New Zealand27 and 1-year post discharge survival after IHCA in our study (ie, 34.8%=39.8%×87.5%) was also much higher than that (ie, 13.4%) reported in a recent systematic review including 40 studies.26 Our study showed that the before–after differences of these outcomes had an age gradient effect with the older age groups have larger magnitude of improvement. For example, our study has showed that the IHCA-related mortality has decreased by 25% in patients aged 18–34 years and by 62% in those aged 75 years and over. Our study has also provided detailed epidemiology on all these patient outcomes before and after implementation of an RRS and could be compared with other individual study or system reviews.1 2

    The life-saving impact on DLMDRG among those patients aged 35–54 years and patients aged 75 years and over have not been reported anywhere else. The rates of DLMDRG are potentially important as they represent a group of patients having simple diagnostics or procedures where serious adverse events would not be expected. Australian maternal deaths28 which are part of the DLMDRG were 8.5 per 100 000 for women having live births in 2016. Most of the leading causes of these deaths included non-obstetric haemorrhage, cardiovascular, thromboembolism and sepsis where deterioration would have most likely been preceded by deterioration of vital signs. Our study results have shown the positive reduction of the DLMDRG among one of the higher risk maternal death age group (ie, 35–54 years). In contrast, in a recent systematic review which included 17 studies examining 16 different early warning systems in obstetrics, only 1 before–after study29, which included 200 women who underwent emergency caesarean section in a developing country, assessed the impact on maternal deaths after implementing a National Early Warning Score (NEWS) system. The study recorded no maternal deaths both before and after the NEWS implementation given its relatively small sample size and short study period. The significantly improved IHCA hospital survival rate for those aged 18–34 years old female patients is also unique. This result may be due to the proactive nature of BTF that enabled earlier identification and treatment of potential deterioration of the female patients among this age group.

    Study implications

    Our study has shown a continuing improvement of the leading patient outcome indicators among female patients after the implementation of a standardised RRS at scale across a large health jurisdiction. Our study also showed significant differential age group effects both for the incidences of the patient outcomes and the treatment effects of BTF. The detailed epidemiology of these outcomes across the different age groups before and after the implementation of BTF provides important benchmarks for other large health jurisdictions. Moreover, our study finding of reduced unexpected deaths (ie, DLMDRG) raises important policy implications. The WHO estimated 303 000 maternal deaths occurred globally in 2015 at the end of the Millennium Development Goals year.30 Good quality care including timely identification and management of obstetrical complications can contribute to reducing the burden of material deaths and associated complications.31 As a result, modified early warning systems for the obstetric population have been recommended and implemented across the world.4 32–35 However, despite recommendations in implementing an obstetric early warning system (OEWS), there were no published studies demonstrating any effectiveness on reducing maternal deaths while most of the published studies focused on the evaluation of the different OEWSs for their predictive accuracies on different patient subpopulations.4 Interestingly, our study showed that a standardised non-obstetrics specific and two-tiered RRS has saved patient lives in a higher risk maternal death age group. Similar to other general RRSs, the OEWSs lack agreement on which vital signs, or combination of vital signs, are predictive of material deterioration during and after pregnancy.1 It is worth noting that the BTF programme released the NSW SMOC (for women over 20 weeks gestation) during 2012 but the exact implementation time for each participating hospital is yet to be confirmed. There is also a paucity of understanding regarding the normal maternal vital signs ranges for each stage of pregnancy, labour and the postpartum period.36 No prior study explored the incremental values of any OEWSs over the widely adopted RRSs.37 Rather than more complex scoring systems this study used a simple single criterion for triggering a response. Other RRSs need to be rigorously tested for their effectiveness.

    Our study findings that the BTF had also reduced risk of DLMDRG among those 75 years or older women patients were worth further investigation. It was well-known that older female patients were at higher risk of frailty38 and other comorbidities such as diabetes, hypertension or arthritis than men. Older women were also more likely to suffer from Alzheimer’s disease, depression, urinary tract infections39 and falls that may lend themselves more susceptible to further complications and mortality if their deteriorations were not identified and responded to earlier. Future studies may need to provide in-depth understanding on this group of patients.

    Strengths and limitations

    Our study employed the linked databases from a large health jurisdiction that was the first to implement a standardised rapid response system. To the best of our knowledge, this is the first large study of its kind to evaluate the impact of the rapid response system on women across a 7-year time span. Given the lack of other evidence on the differential effect of implementing an RRS on female patients, our study provides critical information on both the epidemiology and potential BTF impact on patient outcome indicators across different age groups. Our study also included a set of comprehensive patient outcome indicators including DLMDRG that capture deaths that were mostly likely to be unexpected thus reflecting potential benefits of an RRS. Our study finding of BTF being able to save female patient lives in the 35–54 years group may have important policy implications not only for developed countries but also for developing countries as 99% of maternal deaths occur in the low-income countries.4 30 Our study included 5 114 170 female patient admissions aged 18 years or older that enabled us to explore the age group differential effects in detail and increased precision for the patient outcome indicators based on rare events such as IHCA and DLMDRG.

    Our study has its limitations. First, the study data were derived from the NSW administrative data sets, and despite the use of professional and certified coders to extract chart data, the accuracy of the data extraction cannot be guaranteed. However, the administrative data, which were extracted by certified professional coders based on standardised guidelines at each hospital should minimise potential investigator biases. Second, the cardiopulmonary arrest codes employed in the current study were restrictive and specific but may miss some ‘cases’ in comparison to other broader range but less specific codes adopted by some studies based on ICD-9 codes and in other settings. Thus, the caution should be exercised when making strict point estimate comparisons between current study endpoints and other studies.40 However, the fact that our analyses were based on interrupted time series trends with total six endpoints including more reliable mortality-based outcomes may enhance the validity of our findings. Third, the current study is based on data from the state of NSW and might not be generalisable to other healthcare jurisdictions, however, we adopted a segmented regression approach using an interrupted time series design with the analyses and methodology repeatable in the future for both national and international level, offering a means for evaluating an RRS adjusted for the historical secular trend. Another limitation is that our data included only the patients between 2007 and 2013 and further study may be needed to include more up-to-date data to understand the most recent results. Our study set the significance level of p value at <0.05. Given the multiple tests conducted, some of the significant results should only be considered as preliminary and need confirmation from other studies.

    Conclusions

    The BTF programme was associated with progressive decrease in the rates of IHCA-related mortality and hospital mortality. In addition, it found a new and significant post-intervention reduction in DLMDRG among female 35–54 years old patients and those patients aged 75 years and older. The BTF reversed previous decreasing cardiac arrest survival rates among 18–34 years female patients. Conservatively estimated, the total extra lives saved between 2011 and 2013 for reduction in DLMDRG and increased IHCA survival amounted to 385 among participating hospitals. However, our study results were based on observational data and should be confirmed by other large and experimental studies.

    Data availability statement

    Data are available upon reasonable request. As stipulated by the ethical requirement for the confidentiality of the data, the individual patient data are not available but data are available in the aggregated form upon reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study involves human participants and was approved by The study was approved by the New South Wales (Australia) Population & Health Services Research Ethics Committee (HREC/13/CIPHS/12). The study used retrospective routinely collected administrative data.

    Acknowledgments

    The authors would like to acknowledge the Australian National Health and Medical Council for providing financial support for this research and the Southwestern Sydney Clinical School of Faculty of Medicine, the University of New South Wales, in providing all administrative and logistical support for the conduct of the research.

    References

    1. ↵
      1. Maharaj R,
      2. Raffaele I,
      3. Wendon J
      . Rapid response systems: a systematic review and meta-analysis. Crit Care 2015;19:254.doi:10.1186/s13054-015-0973-ypmid:http://www.ncbi.nlm.nih.gov/pubmed/26070457
      OpenUrlCrossRefPubMed
    2. ↵
      1. Winters BD,
      2. Weaver SJ,
      3. Pfoh ER, et al
      . Rapid-response systems as a patient safety strategy: a systematic review. Ann Intern Med 2013;158:417–25.doi:10.7326/0003-4819-158-5-201303051-00009pmid:http://www.ncbi.nlm.nih.gov/pubmed/23460099
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Chen J,
      2. Ou L,
      3. Flabouris A, et al
      . Impact of a standardized rapid response system on outcomes in a large healthcare jurisdiction. Resuscitation 2016;107:47–56.doi:10.1016/j.resuscitation.2016.07.240pmid:http://www.ncbi.nlm.nih.gov/pubmed/27507434
      OpenUrlPubMed
    4. ↵
      1. Umar A,
      2. Ameh CA,
      3. Muriithi F, et al
      . Early warning systems in obstetrics: a systematic literature review. PLoS One 2019;14:e0217864.doi:10.1371/journal.pone.0217864pmid:http://www.ncbi.nlm.nih.gov/pubmed/31150513
      OpenUrlCrossRefPubMed
    5. ↵
      1. Rieß HC,
      2. Debus ES,
      3. Schwaneberg T, et al
      . Gender disparities in fenestrated and branched endovascular aortic repair. Eur J Cardiothorac Surg 2019;55:338–44.doi:10.1093/ejcts/ezy249pmid:http://www.ncbi.nlm.nih.gov/pubmed/29982400
      OpenUrlPubMed
    6. ↵
      1. Poon S,
      2. Goodman SG,
      3. Yan RT, et al
      . Bridging the gender gap: insights from a contemporary analysis of sex-related differences in the treatment and outcomes of patients with acute coronary syndromes. Am Heart J 2012;163:66–73.doi:10.1016/j.ahj.2011.09.025pmid:http://www.ncbi.nlm.nih.gov/pubmed/22172438
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Bandyopadhyay D,
      2. Chakraborty S,
      3. Amgai B, et al
      . Acute myocardial infarction in the young - National trend analysis with gender-based difference in outcomes. Int J Cardiol 2020;301:21–8.doi:10.1016/j.ijcard.2019.11.096pmid:http://www.ncbi.nlm.nih.gov/pubmed/31757650
      OpenUrlPubMed
    8. ↵
      1. Bucholz EM,
      2. Strait KM,
      3. Dreyer RP, et al
      . Editor's choice-sex differences in young patients with acute myocardial infarction: a VIRGO study analysis. Eur Heart J Acute Cardiovasc Care 2017;6:610–22.doi:10.1177/2048872616661847pmid:http://www.ncbi.nlm.nih.gov/pubmed/27485141
      OpenUrlPubMed
    9. ↵
      1. Kramer CE,
      2. Wilkins MS,
      3. Davies JM, et al
      . Does the sex of a simulated patient affect CPR? Resuscitation 2015;86:82–7.doi:10.1016/j.resuscitation.2014.10.016pmid:http://www.ncbi.nlm.nih.gov/pubmed/25447437
      OpenUrlPubMed
    10. ↵
      1. Perman SM,
      2. Shelton SK,
      3. Knoepke C, et al
      . Public perceptions on why women receive less bystander cardiopulmonary resuscitation than men in out-of-hospital cardiac arrest. Circulation 2019;139:1060–8.doi:10.1161/CIRCULATIONAHA.118.037692pmid:http://www.ncbi.nlm.nih.gov/pubmed/30779655
      OpenUrlPubMed
    11. ↵
      1. Nehme Z,
      2. Andrew E,
      3. Bernard S, et al
      . Sex differences in the quality-of-life and functional outcome of cardiac arrest survivors. Resuscitation 2019;137:21–8.doi:10.1016/j.resuscitation.2019.01.034pmid:http://www.ncbi.nlm.nih.gov/pubmed/30735739
      OpenUrlPubMed
    12. ↵
      1. Hagihara A,
      2. Onozuka D,
      3. Hasegawa M, et al
      . Resuscitation outcomes of reproductive-age females who experienced out-of-hospital cardiac arrest. Eur Heart J Acute Cardiovasc Care 2017;6:121–9.doi:10.1177/2048872616633879pmid:http://www.ncbi.nlm.nih.gov/pubmed/26884619
      OpenUrlPubMed
    13. ↵
      1. Fernando SM,
      2. Tran A,
      3. Cheng W, et al
      . Pre-arrest and intra-arrest prognostic factors associated with survival after in-hospital cardiac arrest: systematic review and meta-analysis. BMJ 2019;367:l6373.doi:10.1136/bmj.l6373pmid:http://www.ncbi.nlm.nih.gov/pubmed/31801749
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. CEC
      . Between the flags project: the way forward CEC, 2008.
    15. ↵
      1. Hughes C,
      2. Pain C,
      3. Braithwaite J, et al
      . 'Between the flags': implementing a rapid response system at scale. BMJ Qual Saf 2014;23:714–7.doi:10.1136/bmjqs-2014-002845pmid:http://www.ncbi.nlm.nih.gov/pubmed/24740239
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Wagner AK,
      2. Soumerai SB,
      3. Zhang F, et al
      . Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther 2002;27:299–309.doi:10.1046/j.1365-2710.2002.00430.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/12174032
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Turner SL,
      2. Karahalios A,
      3. Forbes AB, et al
      . Design characteristics and statistical methods used in interrupted time series studies evaluating public health interventions: protocol for a review. BMJ Open 2019;9:e024096.doi:10.1136/bmjopen-2018-024096pmid:http://www.ncbi.nlm.nih.gov/pubmed/30696676
      OpenUrlAbstract/FREE Full Text
    18. ↵
      1. Barker AL,
      2. Brand CA,
      3. Evans SM, et al
      . "Death in low-mortality diagnosis-related groups": frequency, and the impact of patient and hospital characteristics. Med J Aust 2011;195:89–94.doi:10.5694/j.1326-5377.2011.tb03217.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/21770881
      OpenUrlPubMed
    19. ↵
      1. Zou G
      . A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004;159:702–6.doi:10.1093/aje/kwh090pmid:http://www.ncbi.nlm.nih.gov/pubmed/15033648
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. White H
      . A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 1980;48:817.doi:10.2307/1912934
      OpenUrlCrossRefWeb of Science
    21. ↵
      1. Pink B
      . Socio-Economic indexes for areas (SEIFA) 2011, 2011.
    22. ↵
      1. Quan H,
      2. Sundararajan V,
      3. Halfon P, et al
      . Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005;43:1130–9.doi:10.1097/01.mlr.0000182534.19832.83pmid:http://www.ncbi.nlm.nih.gov/pubmed/16224307
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Wennberg JE,
      2. Staiger DO,
      3. Sharp SM, et al
      . Observational intensity bias associated with illness adjustment: cross sectional analysis of insurance claims. BMJ 2013;346:f549.doi:10.1136/bmj.f549pmid:http://www.ncbi.nlm.nih.gov/pubmed/23430282
      OpenUrlAbstract/FREE Full Text
    24. ↵
      1. Turner SL,
      2. Karahalios A,
      3. Forbes AB, et al
      . Creating effective interrupted time series graphs: review and recommendations. Res Synth Methods 2021;12:106-117.doi:10.1002/jrsm.1435pmid:http://www.ncbi.nlm.nih.gov/pubmed/32657532
      OpenUrlPubMed
    25. ↵
      1. Chen J,
      2. Ou L,
      3. Hillman KM, et al
      . Cardiopulmonary arrest and mortality trends, and their association with rapid response system expansion. Med J Aust 2014;201:167–70.doi:10.5694/mja14.00019pmid:http://www.ncbi.nlm.nih.gov/pubmed/25128953
      OpenUrlCrossRefPubMed
    26. ↵
      1. Schluep M,
      2. Gravesteijn BY,
      3. Stolker RJ, et al
      . One-year survival after in-hospital cardiac arrest: a systematic review and meta-analysis. Resuscitation 2018;132:90–100.doi:10.1016/j.resuscitation.2018.09.001pmid:http://www.ncbi.nlm.nih.gov/pubmed/30213495
      OpenUrlCrossRefPubMed
    27. ↵
      1. Fennessy G,
      2. Hilton A,
      3. Radford S, et al
      . The epidemiology of in-hospital cardiac arrests in Australia and New Zealand. Intern Med J 2016;46:1172–81.doi:10.1111/imj.13039pmid:http://www.ncbi.nlm.nih.gov/pubmed/26865245
      OpenUrlPubMed
    28. ↵
      1. AIHW
      . Maternal Deaths in Australia. In: AIoHa W, ed, 2018.
    29. ↵
      1. Sheikh S,
      2. Qureshi R,
      3. Nausheen S, et al
      . Implementation of warning tool to improve maternal newborn health outcomes in a developing country. J Pak Med Assoc 2017;67:111–5.pmid:http://www.ncbi.nlm.nih.gov/pubmed/28065966
      OpenUrlPubMed
    30. ↵
      1. Alkema L,
      2. Chou D,
      3. Hogan D, et al
      . Global, regional, and national levels and trends in maternal mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the un maternal mortality estimation inter-agency group. Lancet 2016;387:462–74.doi:10.1016/S0140-6736(15)00838-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/26584737
      OpenUrlCrossRefPubMed
    31. ↵
      1. Machiyama K,
      2. Hirose A,
      3. Cresswell JA, et al
      . Consequences of maternal morbidity on health-related functioning: a systematic scoping review. BMJ Open 2017;7:e013903.doi:10.1136/bmjopen-2016-013903pmid:http://www.ncbi.nlm.nih.gov/pubmed/28667198
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Smith C,
      2. Brown J
      . Could pregnant Canadian women benefit from an early obstetric warning system? J Obstet Gynaecol Can 2018;40:397–8.doi:10.1016/j.jogc.2017.10.012pmid:http://www.ncbi.nlm.nih.gov/pubmed/29306682
      OpenUrlPubMed
    33. ↵
      1. Zuckerwise LC,
      2. Lipkind HS
      . Maternal early warning systems-towards reducing preventable maternal mortality and severe maternal morbidity through improved clinical surveillance and responsiveness. Semin Perinatol 2017;41:161–5.doi:10.1053/j.semperi.2017.03.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/28416176
      OpenUrlPubMed
    34. ↵
      1. Edwards SE,
      2. Grobman WA,
      3. Lappen JR
      . Modified obstetric early warning scoring systems (MOEWS): validating the diagnostic performance for severe sepsis in women with chorioamnionitis. Am J Obstet Gynecol 2015;212:e1–536.doi:10.1016/j.ajog.2014.11.007pmid:25446705
      OpenUrlCrossRefPubMed
    35. ↵
      1. Maguire PJ,
      2. O'Higgins A,
      3. Power K, et al
      . The Irish maternity early warning system (IMEWS). Ir Med J 2014;107:309.pmid:http://www.ncbi.nlm.nih.gov/pubmed/25551897
      OpenUrlPubMed
    36. ↵
      1. Smith GB,
      2. Isaacs R,
      3. Andrews L, et al
      . Vital signs and other observations used to detect deterioration in pregnant women: an analysis of vital sign charts in consultant-led UK maternity units. Int J Obstet Anesth 2017;30:44–51.doi:10.1016/j.ijoa.2017.03.002pmid:http://www.ncbi.nlm.nih.gov/pubmed/28385419
      OpenUrlPubMed
    37. ↵
      1. Chen J
      . In search of the 'best' rapid response early warning system - the journey has just begun. Resuscitation 2018;123:A1–2.doi:10.1016/j.resuscitation.2017.12.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/29242056
      OpenUrlPubMed
    38. ↵
      1. Gardiner PA,
      2. Mishra GD,
      3. Dobson AJ
      . The effect of socioeconomic status across adulthood on trajectories of frailty in older women. J Am Med Dir Assoc 2016;17:372.e1–372.e3.doi:10.1016/j.jamda.2015.12.090pmid:http://www.ncbi.nlm.nih.gov/pubmed/26837597
      OpenUrlPubMed
    39. ↵
      1. Mody L,
      2. Juthani-Mehta M
      . Urinary tract infections in older women: a clinical review. JAMA 2014;311:844–54.doi:10.1001/jama.2014.303pmid:http://www.ncbi.nlm.nih.gov/pubmed/24570248
      OpenUrlCrossRefPubMedWeb of Science
    40. ↵
      1. Khera R,
      2. Spertus JA,
      3. Starks MA, et al
      . Administrative codes for capturing in-hospital cardiac arrest. JAMA Cardiol 2017;2:1275–7.doi:10.1001/jamacardio.2017.2904pmid:http://www.ncbi.nlm.nih.gov/pubmed/28877294
      OpenUrlPubMed

    Supplementary materials

    Footnotes

    • Contributors JC, LO, KH, MP, AF and MG conceived and designed the study. JC and LO conceptualised the analytical framework, conducted data management and statistical analysis. JC provided the first draft of the manuscript and produced all the final tables and figures as well as the final draft for review. All contributed to final revised manuscript. All authors read and approved the final manuscript. JC is the guarantor of this work, has full access to all the data in the study and takes responsibility for its integrity and the accuracy of the data analysis.

    • Funding The study was funded by National Health and Medical Research Council (Australia) project grants (APP1133781; APP1020660). However, the funding body played no roles in the design, conduct and analysis of the study as well as in the publishing of the study results.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.