Discussion

This review suggests that the facility-based individual maternal NMCR cycle may be an effective strategy for reducing maternal mortality in high-burden countries and for improving overall quality of maternal care in LMIC. Results of a pooled analysis of findings from eight studies showed that the NMCR cycle significantly reduced maternal mortality (OR 0.77, 95% CI 0.61 to 0.98, figure 2), with relatively low heterogeneity of results (I²=39%). Three out of six studies reported a significant reduction in the incidence of preventable obstetric complications such as uterine rupture, major postpartum haemorrhage and maternal sepsis. Out of ten studies reporting on the process of care when measured against predefined standards all showed some statistically significant improvement. Additionally, in all studies, the implementation of the NMCR cycle resulted in some amelioration in the structure of the hospital, such as an increased availability of essential equipment and supplies, additional training, monitoring and supervision and the implementation of new policies and better organisation of services.

Previous systematic reviews had observed a benefit of criterion-base audits in improving the quality of obstetric care.30–32 However, a review on the effectiveness of criterion-base audits in LMIC published some years ago concluded that, despite criterion-base audits being increasingly used, few studies had reported on their effectiveness.33 The present review retrieved all latest evidence on the effectiveness of NMCR cycle as a type of criterion-based audit, synthesised studies from LMIC in different geographical regions—including Africa, Central Asia, South East Asia, Latin America and Caribbean—and adds as new knowledge that this approach may be effective in reducing maternal mortality and in improving quality of healthcare provided.

Findings of this review are limited by the paucity of existing scientific literature: the NMCR approach has been used in many more countries than could be included in this reviews, such as China,34 India,35 South Africa36 and the WHO European Region,37–41 but scientific literature reporting on the NMCR effectiveness in these countries could not be retrieved. Second, all included studies had an UCBA or ITS design, thus being exposed to a high risk of bias (although most studies checked for potential confounding factors, such as the case mix in the before and after phase). Despite these limitations, this review collected an appreciable number of studies, including also some large studies,17 22 26 27 reporting on the impact of the NMCR cycle from different regions worldwide. Although quantitative findings of the review were to some extent affected by one large study,26 it must be acknowledged that results of most studies were in the same direction (figure 1) and in all studies some significant gains, either in the standards of care or in the process outcomes, were observed. In some studies, a significant benefit in maternal mortality or in standards of care could not be detected because in-hospital maternal mortality was too low18 20 or because standards of care were already good at the baseline.13 23 27 Ideally, it will be advisable to perform large multicentre RCTs to properly document NMCR effectiveness. However, in practice conducting a RCT on criterion-based audit alone may be challenging and may even be perceived as unethical, if no appropriate comparison is chosen. This is because in current practice criterion-based audits are already one of the recommended strategies to improve QoC promoted by many agencies and bodies, such as the National Institute for Clinical Excellence (NICE).42 Notably, the review of ‘near-miss’ cases is already recommended by WHO as a ‘key action to eliminate avoidable maternal and perinatal mortality and morbidity and improve the QoC,43 and as such it is already implemented in several countries.

The audit of maternal near-miss cases is an approach also used in several high-income settings: UK has a well-established programme of confidential enquiries into maternal deaths and a national system for research on maternal near-miss-the UK Obstetric Surveillance System (UKOSS);44 45 New Zealand established a national system for severe maternal morbidity review;46 several countries within the International Network of Obstetric Survey Systems (INOSS) are collecting data on severe maternal morbidities for study purposes,47 while other countries such as Italy (ITOSS) are starting the implementation of near-miss audits.48 49 Although there are some differences in the type of interventions applied (eg, not all of these approaches are facility-based), still the existence of these large networks on maternal NMCRs and the amount of resources devoted to them somehow testify the importance recognised in reviewing near-miss cases.

In the future, rather than investing resources in exploring whether near-miss audits or criterion-based audits in general are overall effective, it will be more interesting to explore which characteristics make them effective and sustainable. Available literature synthesised in this review does not allow for directly comparing the effectiveness of different methodologies on how to perform audits in practice, but at least it does provide some useful starting point for discussion and for future research. First, with regard to the number of cases audited, this varied largely in the included studies from a minimum of less than 10 cases per year18 20 to a maximum of several hundred cases in a few months,14 29 with a third approach consisting in performing a large retrospective review of past cases as the baseline and then collecting fewer new cases prospectively. When many cases were reviewed, this allowed for an in-depth description of the gaps in care. However, the analysis of a large number of cases does not necessarily ensure the development of good recommendations for quality improvement, neither their implementation. Additionally, the sustainability of auditing on a large number of cases, outside a research setting with dedicated human and economic resources, is questionable. Studies included in this review suggest that even the periodic review of few cases may help identifying gaps in routine care, developing SMART recommendations (ie, Specific, Measurable, Achievable, Realistic, Time-bound),50 and improving QoC significantly.18 20 WHO recommends to organise one session of NMCR per month and to review in each session few cases (one or two), but pretends a high quality in the process: each session should start by checking if previous recommendations have been implemented; there should be a in-depth discussion of the underlying causes of the near-miss event (‘why but why’ approach); recommendations should be SMART; regular sessions should be organised; dissemination of results should be ensured and so on.5 At first, few facilities should be selected for pilot implementation and the NMCR approach should be further scaled up only when quality in the process has been ensured.

Second, studies included in this review revealed that most experiences of implementation of NMCR cycles were externally supported, either by the WHO, academia and/or other development partners.15 18 20–24 26–28 This is in line with other existing literature51 52 highlighting that in particular the second part of the audit cycle (ie, developing recommendations, implementing them, checking on progress) is in general problematic and usually less well conducted compared with the first part of the audit cycle. The attitude to openly discuss cases within a multidisciplinary team and agreeing solutions was described as challenging in different settings, especially for mid-level staff (midwives, nurses) who may not be used to voice their views in the presence of doctors and managers.18 20 Hospital staff, managers included, often do not receive any formal training in quality improvement methods or any guidance in correctly performing an audit cycle. The need for ensuring sustained external support, and for establishing a functional quality assurance mechanism, is recognised by WHO as crucial for ensuring an effective NMCR implementation.5

Third, although having a single person appointed to perform the case review—as performed in some studies included in this review10 18 25 29—may increase feasibility, this actually largely reduces ownership of the process, together with minimising occasions for discussion and team building among staff. Studies noted that involvement of all healthcare providers in the audit process promoted successful implementation, ownership and sustainability of the process.14 20 28 The involvement of mid-level staff such as nurses and midwives was reported to result in improved staff autonomy and team work.14 21 27 Some studies observed that participation of the senior management promoted the implementation of recommendations that required allocation of resources and changes in policies and organisation of care.26 28 Currently, the WHO approach5 recommends the NMCR to be performed by the staff who managed the cases, including nurses, midwives and any other staff directly or indirectly involved in case management.

Fourth, the patient experience of care was assessed only in very few of the existing studies and yet not fully taken into account. In the last few years, WHO has given increasing importance to patient experience of care.1 Listening to women’s views may provide important information, as testified by studies in Brazil, Rwanda and the UK53–55 and by a study in Iran where women’s views were successfully used to improve QoC.56 Currently, WHO recommends to always interview women and their families and to use their inputs for improving care.5

Finally, as pointed out by authors of the included studies, interventions aiming at improving QoC without strengthening the health systems and improving community awareness may have minimal success.15 22 A study in Malawi reported that availability of essential supplies, such as blood for transfusions, remained low even after the NMCR, due to health system failures and this clearly was a barrier for improving case management.22 Qualitative findings, collected through focus groups among staff in a study in Uganda,15 pointed out, among issues that may have hampered the effectiveness of NMCR, health facility factors such as: stock-out of essential supplies, shortage of human resources, lack of task allocation, inadequate supervision. However, in most studies, even if the number of staff and available resources remained stable in the before and after phase, as a result of the audit, there was a reorganisation of staff activities, such as better specification of roles and responsibilities, task shifting and improved communication.14 16 17 20 28

Cost of the NMCR approach in improving health outcomes and QoC was not formally evaluated in the retrieved studies. However, several papers stated that the NMCR was an inexpensive and simple intervention, requiring little technology.24 26–28 A study involving 12 health centres in Malawi reported that each audit meeting cost about US$150, including foods and transport of participants to the District Hospital.27 Another study in Uganda stated ‘the audit process had challenged the assumption that all quality improvements need to be externally provided and are expensive’.28 These findings are in line with a systematic review of barriers and facilitators for effective NMCR implementation, reporting that a relatively low budget is needed to facilitate activities.37 In some experiences, the NMCR improved use or availability of existing economic resources: in Malawi, it ‘promoted a wiser allocation of resources for maternity care at the district level’;27 in Uganda, a fundraising committee was established to raise funds for drugs and equipment needed according to the recommendations.28