Article Text
PDF
Abstract
Introduction The UK Department of Health have targeted a reduction in stillbirth by 50% by 2025; to achieve this, the first version of the Saving Babies’ Lives Care Bundle (SBLCB) was developed by NHS England in 2016 to improve four key areas of antenatal and intrapartum care. Clinical practice guidelines are a key means by which quality improvement initiatives are disseminated to front-line staff.
Methods Seventy-five clinical practice guidelines covering the four areas of antenatal and intrapartum care in the first version of SBLCB were obtained from 19 maternity providers. The content and quality of guidelines were evaluated using the Appraisal of Guidelines for Research and Evaluation (AGREE II) tool. Maternity health professionals in participating organisations were invited to participate in an anonymous survey to determine perceptions toward and experiences of the use of clinical practice guidelines using a series of Likert scales.
Results Unit guidelines showed considerable variation in quality with median scores of 50%–58%. Only 4 (5.6%) guidelines were recommended for use in clinical practice without modifications, 54 (75.0%) were recommended for use subject to modifications and 12 (16.7%) were not recommended for use. The lowest scoring domains were ‘rigour of development’, ‘stakeholder involvement’ and ‘applicability’. A significant minority of unit guidelines omitted recommendations from national guidelines. The majority of staff believed that clinical practice guidelines standardised and improved the quality of care but over 30% had insufficient time to use them and 24% stated they were unable to implement recommendations.
Conclusion To successfully implement initiatives such as the SBLCB change is needed to local clinical practice guidelines to reduce variation in quality and to ensure they are consistent with national recommendations . In addition, to improve clinical practice, adequate time and resources need to be in place to deliver and evaluate care recommended in the SBLCB.
- obstetrics and gynaecology
- clinical practice guidelines
- healthcare quality improvement
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Background
In 2015, the stillbirth rate in the UK was higher than in many comparable high-income countries at 4.7 per 1000 live births after 24 weeks’ gestation.1 When late stillbirths (≥28 weeks’ gestation) were compared, the UK ranked 24th out of 49 high-income countries with 2.9 per 1000 live births2; the annual rate of reduction of stillbirth from 2000 to 2015 was 1.4%, placing the UK in the lowest third of high-income countries.2 Following the release of these figures, the UK Department of Health stated their aim to reduce stillbirths by 20% by 2020 and by 50% by 2025. To work toward this aim, the first version of the Saving Babies’ Lives Care Bundle (SBLCB) was launched by NHS England in March 2016, with early adopters implementing the programme from March 2015.
The SBLCB aims to reduce the incidence of stillbirth by improving the quality of maternity care and outcomes by improving care in four key areas: (1) Reducing smoking in pregnancy by carrying out carbon monoxide (CO) breath tests at antenatal booking appointment to identify smokers (or those exposed to tobacco smoke) and referring to stop smoking service/specialist as appropriate; (2) Risk assessment and surveillance of pregnancies for fetal growth restriction (FGR) by measurement of symphysis fundal height if low risk and serial ultrasound scans if at high risk; (3) Raising awareness among pregnant women about detecting and reporting reduced fetal movement (RFM), and ensuring providers have protocols in place, based on best available evidence, to manage care for women who report RFM; and (4) Effective fetal monitoring in labour by ensuring staff are trained and employing a buddy system for fetal monitoring in labour.3 The key recommendations of each component are in table 13. Due to synergies between the components, they were implemented together as a care bundle as this can provide better targeted solutions and have a bigger impact in terms of effectiveness than when components are instigated separately, as demonstrated by the necrotising enterocolitis care bundle.4 To achieve their aims, recommendations in care bundles need to be translated into alterations in local practice. Clinical guidelines are one means by which health professionals’ practice can be modified and improvements in care promulgated.
Agreement between unit guidelines and recommendations in the Saving Babies’ Lives Care Bundle (SBLCB).
Clinical Practice Guidelines
The Institute of Medicine defines clinical practice guidelines as ‘statements that include recommendations intended to optimise patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’.5 Evidence-based clinical practice guidelines can improve both the process and structure of care and can improve outcomes, although a systematic review noted significant variation in effect sizes of such improvements.6 Salient examples of improved outcomes in maternity care include implementation of guidelines for antenatal care,7 diagnosis of neonatal hypoglycaemia8 or sepsis.9 Conversely, reviews of perinatal deaths have identified that national clinical guidelines were not followed in a high proportion of cases.10–12 The role of clinical practice guidelines in maternity care is strongly supported by the UK Royal College of Obstetricians and Gynaecologists (RCOG) who emphasise that ‘optimal standards of clinical care will be achieved only by following national guidelines and through the quality of staff training and clinical research.’13 Importantly, clinical practice guidelines must be effectively implemented to lead to improvements in the quality of care; however, levels of implementation vary considerably. Factors which increase the likelihood of clinicians adhering to guidance include: evidence-based nature of the recommendations, recommendations which are not controversial or in agreement with the clinicians’ views, those with clear and specific recommendations and also that the practitioner has the time and resources to perform the recommended practice.14–16 However, several studies have demonstrated that guidelines in maternity care are often of low and variable quality in these domains.17–20 Consequently, variable implementation of clinical practice guidelines may underpin variations in clinical practice which would impair uptake of quality improvement initiatives such as SBLCB. This study was undertaken to describe the variation in content and quality of clinical practice guidelines relating to the SBLCB and to report staff views and experiences of using them. It was anticipated that this information would aid ongoing implementation of the SBLCB and other quality improvement initiatives in maternity care.
Methods
This study formed part of the Saving Babies’ Lives Project Impact and Results Evaluation (SPiRE) study carried out in 19 NHS Hospital Trusts in England.21 Maternity units encompassing early and late adopters of the SBLCB were identified in March 2015. Both secondary and tertiary maternity units were included. Ethical approval for the SPiRE study was obtained from Edgbaston Research Ethics Committee (17/WM/0197) and the Health Research Authority. The study was registered on www.clinicaltrials.gov (NCT03231007) and conducted between May 2016 and June 2018.
Patient and public involvement
As this was an analysis of clinical practice guidelines, it was not appropriate to involve patients and the participants in this aspect of the SPiRE study.
Assessment of guideline quality using AGREE II
Maternity guidelines in relation to each element of the SBLCB were requested from all 19 trusts between July 2017 and December 2017. Participating trusts were asked to submit their current clinical guidelines relating to the four components of the SBLCB: (1) Reducing smoking in pregnancy; (2) Risk assessment and surveillance of pregnancies for FGR; (3) Raising awareness among pregnant women about detecting and reporting reduced fetal movement (RFM) and (4) Effective fetal monitoring in labour. Guidelines that were under review at the time of request were excluded from the assessment. Guidelines that were currently in active use although the expiry date had passed were reviewed in the assessment.
Guidelines were reviewed and scored by two independent observers (from YZL, SK, GLS, SR, AEPH) using the AGREE II tool which has previously been employed to assess maternity guidelines.17–20 The tool comprises 23 items categorised into six domains: Scope and Purpose, Stakeholder Involvement, Rigour of Development, Clarity of Presentation, Applicability and Editorial Independence. A quality score between 1 and 7 was generated for each of the 23 items, with 7 being the highest possible quality, expressed as a percentage. In addition, each observer was asked to provide an overall score (from 1 to 7) for each guideline and whether they would recommend use of the guideline. The mean of the reviewer’s scores for the different components were used for subsequent analysis. Item 4 (from Stakeholder Involvement domain ‘The guideline development group includes individuals from all the relevant professional groups’) and item 13 (from Rigour of Development domain ‘The guideline has been externally reviewed by experts’) were excluded from the assessment as they were not relevant to local guideline development; scores were modified accordingly to the AGREE II criteria.22 Where maternity units submitted more than one guideline for an element (eg, intrapartum monitoring), the highest score was recorded. It was anticipated from an earlier analysis that Domain 6 ‘Editorial Independence’ would not be relevant to the NHS context,20 as this addresses personal financial interests of the authors, so a sensitivity analysis excluding this domain was conducted.
Assessment of guideline content related to SBLCB
The recommendations in the unit guidelines were compared against 12 recommendations in the SBLCB (3 for element 1, 5 for element 2, 2 for element 3 and 2 for element 4, table 1). For each SBLCB recommendation, a score of 2, 1 or 0 was assigned for fully, partially or not included in the unit guideline, respectively. For each element, a score was calculated by the sum of the score for each recommendation divided by the maximum possible score for each element, expressed as a percentage.
Staff survey of views regarding clinical guidelines
The methodology was described in the study protocol21; briefly, all health professionals involved in delivering maternity care (midwives on antenatal ward, antenatal clinic, community-based staff, antenatal assessment unit and labour ward, sonographers, junior doctors and consultant obstetricians, Clinical Directors and Heads of Midwifery) were invited to participate in an online or paper survey if they had been employed in their current Trust prior to the launch of the care bundle initiative in April 2015. The survey addressed a variety of staff experiences of implementing the SBLCB and staff views about the unit culture and experiences of the use of clinical practice guidelines in their maternity unit. The responses regarding staff perception and experiences of using clinical practice guidelines were analysed and presented in this manuscript. Agreement with statements was assessed using a series of Likert scales ranging from Strongly Agree to Strongly Disagree. Surveys were carried out anonymously and were completed between July and December 2017.
Analysis
To assess inter-rater variability of AGREE II scores, Fleiss’s kappa was calculated where there were three or more appraisers per guideline. Pearson’s correlation coefficient was used to assess the relationship between the overall AGREE II score assigned by the assessor (1–7) and the total of the individual domain scores.
If completed on paper, survey data were transcribed on to the online template. Staff survey results were downloaded from the online questionnaire (Selectsurvey.net) and processed in R (www.R-project.org). Descriptive statistics were used to summarise participant responses using frequencies and percentages.
Results
Response and characteristics of guidelines received
A total of 75 clinical practice guidelines were received from 19 trusts (figure 1). Seventeen units (89%) submitted a guideline for smoking cessation, 18 units (95%) submitted guidelines for FGR and RFM and all participating trusts returned clinical guidelines for intrapartum fetal monitoring. The majority of guidelines for FGR, RFM and intrapartum monitoring focused solely on the topic of the guideline, and this was not true for guidelines for CO monitoring and smoking cessation in pregnancy which were often contained within a generic antenatal care guideline. Only 8 trusts had stand-alone guidance about smoking cessation in pregnancy and 1 trust had a guideline specifically for ‘CO’ testing in pregnancy. Two trusts submitted more than one guideline for each element (eg, general antenatal care and smoking cessation as well as intrapartum monitoring for low-risk and high-risk pregnancies). Consequently, 72 guidelines were included in the final analysis, 6 of which (8%) were out of date and 7 (10%) had no review date (figure 1).
Flow chart demonstrating the number of clinical practice guidelines submitted for evaluation against the four elements of NHS England’s Saving Babies’ Lives Care Bundle. *Where units submitted more than one guideline related to each element (eg, element 4), the most relevant guideline was used (eg, continuous electronic fetal monitoring) for analysis. FGR, fetal growth restriction.
Guideline quality scores using AGREE II
There was significant variability in appraisers’ scores ranging from k=0.04 to k=0.39. However, the total domain scores and the independently assigned overall score were well correlated (r=0.78) indicating a good relationship between the overall score and the sum of the individual domains. For each element, the overall scores for the unit guidelines showed considerable variation (figure 2A), but the median overall scores for each element ranged from 50% (element 3) to 58% (element 1). There was no difference in the overall score between guidelines for different elements. Following review, only 4 (5.6%) guidelines were recommended for use in clinical practice without modifications and 54 (75.0%) were recommended for use subject to modifications usually making a clearer link with underpinning evidence and including an assessment of barriers/facilitators to implementation and audit standards (figure 2B). Twelve (16.7%) guidelines were not recommended for use in clinical practice for reasons such as recommended practice falling below national standards and significantly exceeding the expiry date which led to obsolete recommendations; six of these were for RFM.
(A) Box and whisker plots showing scores of each element of the Saving Babies’ Lives Care Bundle (SBLCB)—line=median, box=IQR range, error bars=range. (B) The proportion of guidelines relating to each element which were deemed suitable for use in clinical practice, suitable for use following modifications or not suitable for use. (C) The median scores for each domain within the AGREE II assessment framework for each element (error bars denote IQR). Element 1—smoking cessation, element 2—detection and management of fetal growth restriction, element 3—management of reduced fetal movements, and element 4—fetal monitoring in labour.
Some individual domains scored higher than others (figure 2C), ‘Scope and Purpose’ (domain 1) and ‘Clarity of Presentation’ (domain 4) received the highest scores for all four guideline categories with a median score of 80% or more. This was because the objective and target population of the guideline was frequently well described and guidelines were clearly presented. In contrast, ‘Rigour of development’ was lower for all four elements categories, with most having a score of less than 50%. This was due to source references being frequently omitted making it difficult to link recommendations with the supporting evidence. Although most included a review date, the procedures for updating the guidelines were often not provided. Stakeholder involvement (domain 2) also scored low (48%) likely reflecting inadequate evidence of consultation with service users to obtain their views and preferences during guideline development. Applicability (domain 5) also had a low score (40%) with most units failing to identify barriers to implementation and resources required to implement the guideline recommendation(s). Editorial independence (domain 6) had the lowest score as few guidelines reported whether their authors had any conflict of interests. A sensitivity analysis excluding this domain made no impact to the findings (data not shown).
Unit guideline recommendations and agreement to SBLCB
The 12 recommendations from the SBLCB are shown alongside the number of guidelines that either fully or partially included each recommendation in table 1. The first element was the most fully adopted within local guidelines, with the majority of guidelines containing all three SBLCB recommendations in element 1; CO testing of all pregnant women at antenatal booking was included in 76.4% of guidelines and referral to smoking cessation services with follow-up was included in 64.7% of guidelines. The second element (FGR) was rarely fully included in unit guidelines, but all contained a form of the SBLCB recommendations for the assessment of fetal growth using serial ultrasound scanning, estimated fetal weight derived from ultrasound measurements and measurement of symphysis fundal height. The risk-assessment algorithm for FGR was present in 5% of unit guidelines. Of particular concern was that none of the guidelines contained the SBLCB recommendation for ongoing audit and reporting for small-for-gestational age babies (which also relates to the low score for applicability in the AGREE II assessment). Recommendations for information giving for RFM (element 3) were less frequently present than the proposed management algorithm (33% vs 94%). Few guidelines for intrapartum fetal monitoring (32%) mentioned the need for annual training in cardiotocography (CTG) assessment and none fully included this recommendation, whereas a buddy system for CTG interpretation was included in all guidelines to some extent.
Staff opinions on the use of clinical guidelines
One thousand and sixty-four health professionals completed the survey across the 19 maternity providers (characteristics of participants are shown in online supplementary file 1). The number of responses ranged from 17 to 126 per institution. The majority of responses (78.0%) were from midwives, although responses were received from all relevant professional groups; 69% of respondents were involved in delivery of antenatal care. The median duration in respondents’ current role was 7 years with an IQR from 3 to 12 years. The majority of staff held positive views about guidelines, stating that guidelines were important for delivering high-quality consistent care to women (table 2). However, over 30% of staff said they did not have time to use to guidelines and 24% said they were not able to implement their recommendations, which in some cases was due to a lack of equipment (eg, CO monitors). Generally most staff found their guidelines readable although 16% said they are not easily accessible although this varied from 6% to 39% between different trusts.
Supplemental material
Staff opinions regarding the use of clinical guidelines in participating trusts
Discussion
Key findings
This study of maternity guidelines from 19 NHS providers found that staff generally held positive views about clinical guidelines and most units had clinical guidance covering all components of the SBLCB. However, the available guidance was of variable quality and often did not contain central recommendations from the SBLCB. The variation in content and quality did not affect all aspects of clinical guidance: the ‘scope and purpose’ and ‘accessibility’ domains scored significantly higher than ‘rigour of development’ and ‘applicability’. Most clinical guidance relating to the SBLCB required some modification to be optimised for clinical use.
Strengths and limitations
This study was strengthened by the study of a large number of guidelines for different areas of maternity care provision covering aspects of both antenatal and intrapartum care covered by the SBLCB. It applied the AGREE II tool, a standardised approach for evaluating clinical guidance. The use of this objective tool was helpful although the level of agreement between assessors varied and some domains, particularly ‘editorial independence’ were arguably less relevant in a state-funded healthcare system. When the scores were adjusted for the removal of domains perceived to be less relevant this did not alter the results. The study also combined assessment of clinical guidance with data from a range of maternity healthcare professionals from the participating organisations.
It is possible that studying guidelines from maternity units which included early adopters of the SBLCB is not representative of all UK maternity units. If this potential bias altered the findings, it would have been likely to overestimate the quality of guidelines as early adopters may have more robust quality improvement and governance programmes.
Context and implications
The findings regarding the quality of clinical practice guidelines relating to the SBLCB are largely in agreement with earlier analyses of guidelines within maternity care which show wide variation in guideline quality and lower average scores for stakeholder involvement (15%–86%), rigour of development (15%–88%) and the lowest scores for applicability (0%–61%).17 18 20 23 Similar to our findings, other evaluations conducted within the UK National Health Service and Australia found editorial independence was not reported in guidelines; thus, assessment of this domain is viewed as not as informative in state-funded healthcare systems.18 20 23 A review of WHO guidelines found that newer guidelines scored more highly17; we were only able to assess changes to clinical practice guidelines over time for one element of SBLCB. There has been improvement in guideline scores for RFM since 2013 with increasing average scores in scope and purpose (71% to 92%) and clarity of presentation (80% to 88%), but there was no improvement or deterioration in lower scoring domains such as stakeholder involvement (66% to 65%), rigour of development (64% to 47%) and applicability (38% to 39%).20 Efforts to implement the SBLCB should address these domains, particularly ensuring the national recommendations from the SBLCB are contained within the unit clinical practice guidelines and a link is made between the underpinning evidence and the recommendations.
The observed variation between practices recommended in local guidelines compared with national guidance may be because the latter is often not based on high-grade evidence. Analyses demonstrate that only 9%–12% of recommendations in guidelines from the RCOG and 33% of recommendations from the American College of Obstetricians and Gynecologists are based on Grade A evidence.24 25 Furthermore, a review of 1250 recommendations in 95 guidelines of the Society of Obstetricians and Gynaecologists of Canada found that 43% of recommendations had ‘good’ underpinning evidence, but only 57% of these were obtained from at least one randomised controlled trial.26 Thus, local guideline authors may draw different conclusions from those of the national guidance and consequently not reference the underpinning evidence which may account for the low scores within the ‘Rigour of development’ domain. To address this, local guidelines should clearly cite the evidence underpinning their recommendations. Furthermore, efforts to improve the quality of evidence informing maternity care must also continue.
While the majority of respondents held positive views about the role of clinical guidelines, this study identified system factors which may prevent effective implementation, including insufficient time to implement guidance, poor readability of guidelines, respondents unable to carry out all the recommendations in the guidance due to time and resources, and inaccessibility of guidelines. These challenges are not unique to the SBLCB, and similar barriers were identified in the implementation of national maternity guidance in Australia where approximately 15% of respondents had difficulty accessing guidelines and 12% were not able to perform the guideline recommendation due to a lack of training or confidence.23 Other studies evaluating the implementation of screening for gestational diabetes and implementation of smoking cessation guidelines both highlighted a lack of capacity and time, resource and funding as significant barriers to effective uptake of guidance.27 28 In our study, the comparatively low scores in the ‘Applicability’ domain were often due to lack of evidence that barriers and facilitators to implementation had been considered and/or that audit tools were not in place to check guidance was being followed. Feedback from clinical audit is essential tool to document performance and deviations from recommended practice; a study from New Zealand significantly improved compliance with a clinical practice guideline for fetal fibronectin testing by audit and training needs identified in the audit.29 This emphasises that from the outset unit guidelines should consider practical implications of how recommendations in the SBLCB can be implemented (eg, the number of CO monitors required and referral pathways for smoking cessation services for element 1), how barriers may be overcome (eg, training to have conversations about smoking cessation) and how practice can be evaluated and fed back to clinicians.
Finally, professionals’ views about guidelines and experiences of using them may influence their uptake.15 Our finding that staff largely held positive views about clinical practice guidelines is in agreement with a qualitative study of 591 maternity healthcare professionals about safety in maternity services which suggested that local implementation of national guidelines is an effective way of working and new guidelines should be rapidly disseminated.30 When disseminating clinical guidelines, it should be recognised that practitioners occasionally hold negative views about them reducing individual practitioner’s autonomy and being unsuited to the provision of individualised patient care.15 23 Mahran et al reviewed responses from 181 obstetricians at two UK maternity units and found that respondents from secondary care held more positive views about guidelines than those from a tertiary centre. As with our study, a significant proportion of staff were unable to follow clinical guidelines due to a lack of time and resource but this study also identified disagreement with the recommended practice as reason guidelines were not followed.31 This provides further evidence of the need to ensure guidelines are up to date, contain the best evidence available and that staff training should be included as an essential component of the dissemination of evidence-based guidelines. Senior staff and ‘champions’ (front-line staff with specific additional training) have important roles in promoting practice recommended by guidelines.32
Conclusion
Our findings and the findings of other studies indicate that change is needed to more effectively implement national clinical guidance in maternity care, such as that underpinning the SBLCB. In addition to improvements to the evidence base, the quality of clinical practice guidelines needs to be addressed, particularly that: (1) recommendations in local guidelines should reflect national standards and best available evidence, (2) staff and stakeholders should be involved in guideline development to ensure clarity and local applicability and (3) evaluation and audit of practice are set out in the clinical guidance. Staff value guidelines, but appropriate levels of time and resource are required to ensure that recommended practice can be delivered effectively28; these costs need to be considered when developing large-scale quality improvement initiatives. It is anticipated that using this approach to improve the quality of clinical practice guidelines and facilitate their implementation will increase the likelihood that recommended practice is implemented to deliver standardised and effective care.
Acknowledgments
The authors would like to acknowledge the assistance of Dr Elizabeth Camacho from the Manchester Centre for Health Economics, Division of Population Health, University of Manchester for her assistance with the design of the SPiRE study of which this analysis was embedded. We would like to thank the staff in the 19 Trusts who gave up their time to make this evaluation possible: Barnsley Hospital NHS Foundation Trust, Birmingham Women’s NHS Foundation Trust, Countess of Chester Hospital NHS Foundation Trust, Doncaster and Bassetlaw Hospitals NHS Foundation Trust, Gateshead Health NHS Foundation Trust, Liverpool Women’s NHS Foundation Trust, Manchester University NHS Foundation Trust, Norfolk and Norwich University Hospitals NHS Trust, North Cumbria University Hospitals NHS Trust, Oxford University Hospitals NHS Trust, Plymouth Hospital NHS Trust, Royal United Hospitals Bath NHS Foundation Trust, Sherwood Forest Hospitals NHS Foundation Trust, St Helens and Knowsley Teaching Hospitals NHS Trust, Taunton and Somerset NHS Foundation Trust, The Mid Yorkshire Hospitals NHS Trust, The Royal Devon & Exeter NHS Foundation Trust, University Hospitals of Morecambe Bay NHS Foundation Trust and the York Teaching Hospital NHS Foundation Trust.
References
Footnotes
Twitter @MCR_SB_Research
Contributors KW, SAR and AEPH designed the SPiRE evaluation and obtained funding. YZL, SK, GLS, SR and AEPH reviewed the clinical guidelines. KW, SAR and AEPH conducted the data analysis. The manuscript was prepared by YZL, KW and AEPH and edited by all authors.
Funding The SPiRE project was funded by NHS England. AEPH is Director of the Maternal and Fetal Health Research Centre funded by Tommy’s.
Disclaimer The funders had no role in the collection, data analysis, writing of or decision to present this manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement There is no ethical approval in place to share data from participating Trusts. Anonymised data from staff and patient questionnaires can be made available subject to appropriate confidentiality agreements between participating organisations. Investigators wishing to obtain data should contact the corresponding author AH.