You are here

  • Home
  • Archive
  • Volume 12, Issue 2
  • Effectiveness of quality improvement collaboratives in UK surgical settings and barriers and facilitators influencing their implementation: a systematic review and evidence synthesis

Article Text

Article menu

Download PDFPDF

Systematic review
Effectiveness of quality improvement collaboratives in UK surgical settings and barriers and facilitators influencing their implementation: a systematic review and evidence synthesis
  1. Eleanor Atkins1,2,
  2. Panagiota Birmpili1,2,
  3. Liz Glidewell3,
  4. Qiuju Li1,4,
  5. Amundeep S Johal1,
  6. Sam Waton1,
  7. Jon R Boyle5,
  8. Arun D Pherwani6,
  9. Ian Chetter2,
  10. David A Cromwell1,4
  1. 1Clinical Effectiveness Unit, Royal College of Surgeons of England, London, UK
  2. 2Hull York Medical School, Hull, UK
  3. 3University of York, York, UK
  4. 4Department of Health Services Research and Policy, School of Hygiene and Tropical Medicine, London, UK
  5. 5Department of Vascular Surgery, Cambridge University Hospitals, Cambridge, UK
  6. 6Department of Vascular Surgery, Royal Stoke University Hospital, Stoke-on-Trent, UK
  1. Correspondence to Eleanor Atkins; eleanor.atkins{at}nhs.net

Abstract

Background High-quality surgical care is vital to deliver the excellent outcomes patients deserve following surgical treatment. Quality improvement collaboratives (QICs) are based on a multicentre model for improving healthcare. They are increasingly used but their effectiveness in the context of surgical services is unclear. This review assessed effectiveness of QICs in National Health Service (NHS) surgical settings, and identified factors that influenced implementation.

Methods A systematic search of MEDLINE and EMBASE, as well as grey literature, was conducted in January 2022 to identify evaluations of QICs in NHS surgical settings. Data were extracted on the intervention, setting, study results and factors that were identified as facilitators or barriers. These were coded using the Consolidated Framework for Implementation Research (CFIR). The quality of study reports was assessed using Quality Improvement Minimum Criteria Set.

Results Fifteen reports on 10 QICs met inclusion criteria. The evaluations used study designs of different strength, with one using a stepped-wedge randomised controlled trial (RCT). Eight studies reported the QIC had been successful in achieving their principal aims, which covered a mix of patient outcomes and process indicators. The study based on the RCT found the QIC was not successful (no improvement in patient outcomes). Each article reported a range of facilitators and barriers to effectiveness of implementation of the QIC, which were spread across the CFIR domains (intervention, outer setting, inner setting, individuals and process). There were few barriers reported in the intervention domain that related to the QIC. There was no clear relationship between numbers of facilitators and barriers reported and effectiveness.

Conclusions Studies have reported QICs to be effective in increasingly complex contexts, but their results must be treated with caution. The evaluations often used weak study designs and the quality of reports was variable. Evaluation with strong study design should be integral to future QICs.

PROSPERO registration number CRD42022324970.

  • Implementation science
  • Surgery
  • Quality improvement

Data availability statement

Data are available upon request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2022-002241

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • There is increasing use of quality improvement collaboratives (QICs) to improve the process of care and the outcomes of patients who have surgery in the UK, but there is little evidence on the extent to which they are effective, or what features of a QIC may contribute to this. Previous reviews in different contexts have shown mixed results.

    WHAT THIS STUDY ADDS

    • This systematic review of studies evaluating QICs in the UK surgical setting uses well-evidenced theory, the Consolidated Framework for Implementation Research, to identify facilitators and barriers to effectiveness.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • The evidence synthesis highlighted that QICs can be effective in the UK surgical setting, but interpretation is hampered by weak study designs and a poor quality of reporting. The authors provide recommendations for future QICs in the surgical context, both for participants and QIC developers.

    Background

    Surgical care can always be improved. Time and money, as well as patient lives, can be wasted by not providing care of the best quality. In the National Health Service (NHS), quality is defined as care that is safe, effective and provides a positive experience for the people that need it.1 The General Medical Council states in Good Medical Practice that doctors have an overriding duty to take part in systems of quality assurance and improvement,2 and the four UK and Ireland Surgical Royal Colleges recommend surgeons are committed to quality improvement (QI) as a core part of clinical duties.3 There are many different approaches used to improve quality in healthcare, and the evidence for these techniques continues to evolve, with none as yet being recognised as superior.4

    QI collaboratives (QICs) are one such approach. They are based on a multiorganisational model for shared learning in order to improve patient outcomes and have been used in medicine and surgery since the late 1980s.5 Evaluations of their effectiveness have reported mixed results, but this has not prevented them being adopted worldwide.6 The majority of QICs described in the literature have been implemented in medical specialties, and few are from the UK.7 Wells et al suggested in their review that ‘collaboratives reporting success generally addressed relatively straightforward aspects of care’.7 Delivery of surgical care is an example of a moderately complex process of care8 and examining the effectiveness of QICs in a surgical setting could provide insight into whether and how collaboratives could be effective beyond simple care processes.

    Previous systematic reviews have found effectiveness of QICs is highly dependent on context, which is typically defined as ‘anything external to the intervention that may act as a barrier or facilitator to its implementation, or its effects’.9 10 Context can be modified by factors related to the healthcare setting, the project itself and organisational characteristics.11 12 The procedural aspect of the surgical specialties leads to a specific context for QI in surgery, with care pathways needing to negotiate the complex tension of managing capacity when having to provide rapid access to theatres alongside planned activity.

    There is a burgeoning trend of QICs being carried out in UK surgery, and in the current resource-limited environment, exploration of whether ongoing investment should be made in this QI approach is warranted. The aim of this review was to examine the evidence on whether QICs are effective in improving the delivery of surgical services in the UK and to explore facilitators and barriers to effective implementation of QICs. Focus on a single healthcare system aimed to limit the degree of heterogeneity in the care process, given the importance of context in influencing the success and failure of QICs.

    Methods

    Search strategy

    This systematic review was registered with PROSPERO (CRD42022324970) and the protocol prepared using the Cochrane Effective Practice and Organisation of Care’s (EPOC) Protocol and Review Templates for Intervention Reviews and Qualitative Evidence Synthesis.13 14 The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist can be seen in online supplemental additional file 1. Relevant reports were identified by searching two databases (MEDLINE and EMBASE) from inception to 7 January 2022. Search strategies were developed with the help of an information specialist and contained Medical Subject Headings and keywords related to “surgery”, “UK” and “quality improvement collaborative”. These can be seen in online supplemental additional file 2. Grey literature searches were carried out on www.opengrey.eu, www.pdq-evidence.org and www.epistemonikos.org. Reports written for the Health Foundation ‘Scaling up’, ‘Closing the Gap’ and ‘Spreading Improvement’ programmes were reviewed for inclusion. Reference lists of all included studies were screened for additional studies that would merit inclusion, as were relevant systematic reviews.

    Supplemental material

    Supplemental material

    Inclusion criteria and study selection

    Our definition of a QIC was a prospectively planned QI project with the involvement of a number of sites over a specified time period, which had a defined patient group who received surgical treatment and a defined set of improvement outcomes. Studies involving any surgical specialty, carried out in the NHS, with the same expert team leading the project across multiple sites were eligible for inclusion. Other common features of QICs described in the literature are listed in table 1.9 11 The study used broad inclusion criteria in order to incorporate multicentre QI programmes, which deviated from previous descriptions of QIC in the literature9 11 but retained the spirit of collaboration, to reflect real life practice. The review excluded studies which evaluated the comparative effectiveness of different devices or surgical interventions, or education/training programmes for surgical staff. The review focused on primary evaluations and excluded conference abstracts, reviews, editorials and guidelines. Two members of the review team independently screened the titles and abstracts to determine suitability for full text review. Full texts of potentially eligible reports were then obtained, and independently assessed against inclusion criteria by the same two reviewers. Any disagreements were resolved by discussion, with a third reviewer involved when required, to determine inclusion.

    View this table:
    Table 1

    Key features for definition of a quality improvement collaborative

    Data extraction and quality assessment

    Data were recorded in a previously piloted Microsoft Excel spreadsheet template. One reviewer extracted data from reports concerning half of the QICs, and one reviewer extracted data from the other half. Each then reviewed and checked the other’s extractions for accuracy. Any discrepancies were resolved by discussion, with a third reviewer consulted when required. Data extracted on the study and the attributes of the QIC included: study aim, study design, process and outcome measures, the pre-existing care pathway, details of the structures of the QI intervention, the planned QI processes and those that actually took place, the intervention outcomes and the effectiveness barriers and facilitators. Each quantitative or mixed-methods report including quantitative analysis was scored against the QI-Minimum Criteria Set (QI-MCS), a tool for critical appraisal of QI publications which scores quality on a scale from 0 (poor quality) to 16 (high quality).15

    Data synthesis

    The effectiveness of each included QIC was assessed according to the aim of the study. A QIC was deemed effective if there was an improvement in a process or patient outcome indicator that aligned with specified study objectives. Due to heterogeneity of indicators of effectiveness, meta-analysis was not possible.

    In order to explore reasons for the success or failure in achieving the collaborative objectives, the Consolidated Framework for Implementation Research (CFIR)16 was used. All reports relating to each QIC, including quantitative, qualitative and mixed-methods reports, were coded, and factors could be suggested by study authors or identified by participants as part of a qualitative process. The CFIR is a collection of constructs grouped within five domains (intervention, outer setting, inner setting, individuals and process) that influence implementation of interventions. For every study, each construct of the CFIR was coded as +1 (facilitator), −1 (barrier), 0 (neither facilitator nor barrier) or X (both a facilitator and barrier).17 It has been previously noted that the CFIR does not contain constructs relevant to teams, which are an important part of QICs. For this reason, Rogers et al’s constructs related to teams18 were used alongside the CFIR in our coding strategy. This could be related to either the expert team, or the participating site team. Factors related to the effectiveness of implementation presented in each report were mapped to individual CFIR constructs. As before, one reviewer coded the reports concerning half of the QICs, and one reviewer coded the other half. Each reviewer then reviewed the other’s coding for accuracy. Any discrepancies were resolved by consensus, with a third reviewer when required. All reviewers are experienced in QI and the two primary coders have a clinical background.

    Results

    The search was conducted on 7 January 2022 and retrieved 823 unique citations, which underwent screening. Forty citations and three reports identified in grey literature searches were retrieved for full-text review. Following full-text assessment, 15 reports were selected for inclusion in the review. Those not included are listed in online supplemental additional file 3. Figure 1 demonstrates the PRISMA flow chart.

    Supplemental material

    Figure 1
    Figure 1

    PRISMA flow diagram.

    Study characteristics

    The 15 selected reports described 10 QICs in UK surgical contexts. Table 2 gives a summary of the included studies, presented according to quality of study design, then type of surgery (elective or emergency). The primary aim of five QICs was to improve patient outcome indicators such as mortality or length of stay.19–23 Four QICs had primary aims to improve process indicators: two related to time to surgery,24 25 and one related to having a booked ‘to come in’ date for surgery26 and one looking at implementation of a monitoring device.27 The one remaining QIC had a broad aim of ‘providing… care of the highest quality’,28 and looked for changes in a set of patient outcome and process measures. All studies reported process indicators, with eight reporting patient outcome indicators19–24 27 28 (table 2).

    View this table:
    Table 2

    Summary of included QICs

    There was heterogeneity in the design of the studies. Four used uncontrolled before-and-after designs,19 20 24 27 with a historical cohort acting as a comparison group. Four studies used an interrupted time series design, with three using population-level data as a control23 26 28 and one uncontrolled.21 One study used a controlled cohort evaluation design, again using population-level data (excluding participant sites) as a control,25 and one was a stepped-wedge cluster randomised controlled trial (RCT).22 Four studies met criteria for inclusion in a Cochrane Effective Practice and Organisation of Care (EPOC)29 review.21–23 25 Nine studies documented the duration of the QIC intervention which ranged from 5 weeks (the shortest duration of the stepped-wedge cluster RCT) to 2 years. The numbers of centres included in the collaboratives ranged from 3 to 93. The number of expert team-led sessions varied from 2 to 16 across the QICs. QI methods used included Plan–Do–Study–Act cycles,19–22 28 lean-based QI methodology,24 driver diagrams,21 human factors24 and systems analysis,21 and the care bundle approach.20 Data sources for the process and patient outcome indicators included national databases such as KH07 Central Returns,26 national administrative hospital data (Hospital Episode Statistics),19 22 23 25 national clinical registries the National Emergency Laparotomy Audit,21 22 the National Hip Fracture Database28 and the National Vascular Database,19 as well as locally collected data.20 24–28

    An expert team leading the collaborative was a defining key feature of a QIC in this review. Eight studies described at least one attribute of at least one member of the expert team, including expertise,24 28 30 job role19 21 23 and employing organisation.26 27 Multidisciplinary QI teams were described in seven studies,19–22 24 25 27 30 and in the remainder the composition of the team was not discussed. Patient involvement was described in four studies.19 23 28 30

    Networking opportunities generated by the QIC for the participating sites were described by all but one study.24 These opportunities were generally organised by the expert team, and did not occur organically. There was no description of the structure or content of any networking activities undertaken by any of the QICs, but learning from other teams’ experiences at meetings between QI teams was described by four studies.20 22 26 27

    Data sharing between collaborative sites was described by three studies.19 21 28 Feedback of data to individual sites was reported in six studies.19–21 25 27 28

    Quality assessment

    A quality assessment of the quantitative report of the 10 included QICs was carried out using the QI-MCS.15 The QI-MCS scores typically ranged from 11 to 13 (table 3). The reports identified from the grey literature19 23 28 were of poorer quality than those published in peer-reviewed journals.

    View this table:
    Table 3

    Quality assessment of included studies using QI-MCS (met=1, not met=0)

    Poorly reported domains included organisational characteristics (where reports had to list at least two organisational characteristics), penetration/reach (where reports had to describe the number of eligible units that actually participated) and the type of study design. The best reported domains were spread, data source, organisational readiness (where at least one barrier or facilitator is reported), organisational motivation and intervention description (describing one specific change in detail).

    Effectiveness of QICs in UK surgical settings

    Effectiveness of the QICs was assessed according to the primary indicator most relevant to the aim stated by the study (table 4). This was not always the prespecified outcome reported by the study. If there were a number of different indicators related to the aim of the collaborative reported, the one reported first by the study is described (eg, risk-adjusted mortality vs unadjusted mortality, where both are reported). Where there was no p value reported in the study for that outcome, we used the conclusion of the study authors to reflect what exists in the literature.

    View this table:
    Table 4

    Effectiveness of quality improvement collaboratives (ordered by study quality)

    Facilitators and barriers to QIC effectiveness are summarised in table 5. All reports related to each QIC, whether quantitative, qualitative or mixed methods, were coded if they contained information on factors affecting effectiveness of its implementation. Two of the five reports relating to the Enhanced Peri-Operative Care for High-risk patients trial contained no description of facilitators or barriers.31 32

    View this table:
    Table 5

    Barriers and facilitators to QIC effectiveness

    Numbers of facilitators and barriers reported per QIC ranged from 1 to 28. Just one report presented key influences on success according to a framework (normalisation process theory).33 More facilitators than barriers were reported throughout (106 of 158, 67.1%). There was no obvious correlation between numbers of facilitators and barriers reported and the measured effectiveness of the QICs. The key facilitators and barriers in each domain for QICs in UK surgery are described below.

    Intervention characteristics

    Facilitators and barriers related to the intervention were described by eight studies. Interventions which contained a degree of flexibility were described as having a greater chance of being successful,25 26 28 and this adaptability of the intervention was useful in meeting the needs of diverse sites.19 22 An evidence-based intervention was welcomed by QI teams,21 22 30 34 but where the evidence was not perceived to be strong by clinicians,25 27 implementation was a challenge. Quality of the design and presentation of the intervention to the implementing teams helped to motivate clinicians and influenced engagement with the programme,19 22 33 but where the assembly of the intervention was in conflict with clinician expectation, this acted as a barrier to change.21 28

    Outer setting

    Outer setting constructs were powerful facilitators for QI across six studies that described them as such. The backing of Royal Colleges, specialty associations and national initiatives19 23 33 were effective drivers for change. However, broader external pressures led to service reorganisation and fiscal instability in one study, which did not support change efforts.30 Other strong facilitators were: (1) the collaboration between clinicians which often led to the exchange of ideas, sharing of good practice and a sense of community beyond one’s own organisation19 28 33 and (2) the competitive pressure of having data shared within the collaborative.19 21 28

    Inner setting

    The inner setting was the domain factors affecting QIC implementation were mapped to most frequently. Resource availability appeared to be key, specifically with regard to time for staff to participate in QI activities,22 24–26 28 30 33 financial resource19 23 26–28 30 and human resource.24 26 28 30 Members of one collaborative used sharing of local data to encourage the provision of extra resource.19

    The presence or absence of organisational leadership engagement was also important for the success of individual sites within the collaborative; support from senior leadership helped to overcome financial issues27 and embed change,19 but lack of leadership engagement was seen as a barrier to performance by several other studies.24 26 30

    Studies found structural characteristics of involved organisations had a significant bearing on the success of the intervention, especially as a barrier with regard to units facing service reorganisation.19 26 Other challenges related to the relative priority of the intervention within the organisation, with other improvement targets focused on, to the detriment of the QIC.28 33 34 Studies did not typically highlight specific aspects of the surgical setting, such as theatres, surgical or anaesthetic departments as barriers or facilitators, but one did describe a locally challenged emergency department as a barrier to improvement in a related process indicator.28

    Individuals

    Characteristics of individuals were the only domain to have more barriers than facilitators mapped to it. Positive beliefs about the intervention were important as a facilitator,19 22 23 34 but negative beliefs acted as a barrier to QIC effectiveness, even if positive beliefs were coexistent.25–27 33 Reluctance to change usual ways of working, specifically in relation to clinicians, was described as a barrier for two QICs.26 27

    Process

    All 10 studies reported factors coded to the process domain. It was clear that engagement was a strong facilitator when present, and a barrier when absent.24 25 Engagement was achieved in different ways for different QICs, and for some clinical opinion leaders in anaesthetist, specialist nurse and surgeon roles were important.19 23 26 Some referenced clinical champions as a specific role to increase engagement,23 24 27 and others used patients and their stories as external change agents to motivate staff.19 23 28 33 Data collection and feedback was recognised as an important enabler of QI evaluation,19–21 23 28 and lack of data was a common barrier to improvement, usually because of inadequate resources to support data collection.28 30 33

    Teams

    Eight studies reported factors relevant to QI teams at participating QIC sites. Instability of the team was a frequently reported barrier to effectiveness, with team members leaving,24 absence of surgical specialty leads30 or lack of management continuity26 all implicated. Teamwork and positive culture, however, were universal facilitators when present, while their absence was not reported as a barrier. In some cases, QICs led to improved multidisciplinary team working across all members of the surgical team, by providing a common language for discussions between surgeons and anaesthetists,34 increasing perceived value of team members23 and changing ways of working between ward, surgical, anaesthetic, theatre and therapy teams.28 Strong teams led to effective QI in several QICs.19 23 26 28 33 No factors relating to the expert team were discussed.

    Not all of the facilitators and barriers coded in the reports were associated with the collaborative method for QI. Constructs in the outer setting were most likely to be associated with QIC participation, and facilitators and barriers in the inner setting and process domains would have been relevant to many other approaches to QI.

    Discussion

    There are an increasing number of published studies on QICs7 and there have been relevant protocols for QICs introduced into surgical services recently published.35–37 However, we currently have a limited understanding of whether (and how) QICs are effective in improving care for surgical patients, with their less simple care processes. Most studies in this review reported that QICs were effective in improving both process indicators and patient outcome indicators that reflected primary aims of the collaboratives. However, consistent with previous less context-specific reviews,7 38 there are various reasons why the study results must be treated with caution. The study designs were not uniformly robust, with only 5 of the 10 studies incorporating a contemporaneous control group in order to adjust for secular trends in process or outcome indicators, and only 1 of these was an RCT. Only 4 of the 10 studies met criteria for inclusion in an EPOC review. The other reports used historical data as a baseline, and are therefore more susceptible to bias and confounding. We suspect that it is not a coincidence that the two studies that reported QICs as ineffective in achieving their primary aim used study designs with contemporaneous controls, and met EPOC criteria.

    The collaboratives themselves were a heterogeneous set of interventions, and differed in relation to: the numbers of sites included, the duration of the intervention, the measurement of effect, types of indicators reported and their comparators. The statistical significance was not reported for the change in the primary process or outcome indicator in 5 of 10 QICs and 1 report23 relied on graphs alone to demonstrate change over time rather than giving a numerical result. Another report21 described a significant effect in the second year after the implementation of the intervention only.

    The description of interventions in the reports was limited and reproduction elsewhere would prove difficult. Similar issues with reporting of the content of the intervention have been seen in reviews of QICs,38 as well as in other non-drug interventions.39 Resources that offer a framework for intervention descriptions such as the Template for Intervention Description and Replication (TIDieR) checklist could be used to improve intervention reporting.40 The quality of reporting according to scoring on the QI-MCS was variable, and notably poorer in the non-peer-reviewed grey literature.

    Half of the QICs included in this review were published prior to the publication of the Standards for QUality Improvement Reporting Excellence (SQUIRE 2.0) guidelines41 in 2016, which describe 19 items that should be used when reporting formal studies of QI. This may have contributed to the poor quality of reports. In addition, the Medical Research Council recommends process evaluation as an essential part of the design and testing of complex interventions, and provides guidance on how to carry them out.10 Just two QICs in our review had process evaluations published.30 33 Only one published a protocol42 and few reports indicated any differences between the planned and delivered intervention. Limitations in reporting of the QICs meant that specific intervention components could not be linked to structure, process or outcome indicators, and evaluation according to Donabedian’s model43 of care quality could not take place. Paucity of descriptions of current care processes leaves the authors unable to draw conclusions on how effective QICs are at different levels of complexity within surgical care, but we know surgical care is more complex than other specialties8 and our findings could, therefore, be generalisable to other less simple care processes. Reporting future QICs according to SQUIRE 2.0 guidelines and the TIDieR checklist, incorporating process evaluation into the design of QICs, and publishing intervention protocols would allow more in-depth evaluation of what contributes to the effectiveness of QICs.

    Using the CFIR to map facilitators and barriers to effectiveness of implementation of QICs in UK surgery highlights the importance of constructs within the inner stetting as both facilitators and barriers to implementation in moderately complex care processes. A systematic review using the same framework in QIC in stroke care found similar results.44 Available resources and engagement of leadership were key to success in implementation, which emphasises the importance of context in QI, and has been previously found by Zamboni et al11 and Schouten et al9 with regard to QIC.

    Many of the facilitators and barriers reported by the included QICs were not an intrinsic part of a QIC, as opposed to any other approach to QI. Facilitators that were directly related to QIC participation were mainly coded within the outer setting domain, with support from national bodies and specialty organisations being helpful, as well as networking with other QIC participants and the peer pressure associated with data benchmarking. Only two barriers related to QIC participation were reported, due to inadequate leadership by expert team-nominated regional leads and included site characteristics. The paucity of barriers associated specifically with QIC participation may indicate that participation has a universally positive impact on the QI objective, and any barriers to improvement are specific to the participating sites.

    Our recommendations for future QI interventions fall into two halves (figure 2), for researchers planning and evaluating QICs, and those in the participating centres implementing the intervention.

    Figure 2
    Figure 2

    Recommendations for future QICs. EPOC, Effective Practice and Organisation of Care; TIDieR, Template for Intervention Description and Replication.

    Strengths and limitations

    This systematic review and evidence synthesis combined a thorough database review and a grey literature search, but it is possible that relevant studies may not have been included. This risk was minimised by reviewing reference lists of previous systematic reviews and included articles, as well as a comprehensive grey literature search focusing on QI funding reports. Studies were included that did not meet all the criteria in the literature for QIC, in order to maximise inclusion despite poor reporting of these criteria and gain learning where possible. Most included reports showed a positive effect from the QIC, and may, therefore, be subject to a publication bias. Studies dating back to 2003 were included, and the relevance of the earliest studies to the current NHS context could be debated. However, the methodology of QICs has not changed over this time, and frameworks used have not evolved.45 Facilitators and barriers to effectiveness in the current context are likely to have been captured in the more recent reports.

    Conclusion

    The evidence base regarding the effectiveness of QICs in UK surgery, a moderately complex care process, is limited. This review highlights that, while 8 of the 10 UK surgical QICs reported the QIC method was effective, the quality of the studies was poor and these positive results must be treated with caution. QICs do carry benefits for participants in terms of credibility associated with being part of a project endorsed by a national body or specialty organisation. Future QICs in complex care processes should ensure that the limitations are not repeated, with publication of protocols, robust study design including a contemporaneous control group and reporting and evaluation of both process and content of the intervention. In order to overcome barriers to effective implementation, inner setting constructs of the CFIR should be considered when selecting collaborators. Specifically, it is crucial to secure organisational leadership engagement and adequate dedicated resources.

    Data availability statement

    Data are available upon request.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    The authors would like to thank Sarah Frankham and Camila Garces-Bovett for their assistance in developing the search strategy for this systematic review.

    References

    1. ↵
      1. National Quality Board
      . A shared commitment to quality for those working in health and social care systems; 2021.
    2. ↵
      1. General Medical Council
      . Good medical practice. GMC, 2013.
    3. ↵
      1. The Royal College of Surgeons of England
      . Good surgical practice. London: The Royal College of Surgeons of England, 2014.
    4. ↵
      1. Jones B,
      2. Kwong E,
      3. Warburton W
      . Quality improvement made simple: what everyone should know about healthcare quality improvement. 2021.
    5. ↵
      1. Quinton HB,
      2. Kasper JF,
      3. O’Connor GT, et al
      . A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. The Northern New England cardiovascular disease study group. JAMA 1996;275:11. doi:10.1001/jama.275.11.841
      OpenUrl
    6. ↵
      Improvement collaboratives in health care: evidence scan [The Health Foundation]. 2014. Available: https://www.health.org.uk/publications/improvement-collaboratives-in-health-care
    7. ↵
      1. Wells S,
      2. Tamir O,
      3. Gray J, et al
      . Are quality improvement collaboratives effective? A systematic review. BMJ Qual Saf 2018;27:226–40. doi:10.1136/bmjqs-2017-006926
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Ergina PL,
      2. Cook JA,
      3. Blazeby JM, et al
      . Challenges in evaluating surgical innovation. Lancet 2009;374:1097–104. doi:10.1016/S0140-6736(09)61086-2
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Schouten LMT,
      2. Hulscher MEJL,
      3. van Everdingen JJE, et al
      . Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008;336:1491–4. doi:10.1136/bmj.39570.749884.BE
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Moore GF,
      2. Audrey S,
      3. Barker M, et al
      . Process evaluation of complex interventions: medical research council guidance. BMJ 2015;350:h1258. doi:10.1136/bmj.h1258
    11. ↵
      1. Zamboni K,
      2. Baker U,
      3. Tyagi M, et al
      . How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review. Implement Sci 2020;15:27. doi:10.1186/s13012-020-0978-z
    12. ↵
      1. Kaplan HC,
      2. Provost LP,
      3. Froehle CM, et al
      . The model for understanding success in quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Qual Saf 2012;21:13–20. doi:10.1136/bmjqs-2011-000010
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Glenton C,
      2. Bohren MA,
      3. Downe S, et al
      . EPOC qualitative evidence syntheses: protocol and review template v1.2; 2021.
    14. ↵
      1. Effective Practice Organisation of Care
      . EPOC protocol template: @zenodo_org. 2021. Available: https://zenodo.org/record/5106282/files/epoc_protocol_template.pdf
    15. ↵
      1. Hempel S,
      2. Shekelle PG,
      3. Liu JL, et al
      . Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications. BMJ Qual Saf 2015;24:796–804. doi:10.1136/bmjqs-2014-003151
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Damschroder LJ,
      2. Aron DC,
      3. Keith RE, et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. doi:10.1186/1748-5908-4-50
      OpenUrlCrossRefPubMed
    17. ↵
    18. ↵
      1. Rogers L,
      2. De Brún A,
      3. McAuliffe E
      . Development of an integrative coding framework for evaluating context within implementation science. BMC Med Res Methodol 2020;20:158. doi:10.1186/s12874-020-01044-5
    19. ↵
      1. Potgieter R,
      2. Hindley H,
      3. Mitchell D, et al
      . Delivering a national quality improvement programme for patients with abdominal aortic aneurysms. The Vascular Society of Great Britain and Ireland, 2012.
    20. ↵
      1. Huddart S,
      2. Peden CJ,
      3. Swart M, et al
      . Use of a pathway quality improvement care bundle to reduce mortality after emergency laparotomy. Br J Surg 2015;102:57–66. doi:10.1002/bjs.9658
      OpenUrl
    21. ↵
      1. Aggarwal G,
      2. Peden CJ,
      3. Mohammed MA, et al
      . Evaluation of the collaborative use of an evidence-based care bundle in emergency laparotomy. JAMA Surg 2019;154:e190145. doi:10.1001/jamasurg.2019.0145
    22. ↵
      1. Peden CJ,
      2. Stephens T,
      3. Martin G, et al
      . Effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery (EPOCH): a stepped-wedge cluster-randomised trial. Lancet 2019;393:2213–21. doi:10.1016/S0140-6736(18)32521-2
      OpenUrlCrossRefPubMed
    23. ↵
      1. McNaney N
      . Enhanced recovery partnership project report. Department of Health, 2011.
    24. ↵
      1. Feinberg J,
      2. Flynn L,
      3. Woodward M, et al
      . Improving emergency surgical care for patients with right iliac fossa pain at a regional scale: a quality improvement study using the supported champions implementation strategy. Int J Surg 2018;57:105–10. doi:10.1016/j.ijsu.2018.08.003
      OpenUrl
    25. ↵
      1. Bamber JR,
      2. Stephens TJ,
      3. Cromwell DA, et al
      . Effectiveness of a quality improvement collaborative in reducing time to surgery for patients requiring emergency cholecystectomy. BJS Open 2019;3:802–11. doi:10.1002/bjs5.50221
      OpenUrl
    26. ↵
      1. McLeod H,
      2. Ham C,
      3. Kipping R
      . Booking patients for hospital admissions: evaluation of a pilot programme for day cases. BMJ 2003;327:1147. doi:10.1136/bmj.327.7424.1147
      OpenUrlPubMed
    27. ↵
      1. Kuper M,
      2. Gold SJ,
      3. Callow C, et al
      . Intraoperative fluid management guided by oesophageal Doppler monitoring. BMJ 2011;342:bmj.d3016. doi:10.1136/bmj.d3016
    28. ↵
      1. Tadd W
      . Scaling up for safety: standardising the lessons learnt from HipQIP. Royal College of Physicians, 2019.
    29. ↵
      1. Mowatt G,
      2. Grimshaw JM,
      3. Davis DA, et al
      . Getting evidence into practice: the work of the Cochrane effective practice and organization of care group (EPOC). J Contin Educ Health Prof 2001;21:55–60. doi:10.1002/chp.1340210109
      OpenUrlCrossRefPubMed
    30. ↵
      1. Stephens TJ,
      2. Peden CJ,
      3. Pearse RM, et al
      . Correction to: improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial). Implement Sci 2018;13:148. doi:10.1186/s13012-018-0840-8
    31. ↵
      1. Yang F,
      2. Walker S,
      3. Richardson G, et al
      . Cost-effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery: learning from 15,856 patients. Int J Surg 2019;72:25–31. doi:10.1016/j.ijsu.2019.10.001
      OpenUrl
    32. ↵
      1. Stephens TJ,
      2. Peden CJ,
      3. Haines R, et al
      . Hospital-level evaluation of the effect of a national quality improvement programme: time-series analysis of registry data. BMJ Qual Saf 2020;29:623–35. doi:10.1136/bmjqs-2019-009537
      OpenUrlAbstract/FREE Full Text
    33. ↵
      1. Stephens TJ,
      2. Bamber JR,
      3. Beckingham IJ, et al
      . Understanding the influences on successful quality improvement in emergency general surgery: learning from the RCS chole-quic project. Implement Sci 2019;14:84. doi:10.1186/s13012-019-0932-0
    34. ↵
      1. Martin GP,
      2. Kocman D,
      3. Stephens T, et al
      . Pathways to professionalism? Quality improvement, care pathways, and the interplay of standardisation and clinical autonomy. Sociol Health Illn 2017;39:1314–29. doi:10.1111/1467-9566.12585
      OpenUrlPubMed
    35. ↵
      1. Scrimshire AB,
      2. Booth A,
      3. Fairhurst C, et al
      . Scaling up quality improvement for surgical teams (QIST)-avoiding surgical site infection and anaemia at the time of surgery: protocol for a cluster randomised controlled trial. Trials 2020;21:234. doi:10.1186/s13063-020-4152-3
    36. ↵
      1. Wagstaff D,
      2. Moonesinghe SR,
      3. Fulop NJ, et al
      . Qualitative process evaluation of the perioperative quality improvement programme (PQIP): study protocol. BMJ Open 2019;9:e030214. doi:10.1136/bmjopen-2019-030214
    37. ↵
      1. Taylor J,
      2. Wright P,
      3. Rossington H, et al
      . Regional multidisciplinary team intervention programme to improve colorectal cancer outcomes: study protocol for the Yorkshire Cancer Research Bowel Cancer Improvement Programme (YCR BCIP). BMJ Open 2019;9:e030618. doi:10.1136/bmjopen-2019-030618
    38. ↵
      1. Nadeem E,
      2. Olin SS,
      3. Hill LC, et al
      . Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q 2013;91:354–94. doi:10.1111/milq.12016
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Hoffmann TC,
      2. Erueti C,
      3. Glasziou PP
      . Poor description of non-pharmacological interventions: analysis of consecutive sample of randomised trials. BMJ 2013;347:f3755. doi:10.1136/bmj.f3755
    40. ↵
      1. Hoffmann TC,
      2. Glasziou PP,
      3. Boutron I, et al
      . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:bmj.g1687. doi:10.1136/bmj.g1687
    41. ↵
      1. Ogrinc G,
      2. Davies L,
      3. Goodman D, et al
      . SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf 2016;25:986–92. doi:10.1136/bmjqs-2015-004411
      OpenUrlAbstract/FREE Full Text
    42. ↵
      1. Pearse R
      . Protocol 13PRT/7655: enhanced perioperative care for high-risk patients (EPOCH) trial (ISRCTN80682973) [The Lancet]. 2013. Available: https://www.thelancet.com/protocol-reviews/13PRT-7655
    43. ↵
      1. Donabedian A
      . Evaluating the quality of medical care. 1966. Milbank Q 2005;83:691–729. doi:10.1111/j.1468-0009.2005.00397.x
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      1. Lowther HJ,
      2. Harrison J,
      3. Hill JE, et al
      . The effectiveness of quality improvement collaboratives in improving stroke care and the facilitators and barriers to their implementation: a systematic review. Implement Sci 2021;16:95. doi:10.1186/s13012-021-01162-8
      OpenUrl
    45. ↵
      The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement [IHI Innovation Series White Paper]. 2003.

    Footnotes

    • Twitter @ellieroseatkins, @pennybir

    • Contributors All listed authors qualify for authorship based on making one or more substantial contributions to the intellectual content; design (EA, PB, LG, SW, ASJ, QL, JRB, ADP, IC and DAC), data acquisition (EA and PB), analysis and interpretation of data (EA, PB, LG and DAC). All authors participated in drafting of the manuscript (EA, PB, LG and DAC) and critical revisions of the manuscript (EA, PB, LG, SW, ASJ, QL, JRB, ADP, IC and DAC. All authors read and approved the final manuscript. EA takes responsibility as guarantor.

    • Funding EA and PB are in receipt of research fellowships jointly funded by the Royal College of Surgeons and the Circulation Foundation.

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.