Original research•

Development of delivery indicators and delivery enablers for cardiovascular disease in the UK: a modified Delphi study

Abstract

Introduction Standards to define and measure quality in healthcare for cardiovascular disease risk reduction and secondary prevention are available, but there is a paucity of indicators that could serve as facilitators of structural change at a system level. This research study aimed to develop a range of delivery indicators to help cardiac clinical networks assess delivery of and progress towards cardiovascular disease objectives.

Methods This study used an adapted version of the European Society of Cardiology’s four-step process for the development of quality indicators. The four steps in this study were as follows: identify critical factors of enablement, construct a list of candidate indicators, select a final set of indicators and assess availability of national data for each indicator. In this iterative process, a core project group of six members was supported by a wider review group of 21 people from the National Health Service (NHS) clinical and management personnel database.

Results The core project group identified six relevant cardiovascular disease priorities in the NHS Long Term Plan and used an iterative process to identify 21 critical factors that impact on their implementation. A total of 57 potential indicators that could be measures of implementation were developed. The core project group agreed on a set of 38 candidate indicators that were circulated to the review group for rating. Based on these scores, the core project group excluded 5 indicators to arrive at a final set of 33 delivery indicators. National datasets were available for 22 of the final indicators, which were designated as delivery indicators. The remaining 11, for which national datasets were not available but locally available datasets could be used, were designated as delivery enablers.

Conclusion The suite of delivery indicators and delivery enablers for cardiovascular disease could allow a more focused evaluation of factors that impact on delivery of healthcare for cardiovascular disease.

What is already known on this topic

  • Standards to measure quality in healthcare for cardiovascular disease risk are available.

  • There are a few indicators that could serve as facilitators of structural change at a system level.

What this study adds

  • A modified Delphi process was used to developed delivery indicators and delivery enhancers to help cardiac clinical networks assess the delivery of and progress towards national cardiovascular disease objectives.

How this study might affect research, practice or policy

  • Our research project has developed delivery indicators and delivery enablers built on NHS structural indicators that could allow prospective evaluation of factors that impact on the delivery of healthcare for cardiovascular disease to identify barriers to achieving priorities, including in areas with social deprivation or high ethnic diversity.

  • The next step is to implement our delivery indicators and enablers in practice to further refine them.

Introduction

Cardiovascular disease causes a quarter of all deaths in the UK and is the largest cause of premature mortality in deprived areas.1 2 Since 2011, the rate of increase in life expectancy in England had slowed as improvements in mortality from heart disease have plateaued,1–3 however, mortality rates are now rising—particularly since the COVID-19 pandemic.4 5

Cardiovascular disease is the condition from which the NHS in the UK can save most lives, and the NHS Long Term Plan sets out priorities for improving healthcare services, with an ambition to prevent 150 000 strokes, heart attacks and dementia cases over the next 10 years.2 The National Institute for Health and Care Excellence (NICE) provides guidance about optimal cardiovascular disease risk reduction and secondary prevention,6 and NHS England has a clear and longstanding policy regarding promoting NHS Health Checks.7

The NICE has developed processes for creating quality standards and indicators which measure outcomes that reflect quality of care.8 It also considers processes9 linked by evidence to improved outcomes, with indicators used in multiple settings to support high-quality care. These include identifying where improvements are needed; setting priorities for quality improvement and support; creating local performance dashboards; benchmarking performance against national data; supporting local quality improvement schemes; and showing progress local health systems are making on outcomes.

Reporting standards for performance and quality improvement in delivering cardiovascular disease objectives are benchmarked through the collection of primary care audit data as part of the Cardiovascular Disease Prevention Audit (CVDPREVENT) and Getting it Right First Time (GIRFT) national review audit process.10 11 Descriptors of healthcare improvement—such as those represented in CVDPREVENT and the GIRFT audit10 11 —provide mechanisms for evaluating the effectiveness of implemented interventions.12 Quantifying measures of healthcare performance and implementing measures to improve them are associated with improved prognosis.13 14 This serves as a mechanism for stimulating the delivery of evidence-based medicine through quality improvement, benchmarking of care providers, accountability and pay-for-performance programmes.15 For example, the introduction of a pay-for-performance programme in north-west England significantly reduced mortality,16 while its removal led to an immediate decline in performance on quality measures.17

The NHS Long Term Plan states some key cardiovascular disease priorities, such as: ‘People with heart failure and heart valve disease will be better supported by multidisciplinary teams as part of primary care networks’.2 Yet, this objective may be achieved only if certain ‘enablers’ are in place. For example, interconnectivity of patient healthcare records is an enabler that is part of healthcare improvement processes but this is not identified specifically when it comes to setting clinical service objectives and often needs to be in place before clinical objectives can be met. Interconnectivity of patient healthcare records is an enabler that can impact on a broad range of cardiovascular disease processes, including multidisciplinary team support for heart failure, medicines reconciliation, ensuring individuals at higher risk of atrial fibrillation have pulse checks at every opportunity, sharing echocardiography records and recording positive lifestyle choices or data from wearable devices.

Although standards to define and measure quality in healthcare for cardiovascular disease risk reduction and secondary prevention are available, we are not aware of indicators that could serve as facilitators of structural change. The European Society of Cardiology quality indicators for cardiovascular disease15 and the American Heart Association/American College of Cardiology performance measures of cardiovascular disease encompass structural, process and outcomes measures but these are not specifically designed for enabling structural change.18 Moreover, little is currently known about how well local services are configured in order to deliver quality objectives or about which issues need to be mitigated or enablers operationalised to improve patient care, promote prevention and reduce premature mortality or loss of quality of life.

In this research project, we aimed to identify operational factors that impact on achievement of the cardiovascular disease objectives in the NHS Long Term Plan.2 This paper describes the process we undertook to develop quality indicators by identifying system and service configuration measures that may be related to cardiovascular disease care objectives and differentiating whether these indicators are measurable nationally (delivery indicators) or locally (delivery enablers).

Methods

We aimed to develop a qualified range of quality indicators that could help cardiac clinical networks assess their delivery of and progress towards cardiovascular disease objectives. The six authors of this paper formed a multidisciplinary ‘core project group’ responsible for defining the scope of the project and its methods. This core project group had a range of clinical, analytics, health service and quality improvement expertise covering primary and secondary care, nursing and pharmacy.

Development of quality indicators

We used an adapted version of the European Society of Cardiology’s methods for the development of quality indicators,15 which follows a four-step process: (1) identification of the key domains of healthcare, (2) construction of candidate indicators, (3) selection of the final quality indicator set (including obtaining expert opinion using a modified Delphi process) and (4) conducting a feasibility assessment. The four steps in our study to identify delivery indicators and delivery enablers were as follows: (1) identification of the critical factors of enablement, (2) construction of a list of candidate indicators, (3) selection of a final set of indicators and (4) assessment of the availability of national data for each indicator to designate them as delivery indicators (national data available) or delivery enablers (national data not available) (figure 1).

Figure 1
Request permissionrequest permission icon
Figure 1

Process for the development of delivery indicators and delivery enablers.

Step 1: identify the critical factors of enablement

The core project group was tasked with identifying mechanisms that support implementation of the cardiovascular disease objectives in the NHS Long Term Plan2 in a post-COVID-19 environment.

We first identified key cardiovascular disease priorities for delivery within the NHS Long Term Plan2 most relevant to this project. We then used an iterative process, drawing on published literature and our own experience, to review each of the identified cardiovascular disease priorities and highlight critical process or structural factors—defined as overarching operational objectives required to enable NHS Long Term Plan implementation—that impact on delivery of these NHS Long Term Plan cardiovascular disease priorities.2

We also identified critical factors believed to impede on the delivery of relevant NHS Long Term Plan cardiovascular disease objectives from a review of the literature.2 This ensured there was no duplication of reporting and helped us to focus on structural elements that could influence implementation rather than the quality or performance of care.

Step 2: construct a preliminary list of indicators

To construct a preliminary list of candidate indicators, the core project group felt that the critical factors identified in step 1 should be validated against national datasets and metrics in order to assess which indicators could be applied to NHS healthcare activity data19 before they could be presented for service or system evaluation.

We wanted to use an accepted method and opted to use a modified Delphi process based on the European Society of Cardiology’s development of quality indicators for the quantification of cardiovascular care and outcomes.15 We used nationally available datasets, including the Quality and Outcomes Framework 2020/2021,20 general practice prescribing data 2020/2021,21 CVDPREVENT,22 Fingertips public health data,23 NHS Digital,19 the Public Health England Cardiovascular Disease Focus Pack Tool24 and various UK government publications (a full list of datasets used is included in online supplemental material A). The measurement period was defined as the most recent year of available data; however, because data availability and national audit are variable, there may be inconsistency of time periods. Some indicators required compound analysis from different datasets.

For each critical factor (the objective to be achieved), we identified candidate indicators that could be measures of implementation and for each of these, we defined a numerator and a denominator. The factor, indicator, numerator and denominator were all subject to Delphi review by the core project group against a modified set of the criteria used for the development and evaluation of the European Society of Cardiology’s quality indicators for cardiovascular disease (online supplemental table A).15 The importance domain was scored on a Likert scale of 0–5 (0=do not agree; 5=fully agree) and considered on its own for each candidate indicator. The remaining domains were marked yes or no and combined in a ‘sum of others’ score for each candidate indicator. At this stage, the core group identified some indicators as less applicable or more suitable to be pooled to form a more relevant indicator or replaced by an alternative based on their own experience and consensus.

Step 3: select a final set of indicators

A survey of the candidate indicators identified in step 2 was conducted across a ‘review group’ to derive the final set of indicators.

We felt that the most relevant target population for our indicators was the new integrated care systems, which were developed to improve quality of care by standardising clinical practice and addressing geographical variations in care. We also wanted to include provider collaboratives. The project team identified a wider cohort of managers and clinicians involved in cardiovascular disease service implementation and, in accordance with General Data Protection Regulation using the Wilmington Healthcare’s NHS clinical and management personnel database, invited them to form a review group. This group included managers, leads and directors related to cardiology and cardiac rehabilitation and representatives of clinical commissioning groups/integrated care systems with an interest in cardiology, long-term conditions, planned care and transformation/commissioning. The group also included clinical leads and executives of professional and patient organisations. We followed the European Society of Cardiology’s methods for developing quality indicators for cardiovascular disease, so no consent was required for the review group to participate in the voting.

The members of the review group were contacted by email and asked to score each candidate indicator on a Likert scale (0=do not agree; 5=fully agree), rating performance by its potential for impact on delivery of cardiovascular care and relevance of the indicator to its critical factor. To evaluate the indicators, the survey results were considered by median and then mean to indicate the priority and spread of results. The core project group planned to reject indicators with a median score ≤3.0 (3.0 was identified by the group as a point of weakness) and to assess further the relevance of indicators with a median score of 3.0–4.0 and mean of 3.0–3.7.

Step 4: assess availability of national data for each indicator

Where there was an available national dataset, we felt that the indicator could be used as a national comparator and called these delivery indicators. Where national data were not available, but the indicator was felt both to be important and potentially measurable at a local level, we called these delivery enablers.

Patient and public involvement

The invitees to the review group included clinical leads and general practitioners. We did not include patients or the public in this study.

Results

Step 1: identifying the critical factors of enablement

The core project group identified six cardiovascular disease objectives within the NHS Long Term Plan2 that were most relevant to the project:

  • 3.67: Early detection and treatment of cardiovascular disease can help patients live longer, healthier lives.

  • 3.68: Working with local authorities and Public Health England, we will improve the effectiveness of approaches such as the NHS Health Check, rapidly treating those identified with high-risk conditions.

  • 3.69: Where individuals are identified with high-risk conditions, appropriate preventative treatments will be offered in a timely way.

  • 3.70: People with heart failure and heart valve disease will be better supported by multidisciplinary teams as part of primary care networks.

  • 3.71: Fast and effective action will help save lives of people suffering a cardiac arrest.

  • 3.72: Cardiac rehabilitation is an intervention recommended by NICE which can save lives, improve quality of life and reduce hospital readmissions.

Based on these 6 objectives, 21 critical factors that impact on the implementation of these objectives were identified (table 1).

Table 1
|
Critical factors and corresponding National Health Service (NHS) Long Term Plan objectives2

Step 2: construct a list of candidate indicators

Using the modified Delphi process, the core project group identified 57 candidate indicators and defined denominators and numerators for each (online supplemental table B). Figure 2 provides an example for critical factor 6. The core project group reviewed the 57 candidate indicators and, using their expertise, came to a consensus on excluding, replacing or combining candidate indicators. This process resulted in 38 candidate indicators to be put forward to step 3 (see online supplemental table B).

Figure 2
Request permissionrequest permission icon
Figure 2

Example of the formulation of delivery indicators for an National Health Service (NHS) Long Term Plan4 priority and measurable definitions of these indicators.

Step 3: selecting a final set of indicators

The core project group contacted 427 people on the NHS clinical and management personnel database on up to 4 occasions by email and telephone between 8 March 2022 and 7 April 2022 and asked them to score each of the 38 candidate indicators identified in step 2 (see online supplemental table B) on a Likert scale. Respondents scored the potential indicators on their perception of the validity and reliability of the indicators, following the same principles used successfully for the development of the European Society of Cardiology’s international cardiovascular quality indicators.15 The core project group received 21 (5%) complete submissions (4 secondary care cardiologists, 7 consultants, 2 clinical directors, 3 clinical leads/senior consultants, 2 stroke nurse consultants, 1 cardiac specialist nurse and 3 general practitioners). The exact wording for indicators circulated to the review group, which differed slightly from that in the core project group’s original candidate list in some cases, and the median, mean and minimum scores for each candidate indicator are shown in online supplemental table C.

After evaluating the review group’s scores, the core project group rejected one preliminary indicator with a median score of 3 and a mean score of 3.4, as all other candidate indicators had median scores greater >3 and mean scores >3.7. The core project group also rejected three further candidate indicators related to cardiac rehabilitation because they required data from the National Audit of Cardiac Rehabilitation, which was no longer being made publicly available locally or nationally and so could not be used in practice. This resulted in 33 final indicators (see table 1).

Step 4: assessing availability of national NHS healthcare activity data

Based on the availability of suitable national datasets, 22 of the 33 final indicators were designated as delivery indicators and 11 were designated as delivery enablers (see table 2). Five of the delivery indicators (12a, 12b, 13a, 13b and 14a) require data from the National Audit of Cardiac Rehabilitation to which we did not have access.

Table 2
|
Critical factors and 33 final delivery indicators (no shading) for which a national dataset can be applied and delivery enablers (shading) for which there is no national dataset but which may be ascertained/explored locally

Discussion

Improvement in cardiovascular disease care and outcomes, as envisaged by NICE, Public Health England and the NHS Long Term Plan,2 6 7 requires system change for collaborative clinical objectives to be achieved. While the COVID-19 pandemic has had a considerable negative impact on cardiovascular disease care,25 26 the implementation of many of the factors highlighted by the NHS Long Term Plan, such as data integration, virtual multidisciplinary team integrated working and greater community diagnostics, has helped to start to ‘build back better’.

In this research project, our core project group and review group identified a number of determinants of structural process and enablers that impact on quality improvement, performance and, importantly, delivery of the cardiovascular disease objectives in the NHS Long Term Plan.2 We identified 22 delivery indicators, which are nationally measured, and 11 delivery enhancers, which are not currently measured nationally but for which local data may be available.

We believe that the concept of NHS delivery indicators as wider measures of service configuration impacting on specific health objectives (in this case the cardiovascular disease objectives within the NHS Long Term Plan2) is unique. The European Society of Cardiology and American Heart Association working groups specify guidelines and quality improvements, such as medication optimisation and diagnostic pathways, but they do not provide the detailed means by which to operationalise best practice and the steps needed to achieve those goals. Structural and process measures have been used within the Service Delivery Indicator programme27 in Africa, using clinical vignettes of common conditions to assess knowledge among a randomly selected sample of health workers,28 but these have not been validated and were not purposefully designed to address cardiovascular disease in a single national healthcare system.

Using national datasets enables identification of which geographical areas have enablers in place to implement change and improvement. This offers a useful tool for systems and strategic networks in planning—both in terms of local prioritisation of resources and dissemination of best practice—and has important implications for informing enablement programmes that deliver service change and identifying barriers to change. It is notable that, in many cases, quality and performance indicators have either not been achieved or improvement has levelled off.29 This is likely to be due to structural barriers, which is why delivery indicators are so relevant.

Study limitations

Our research was limited by the number of available national datasets, which restricted the number of potential indicators that could be selected. There may be other relevant datasets that we could not access—for example, we did not have access to the National Audit of Cardiac Rehabilitation and so we could not use it as a data source, which may have excluded important information from our analysis.

The response rate of 5% complete submissions from the review group is a study limitation. The project was undertaken during the COVID-19 pandemic, which made engagement with stakeholders difficult. Therefore, our evaluation is potentially incomplete or biased to those likely to understand the importance of engagement. Nevertheless, the 21 respondents had a variety of different roles and skills and thus contributed a range of perspectives. We, therefore, believe that our research has value and can be developed further and remapped in the future following the next stage of implementation.

Another limitation of our study is that critical factors are not always sufficiently explained by delivery indicators, so there may be other important factors that we were not able to address, such as workforce issues. To identify all of the key critical factors that affect delivery of an objective, it is necessary to engage everyone associated with the implementation of that objective. The core project group has representatives that covered the majority of stakeholders; but we may have missed some key roles (eg, finance directors).

Next steps

Following the research project described in this paper, in which we developed delivery indicators and delivery enablers, the next stage will be an improvement project to implement the delivery indicators and enablers in practice to obtain feedback and improve on the concept. Accordingly, we are developing a data visualisation tool that could be made available nationally for local comparison of delivery indicators with indicators such as those from the CVDPREVENT audit. This would first require a rank order evaluation of delivery indicators with relevant quality and performance indicators.

A prospective evaluation of delivery indicator and delivery enabler implementation within a defined locality would provide an indication of feasibility and value. In particular, delivery indicators and delivery enablers should be evaluated in areas with social deprivation or high ethnic diversity to see if those areas have specific barriers. Best practice and Quality and Outcomes Framework measures initially serve engaged patients in areas with high education and low social deprivation.30 Reviewing areas with high deprivation and good performance via personal interviews would reveal what they are doing differently.

Other enablers could be considered for analysis—for example, the impact of research on local services and the development of local patient involvement and engagement groups within the cardiovascular disease services. The same process could also be used for relevant wider local cardiovascular disease objectives beyond those in the NHS Long Term Plan. Ultimately, the value of deliver indicators depends on how useful they are in service development.

Finally, the cost-effectiveness of implementing delivery indicators and delivery enablers would need to be estimated. Translation of evidence-based practice via an approach using delivery indicators and delivery enablers has yet to be evaluated economically. However, we believe that this approach will help areas struggling to meet even the highest key priorities, such as cardiovascular disease prevention.

Conclusion

The 22 delivery indicators and 11 delivery enablers for cardiovascular disease developed based on NHS structural indicators could allow a more focused evaluation of critical factors that impact on delivery of healthcare for cardiovascular disease . Further work is needed to evaluate their utility and the opportunities they offer in practice.

Article metrics
Altmetric data not available for this article.
Dimensionsopen-url