Original research•

Creating an inexpensive hospital-wide surgical complication register for performance monitoring: a cohort study

Abstract

Objectives Basic tools that measure a hospital’s performance are required in order to benchmark or compare hospitals, but multispecialty institutional registries are rarely reported, and there is no consensus on their standard definitions and methodology. This study aimed to describe the setting up and first results of a hospital-wide surgical complication register that uses a minimal set of patient-related risk factors based on bedside data and produces outcomes data based on severity of complications.

Design Cohort study.

Setting Perioperative data related to all adult surgical procedures in a tertiary referral centre in Finland for 3 years (2016–2018) were included in the study. Complications were recorded according to a modified Clavien-Dindo classification, and the preoperative risk factors were compiled based on the literature and coded as numerical measures. The associations of preoperative risk factors with postoperative complications were analysed using the χ2 test or Fisher’s exact test.

Results In total, 19 158 operations were performed between 2016 and 2018. Data on complications (Clavien 0–9) were recorded for 4529 surgical patients (23.6%), and 779 complications were reported (Clavien 1–9), leading to an overall complication rate of 17.2%. Of these, 4.6% were graded as major (Clavien 4–7). Patient-related risk factors with the strongest association with complications were growing American Society of Anesthesiologists Physical Status Classification System score (p<0.001), growing Charlson Index (p<0.001), poor nutritional status (Nutritional Risk Screening 2002), p=0.041) and urgency of surgery (p<0.001).

Conclusions We describe an inexpensive hospital-wide surgical complication monitoring system that can produce valid numerical data for monitoring risk-adjusted surgical quality. The results showed that only a few patient-related risk factors were sufficient to account for the case mix.

What is already known on this topic

  • Complications can serve as a surgical quality measure, but there is no consensus on how they should be measured and reported. Hospital-wide surgical quality monitoring systems are rarely described. There is need for more cost-effective and simple surgical quality assessment.

What this study adds?

  • An inexpensive hospital-wide surgical complication monitoring system can produce valid numerical data for monitoring risk-adjusted surgical complications. Only few patient-related risk factors can be sufficient for preoperative risk adjustment.

How this study might affect research, practice or policy?

  • The described complication registering system is cost-efficient, easy to set up and does not need changes in the clinical process. The system with a minimal set of parametres can produce valid online numerical ready-to-use data, allowing continuous monitoring of the hospital’s surgical performance.

Introduction

Surgical quality measurement remains controversial and expensive. There is currently no consensus on how surgical quality should be measured and reported. This is mainly owing to multiple components of the healthcare system: the payer, the healthcare staff and the patient. Surgical quality is truly a heterogeneous concept, and Donabedian1 suggested that the concept of quality should be divided into three domains: outcome, structure and process. Outcomes can be measured in several ways, including functional gain or health benefit, patient satisfaction, economic gain, quality-of-life measurements, and complications or adverse event frequency. Surgical complications cause a significant economic and human burden and can be used as an outcome quality indicator.2

Commercial registers within surgical subspecialties have emerged in recent years for quality measurement (eg, https://bcbmedical.com). However, there is a lack of data on surgical cross-discipline quality monitoring systems.2 Basic tools that measure a hospital’s performance are required in order to benchmark or compare hospitals. In many cases, complications have been measured as gross parameters, such as mortality or morbidity. On the other hand, the expanding volume of data collection in surgery and medicine poses a substantial financial and administrative burden placed on clinicians. In the USA, the Centers for Medicare & Medicaid Services have invested more than one billion dollars in quality measure development in the past decade.3 Over 2000 quality measures were developed, of which one-third are in use, and even fewer are proofed valid.3 Many countries have a mixed healthcare economy, with the private sector accounting for a certain proportion. The outsourcing of routine elective surgeries to the private sector is increasing owing to the COVID-19 pandemic and public healthcare capacity problems. Robust risk-adjusted evaluation should therefore be performed regularly.4 5 Indeed, the decision and studies on ‘what and how to register’ are critical for quality assurance.

This study aimed to describe a hospital-wide surgical complication register using a minimal set of patient-related risk factors to produce in-hospital outcomes data based on severity of complications. The basic principle of the monitoring system was to collect patient-related risk factors, process-related data and treatment outcomes during clinical care in a simple, numerically coded fashion within the framework of pre-existing electronic patient records.

Methods

The leading principal in the chosen method was to build a simple and cost-effective surgical cross-disciplinary complication registering system that would catch in-hospital complications during clinical care in a tertiary referral centre, related to all surgical procedures with the exclusion of ambulatory, paediatric and gynaecological surgery. The aim was to determine the complication rate according to the severity of the complication (modified Clavien-Dindo classification6), adjusted with a minimal set of preoperative risk factors according to the literature—both as a numerical measure. The Clavien-Dindo classification was chosen since it grades complications according to severity from a patient-centred aspect; it is easy to use and interpret and has been demonstrated to be reliable across users.6 7 A wide literature search was done to determine the relevant patient-related risk factors, which would contain only bedside data, since according to the literature, exclusion of laboratory data set maintains predictive accuracy.8 We designed the system to require as little extra effort as possible, taking advantage of the existing clinical process (no extra staff needed) and the electronic patient record (no extra software needed). The staff registered the chosen set of preoperative parameters and, on discharge, the occurrence of eventual complications. Data were stored in a dedicated locus of the electronic patient record in a numerically coded format that was extracted for subsequent analysis, and monthly reports were created with no extra cost.

There was no patient involvement in this study. This study follows the cohort study Strengthening the Reporting of Observational Studies in Epidemiology guidelines.9

Selection of preoperative risk variables

According to the principle of risk-adjusted outcomes, we aimed to stratify each patient according to the anticipated risk of surgery. We performed a literature search in four medical bibliographic databases: Ovid MEDLINE In-Process and other non-indexed citations and Ovid MEDLINE from 1946 to 19 February 2015, Evidence-Based Medicine Reviews-Cochrane Database of Systematic Reviews between 2005 and January 2015 (OVID), PubMed (only ahead-of-print articles until February 2015) and Web of Science–Core Collection until February 2015 (Core Collection, Indexes=SCI-EXPANDED, SSCI). The search terms are listed in online supplemental appendix 1. Medical Subject Headings terms and text words related to surgical complications, risk adjustment and risk assessment, quality, safety and economic aspects served as the basis for selecting articles on risk factors for this register. Data extracted from the articles describing the preoperative risk factors are shown in table 1.

Table 1
|
Documented patient characteristics in the previous studies describing preoperative risk factors for surgical complications

Patient-related risk factors

As a general principle, we selected a minimal set of patient-related risk factors described in the literature and expressed them numerically. Previous research has demonstrated that a limited model based on a few perioperative risk variables is sufficient to perform risk-adjusted analysis for general surgery.10 11 Objective demographic variables included age12–15 and sex.14–16 Body mass index (BMI),10 14 16 17 alcohol intake18 and smoking14 16 17 19 were included since they reflect ‘lifestyle factors’ that could be monitored and influenced by patient information and advice.

To include comorbidities or symptoms that are described as major risk factors in the literature (congestive heart failure, ascites and chronic pulmonary disease; table 1), we chose the Charlson Comorbidity Index20 21 as a measure of general health status and American Society of Anesthesiologists Physical Status Classification System (ASA) class10–17 19 22 to describe anaesthesia-related risks. We chose Nutritional Risk Screening (NRS) 200223 to describe nutritional status and the metabolic equivalent of task (MET) index24 to describe functional status. Functional and nutritional status are additional general health measures that correlate with surgical risk.

In this registry, we focused on bedside data, and patient-related risk factors were collected and measured based on patients’ general status and comorbidities. Diagnostic medical measures, such as albumin, blood urea nitrogen and alkaline phosphatase, as seen in table 1, have not shown any incremental value for the risk prediction.8 12 A recently developed preoperative risk prediction tool, the SORT (www.sortsurgery.com), gives an estimation of the risk of death within 30 days of inpatient surgery with using only age, ASA and cancer status (yes/no) as patient-related risk factors.25

Surgery-related risk factors

Procedure-related risk factors, such as the urgency of surgery (elective/emergency) and subspecialty, were categorised.

Complication grade index

We chose to measure complications by grading the severity numerically using a modified Clavien-Dindo classification.6 26 Technical and process failures were included in the system, as shown in table 2.

Table 2
|
Modified Clavien-Dindo classification for postoperative complications

Data analysis

All surgical operations with data on complication severity (modified Clavien-Dindo index) from a tertiary referral centre between 1 January 2016 and 31 December 2018 were analysed. Complications were classified into two classes: minor (Clavien 1–3) and major (Clavien 4–7). No complications were marked as zero (0) and other complications as 8–9.

ASA was graded as 1–5 (the lower number, the healthier the patient) and functional status as MET index of 1–5 (1=totally dependent, 2=partly dependent, 3=moves independently indoors, 4=does physical activities and 5=does vigorous physical activities). The urgency of surgery (emergency or elective) and Charlson Index, graded in three categories (0, 1–3 and >4), were recorded. Nutritional status was graded as NRS 2002 index (0–3 with 0 representing normal BMI with no weight loss and 3 representing extremely poor nutrition with weight loss and BMI under 18), and BMI was graded in three categories (>18.49=low, 18.5–31.99=normal and >32=overweight). Smoking was recorded as 0=no smoking, 2=ex-smoker and 3=current smoker. Alcohol intake was recorded as 0=never, 1=less than once a month, 2=2–4 times a month, 3=2–3 times a week, 4=>4 times a week.

Statistical analysis was performed using the cross-tabulation χ2 test or Fisher’s exact test. The data are expressed as numbers and percentages (table 3 and online supplemental table 1). Age was given as the mean in each group.

Table 3
|
Severity of risk factors in relation to surgical complications based on Clavien-Dindo classification in a tertiary referral centre during years 2016–2018

Results

Data collection and complication rate

Complications were recorded at the beginning of 2016. From 1 January 2016 to 31 December 2018, 19 158 operations were performed. Data on complications (Clavien 0–9) were recorded for 4529 surgical patients (23.6%), and 779 complications were reported (Clavien 1–9). The complication rate of 17.2% is well in line with the literature, where the figures have been ranging from 5.8% to even 43.5%.7

Complications and type of surgery

Most patients (82.6%) were classified as 0, that is, no complications. There were 565 (12.5%) minor complications (Clavien 1–3), 207 (4.6%) major complications (4–7) and 7 (0.2%) other complications (Clavien 8–9). These results agree with the literature.7

The data collection frequency varied between hospital wards and subspecialties (online supplemental table 1). As expected, most major complications occurred in gastrointestinal surgeries and surgeries classified as ‘other’ (ie, emergency cases with multiple subspecialties).

Preoperative risk factors and complications

A wide literature search was done, and the parameters are explained in table 1. In this article, we wanted to study which of the parameters would be relevant in our system. The association of preoperative risk factors with postoperative complications was analysed using the cross-tabulation χ2 test or Fisher’s exact test. The frequency of complications is presented in table 3. Due to the large group of patients, there was statistical significance between complications and all risk factors except gender (p=0.961) and BMI (p=0.63). The patient-related risk factors showing the strongest association with complication trends were ASA (p<0.001), Charlson Index (p<0.001) and nutritional status (NRS 2002, p=0.041). In addition, the urgency of surgery was associated with subsequent occurrence of complications (p<0.001). Age, smoking status, alcohol consumption and functional status (MET index) did not show association with complication occurrence, although they were statistically significant. The result is somewhat different from what we have found in the literature (table 1). Also, the number of ASA 4 patients was quite small since the majority (82%) of the operations were elective.

Development and costs of the system

Nurses in the surgical outpatient clinic were trained in using the perioperative system and encoding the parameters of the patients signed up for elective surgery. According to the monthly reports, during the vacation period (June–August), the data reporting frequency dropped, and outside the vacation period, it improved. Permanent staff, frequent reminding of data collection and close follow-up resulted in improving the recording frequency.

The total calculated cost of the system was approximately €1000 for the initial computer programming followed by ca. €19 000 per year, which constituted labour costs of data recording. As a comparison, there is an annual fee of between $10 000 and $29 000 for sites participating in American College of Surgeons National Surgical Quality Improvement Programme (ACS-NSQUIP). This fee covers programme management and administration, on-site audits and ongoing technical support, but does not cover the labour wealth of data collected in ACS-NSQUIP. Also, the commercial registries for single disciplines (BCB Medical) cost €10 000–€11 000 per year per discipline for only the software. The labour costs accompanying the above two registries are anticipated to be much higher due to the multitude of parameters.

Discussion

The aim of this project was to study how robust surgical quality could be measured in a surgical hospital unit by creating a simple risk-adjusted cross-disciplinary surgical complication register. An institutional registry of all surgical specialties could be an implementation tool in quality benchmarking between hospitals and aid in determining their cost-effectiveness. At present, such institutional registries are rarely reported, and there is no consensus on their standard definitions and methodology.2

This study shows a possibility for a broad and clinically relevant quality measurement at a reasonable cost with a combination of a complication index (Clavien-Dindo) and a limited set of risk factor variables. The ACS-NSQUIP and numerous commercial registries provide a wider scope of complication categories and relevant risk factors, but the systems are costly, and demand dedicated staff.10 11 This project showed feasibility in being a simple real-time complication monitoring system that produces relevant data using the existing patient record system and staff commitment. The data extracted are standardised, numerical and quantifiable—either dichotomous or continuous—and can be directly analysed by statistical means. The system leans on the existing patient record system and routine clinical process, which make training of the staff and setting up of the system easier. The monthly reports were formed automatically, required no staff and therefore generated no extra cost per se.

Many preoperative risk factors have been used to perform risk-adjusted analysis for general surgery (table 1), but previous research has demonstrated that a limited model based on a few preoperative risk variables is sufficient.12–15 In this registry, patient-related risk factors were collected and measured based on patients’ general status and comorbidities since diagnostic medical measures have not yielded any incremental value for risk prediction.19 The Clavien-Dindo index was used to classify and describe complications: it is a well-accepted and widely used numerical index that measures complication severity based on the clinical outcome.6 7 26

The incidence of surgery-related major complications in industrialised countries has been reported to vary between 3% and even 42%.7 27 During the study period, the overall complication rate in our study population was 17.2%, of which 4.6% were graded as major (Clavien 4–7). This suggests that a low-cost cross-disciplinary complication registration system, such as the one reported here, can detect and grade complications in a reliable manner.

The challenge in creating a complication register is to decide which parameters are relevant—and enough—to produce clinically significant data. A wide literature search was done2 to reach a minimal set of preoperative risk factors: ASA, Charlson Index, emergency status, nutritional status, gender and age. Parameters associated with ‘lifestyle’ (potentially modifiable patient factors such as BMI and alcohol and tobacco use) were also chosen according to the literature (table 1). In our study, the statistical analysis revealed that, in this type of large material, all risk factors other than BMI and age showed statistical significance (table 3). Further analyses revealed clinical significance only with ASA, Charlson Index, nutritional status and emergency status. ASA and Charlson Index are both multidimensional constructs that reflect many risk domains and the overall patient status: in this respect, only one of them could be chosen to be representative. ASA is the most used and most referred to in the literature and—also in our study—the most clinically and statistically significant factor. Nutritional status provides important information on how to treat the patient perioperatively. The patient’s emergency status itself has been shown to be predictive of postoperative complications in various risk models.9 10

The modifiable risk factors (alcohol and cigarette smoking) have been shown to have an association with complications, and cessation of smoking has been shown to reduce postoperative morbidity.28 The programme for smoking cessation and reduction of alcohol intake has already been implemented in our hospital and may therefore influence the results. The lifestyle risk factors (BMI, smoking, alcohol consumption and nutrition) are relevant in decision making, planning and individual preparation for subsequent surgery.

When assessing quality in healthcare, at least robust risk adjustment is needed, since socioeconomic factors have a major effect on patient health (eg, obesity, diabetes, cardiovascular disease and cancer). People of lower socioeconomic status experience worse health outcomes and lower life expectancy, as the COVID-19 pandemic now demonstrates.29 30 However, previous studies have demonstrated that only a few preoperative risk variables may be needed for risk adjustment at the hospital level.10 11 25 30 For surgical performance monitoring, our study makes a further suggestion: in addition to emergency status, recording the ASA grade may be sufficient for robust risk adjustment. The ASA Physical Status Classification System is based on multiple factors that reflect the patient’s overall health status, has been widely used for over 60 years and has become a routine assessment for a patient’s preanaesthesia comorbidities.21 Reducing the register parameters will cut down the staff workload and system costs, which is essential in this era.

Strengths and limitations

The strength of this register lies in its fundamental principles: it is integrated with the daily routine, requires little financial investment, encompasses several surgical specialties and is based on an existing patient records system. It produces numerical data for statistical purposes. Furthermore, the system’s output on complications and risk factors was consistent with the literature. This article includes the description of implementation of the registry with the first results. It shows that the results correlate with the existing literature and that it can work. However, the validation of the process is still lacking.

Coverage rates have been the challenge with all complication registers,2 31 which is also a limitation of this register. However, the overall complication rate of 17.6% in this report is in line with figures reported earlier,27 which suggests that the potential selection bias has at least partly been compensated by the large size of the study population. Although the Clavien-Dindo classification is a standardised system, it can be a little subjective—the accuracy ranging from 87% to 93% according to the literature.7 During the complication registry project in our hospital, some controversial and confusing topics arose among the staff, which were discussed as the process continued. The full potential of this type of register is in the possibility of obtaining real-time data for a learning healthcare system: complications will comprise a part of such a quality register.

Future perspectives

Healthcare quality can be measured from many perspectives: patient-reported outcome measures (PROMs), patient-reported experience measures, cost-effectiveness and safety measures (vaccination coverage of the staff, hand disinfection consumption, complication/rehospitalisation rate, etc). Several types of data sources can be used: patient and staff questionnaires, claims data, administrative data and subspecialty registries. While it may be tempting for hospitals to use single operation or disease registers, this approach could lead to an unbalanced allocation of resources between subspecialties or disease groups. A hospital-wide system that combines surgical subspecialties with ASA-based risk adjustment may be more useful in ensuring equity and transparency. It can provide a broader view of how the system is performing and allow enough risk adjustment for hospitals in different regions and public versus private hospitals.

Thus far, the system has been implemented and the first results have been achieved. The monthly reports are created with no extra cost and are discussed in the half-year-term meetings. This study showed already that even less patient-related factors can be used for risk adjustment. We are presently studying if the register can make a difference in complications, costs and quality performance within the clinic, and there are plans to study it with other clinics for benchmarking.

Based on our experience, it appears that real-time online complication recording benefits the most from a programmed format, where all the complication data fields must be filled or appear automatically. We suggest further research on how this type of register would work with PROMs. This could form an ideal method for the assessment of surgical performance in hospitals.

Conclusions

Based on the results, we can highlight the following aspects to support the complication registry described in this article.

The complication frequency is in line with the literature indicating that the system works. The system is easy to set up and does not need extra software, staff or change of process, and the cost of putting up and maintaining the register is small. The system produces valid online numerical ready-to use data, which is easy to analyse and allows continuous monitoring of the surgical performance of the hospital. The results show that only a few patient-related risk factors are sufficient for monitoring surgical outcomes accounting for the case mix. The parameters can thus be reduced according to this study.

Article metrics
Altmetricopen-url
Dimensionsopen-url