Article Text
PDF
Abstract
Objective An ageing population and rising healthcare costs place healthcare systems at risk of failure. Our goal was to develop a technology that would identify illness early, initiate action and therein improve patient care, outcomes and save healthcare resources.
Design This was a prospective interventional quality improvement study.
Setting A 40 bed medical floor in a 300 bed Canadian tertiary care regional referral hospital.
Participants General ward patients randomly assigned to control or treatment groups. There was no cross-over or loss to follow-up.
Intervention We designed an algorithm and software programme capable of detecting the sentinel change in a deteriorating patient’s clinical condition and once detected direct early investigation and care. Study duration was 1 year.
Main outcome measures Primary outcome was patient transfer from the general medical ward to the intensive care unit (ICU). The secondary outcome was the time needed to (1) order investigations (2) contact senior medical staff and (3) senior medical staff intervention.
Results We identified a decrease in the transfer of patients from the medical ward to the ICU. Over the course of the study including 273 patients (110 in the control group and 163 in the treatment group), transfers dropped from 14 to 3 with a relative risk reduction of 85.54% (95% CI 84.96 to 86.1), a number needed to treat of 9.19 (95% CI 9.01 to 9.36) and a absolute risk reduction of 10.89% (95% CI 10.7 to 11.1). We also found a statistically significant reduction in the time required to order investigations (p=0.049), contact senior medical staff (p=0.040) and senior medical staff intervention (p=0.045).
Conclusion A novel algorithm and software in the hands of nursing staff identified acute illness with adequate sensitivity and specificity to dramatically reduce ICU transfers and time to clinical intervention on a medical ward.
- artificial Intelligence
- clinical decision-making
- critical care
- decision making
- decision support, clinical
Data availability statement
Data are available upon reasonable request. Available upon request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- artificial Intelligence
- clinical decision-making
- critical care
- decision making
- decision support, clinical
WHAT IS ALREADY KNOWN ON THIS TOPIC
Existing scoring and early warning protocols are limited in their scope as they are static, non-evolving, non-intelligent and nested in if/else statements.
WHAT THIS STUDY ADDS
This quality improvement initiative proved that an intelligent, action-based algorithm with an easy to use software interface can increase the detection of acute illness, reduce critical care admissions and thus patient morbidity and mortality.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The results of this study support further investment in the development of intelligent action-based algorithms and software programmes to aid healthcare workers in detecting and treating acute illness on hospital wards.
Introduction
Patients and their families have historically believed that hospitals were the safest place to be during times of illness and that they would receive thorough and timely care should their illness progress and they become acutely unwell.1 These beliefs were also held by healthcare workers who provide optimal and compassionate care all hours of the day and night. This faith has unfortunately waned in recent years.1 However, as providers and administrators we have the ultimate responsibility and privilege of restoring faith in the healthcare system and ensuring that our patients and their families receive only the best possible care.
Healthcare systems are being challenged globally. These challenges come in many forms including escalating healthcare costs, an ageing population with increased needs and most recently, a global pandemic.2 Most health systems are supported by finite monetary and human resources. This imbalance between supply and demand place our patients at risk of not receiving the best possible care, and our human resources at risk of injury, fatigue and burn-out. This gap between supply and demand is further stretched as we increasingly recognise the importance of timely intervention in all acutely ill patients.3–7
Given this, we find ourselves standing at the edge of an abyss. We need to act now if we want to save our patients and healthcare systems. Fortunately, we believe that recent advances in technology may provide the lifeline we so desperately need and allow us to innovate our way to a bright and healthy future.
Methods
Participants
Our prospective interventional study was conducted on a 40 bed medical ward, in a 300 bed Canadian tertiary care hospital. The 40 bed medical ward was divided in two groups with half the ward comprising the control group (n=110) and the other half the treatment group (n=163). This study was conducted over a 1 year period.
Design
We developed a software programme to be used by nursing staff to aid in identifying acutely ill patients. The software was designed to be broadly applicable and able to identify acute illness irrespective of the disease process or clinical setting. The input data for our algorithm included heart rate, systolic blood pressure, temperature, respiratory rate and Glascow Coma Scale.
Early first steps in building and developing our software and intelligent algorithm included an in-depth review of the literature including physiologic responses to illness, existing early warning protocols, algorithms and intelligent systems. Our software was then tested and adapted through real-world data sets to confirm validity, sensitivity and specificity. Finally, the software and application was put into the hands of the user (nursing staff) to ensure ease of use and practicality. A novel feature of our software and algorithm is that once the programme has interrogated and interpreted the data entered by the user, when indicated, the software directs the user to initiate medical intervention.
Procedure
For this study, the user manually entered data into the programme. While of relatively low fidelity, this procedure allowed us to conduct this study without having to integrate with an electronic health record. Although this was an extra step and added work for the nursing staff, adoption was universal and enthusiastic for the entirety of the study as the nursing staff quickly understood that the programme allowed them to better advocate and care for their patients. Future work will include integration with electronic health records to reduce nursing workload and eliminate the risk of transcription error.
If a patient is stable, the software programme produces a green message directing the user to continue with current management. When the software detects a mild to moderate abnormality, it produces a yellow message, directing the user to order level 1 investigations (lactate, troponin, complete blood count, c-reactive protein, urinalysis and oxygen saturation) and to reassess the patient in 30 min. If at the 30 min reassessment point, the patient remains unstable, the software upgrades the message to red. When the software detects a moderate to severe abnormality, it produces a red message directing the user to order level 1 and level 2 investigations (blood culture, urine culture, chest X-ray and venous blood gas) and to contact senior medical staff urgently.
A patient was included in the study when they became acutely unwell. On the control side of the medical ward, a patient was entered into the study if by traditional clinical assessment they were found to be acutely unwell. On the treatment side of the medical unit every time a patient’s vital signs were measured, the vital signs were entered into the software programme. When a patient’s vital signs triggered either a yellow or red message, that patient was then entered into the study treatment group.
Data collection
This project was supported by our local quality review board (Physician Engagement Society). We presented to them the full scope of our project including software development and research plan. To protect patient confidentiality and ensure anonymity, personal identifiers were not entered into the software and data were not saved between entries. When a patient was entered into the study, all data points were recorded on a paper worksheet that was gathered and collated at the end of every day by a research coordinator. Worksheets were held securely in the research coordinators locked office and the data were entered into a password-protected digital spreadsheet for secure storage and analysis. Electronic copies were not stored on the cloud or emailed between co-investigators.
Patient and public involvement
This quality improvement initiative was launched early into the SARS-CoV-2 global pandemic. As such, at that time, there was no opportunity to engage or work directly with patients or the public.
Results
This was a novel, real-world quality improvement study. Our populations were matched (table 1). The top four disease processes were equal between groups, and patients were admitted to the medical ward randomly by administrative staff unfamiliar with the study. There was no cross-over between our control and treatment groups. There were no patients lost to follow-up and only palliative patients were excluded.
Balanced Populations
Our study included a total number of 273 with 110 in our control group and 163 in our treatment group (table 2).
Total Patients and ICU Transfers
Our primary outcome was to determine whether or not our software and algorithm impacted transfers from the medical ward to the intensive care unit (ICU). We found a dramatic drop in ICU transfers from 14 in the control group to 3 in the treatment group (table 2), with a relative risk reduction of 85.54%, a number needed to treat of 9.19 and a absolute risk reduction of 10.89% (table 3).
Primary Outcomes
Our secondary outcome was to determine whether our software impacted medical intervention time. Time zero for the control group was the clinical determination that a patient was unwell. At time zero, it was up to the ward staff (nurse, nurse practitioner) to determine the appropriate course of action. Time zero for the treatment group was the software programme identifying an unstable patient and producing either a yellow or red message for the user.
To determine if there was a significant difference between the mean of our two independent groups, we used an unpaired t-test for our statistical analysis. We measured the interval from detection of acute illness in the control and treatment groups to three intervention points which included (1) ordering investigations, (2) contacting senior medical staff and (3) the senior medical staff ordering an intervention. We found that our software significantly reduced the time interval from detection of acute illness to all of our intervention points. Table 4 displays the mean times as well as the p-value for each of these intervals.
Secondary Outcomes
Discussion
We are a small physician research group based in a tertiary care centre. Anecdotally, we recognised that patients who are referred to the ICU often have an early sentinel change in their clinical condition. For varying reasons, this sentinel change is sometimes unrecognised by ward staff. We hypothesised that if these clinical changes could be identified at the time of their occurrence, early intervention could be initiated and prevent the need for ICU transfer and overall clinical deterioration. We also recognise that with an ageing population and limited global healthcare resources, healthcare systems will likely come under additional pressure going forward. Fortunately, advances in technology have allowed the medical community to take a step beyond simple early warning systems that are static, non-evolving non-intelligent systems nested in if/else statements, and develop more sophisticated means of recognising and treating medical problems.8 9 Software programmes such as ours are now intelligent and able to do more than simply add and subtract, but to intelligently interpret data in the context of a given clinical scenario and even go as far as to direct intervention and treatment.
This study was conducted during the global SARS-CoV-2 pandemic. Related staffing shortages limited broader data collection for outcomes such as status at discharge, or hospital and critical care length of stay. However, our primary outcome is significant in terms of saving healthcare resources, but more importantly, in reducing patient morbidity and mortality. We found a large drop in the number of ICU transfers in our treatment group (14 vs 3). To further explore this result, we asked ourselves if this reduction in ICU transfers could be fully explained by our secondary outcomes (table 4). We know that when it comes to sepsis, acute myocardial infarction or stroke, time is tissue.10 11 Our software did significantly reduce the time needed to respond to acute illness. But was this time reduction enough to account for the dramatic drop in ICU transfers. As we thought about this question, we revisited our original problem, that being the first or sentinel sign of acute illness is sometimes missed clinically. What became clear to us was that while there was a significant drop in the time to respond to acute illness, the real difference between our study groups was that the software was in fact identifying and recognising that first early sentinel abnormality that was being missed in the control group. Our software was able to recognise the acute illness and initiate the investigation and intervention early, before the patient’s condition worsened, and before the clinical deterioration would have been recognised in the control group (figure 1).
Acute illness identified by software at t=0 and by nursing staff at t+x.
Based on this, the realised difference in our time intervals between control and treatment groups was not measured in mere minutes (table 4), but in fact much longer, as represented by ‘x’ in figure 1. This also explains why we had more events in the treatment group than in the control group. Our treatment group was larger not because of selection bias, lack of blinding or matching, but because the software programme was more sensitive at identifying acute illness than traditional methods. However and importantly, the observed increase in sensitivity was balanced with appropriate specificity evidenced by the reduction in ICU transfers.
Our secondary outcome is best considered in the most practical of terms. In the real-world setting when an abnormal vital sign is identified, we naturally expect nursing staff to act immediately to avert crisis. However, this is not necessarily accurate and the decision-making process is actually quite complicated. This decision-making tree was first described in 1967 by Peter Drucker.12 Today’s literature commonly references a seven-step decision process. The first step (1) is to identify the decision. At this point, we recognise that something is wrong, but we have to weigh this against the many other tasks awaiting our attention. In step (2), we gather all the relevant information, looking internally and externally and in step (3), try to identify the causes (such as medication error or equipment malfunction) that led to the abnormality. In step (4), we weigh the evidence and consider all the alternatives before choosing the best course of action in step (5). In step (6), we take action and call the senior medical staff after which in step (7), we review our decision, the consequences and its outcome. You may argue that we make decisions every minute of the day and decision making is not difficult, especially for a highly trained nursing staff. However, we must keep in mind that today’s nursing staff never have just one task at hand or on their mind and they must constantly be re-evaluating and reprioritising their tasks as they are presented with new information. Alternatively, and likely an important contributor to our findings, an emotionally absent software programme only has to evaluate the data it is given, does not have to consider internal and external alternatives and simply follows its programmed or learnt rules to produce a clear and decisive answer. We believe it is this tedious decision-making process and all the internal and external variables involved in working on a busy hospital or long-term stay ward that, in addition to missing that important sentinel vital sign change, led to a delay in illness recognition and action in the control group.
The results of this quality improvement initiative must be considered in light of a few potential limitations. This study was not blinded or systematically randomised. If we were to do this study again, and had greater resources, we would ideally intentionally randomise patients admitted to either the control or treatment cohort of the medical ward, and blind the user to whether or not the software was being applied. An additional limitation is our relatively small sample size and the fact that we were limited to one medical ward. Our results would be stronger if the patient cohort included surgical patients and was conducted at more than one centre. Thus, external validity will need to be tested as our software is applied to other populations.
Conclusion
Two years into a global pandemic healthcare systems are buckling under the strain of increased demand and dwindling resources.13 Demand stems from an ageing disease-burdened population and multitudes of patient’s sick with SARS-CoV-2. In terms of dwindling resources, we are referring to our most valuable resource, our hospital staff. Two years of working long hours under extremely stressful conditions with little to no reprieve have left our hospital staff exhausted. The National Health Service recently warned that ‘patient safety has been compromised this winter because of a crippling health and social care staff shortage that would require a million additional workers by the next decade’.14 We are seeing similar shortages in Canada.15 16
Our goal with this prospective interventional study was to improve patient care through technology and innovation. Our software was able to identify more acute events, significantly reduce the time needed to identify, investigate and treat acute illness while ultimately reducing ICU transfers. These findings were accomplished through our software identifying the earliest (sentinel) signs of acute illness, eliminating the ambiguities and subjectivity of the clinical assessment and decision-making process, and finally, intelligently directing patient care. There is an important distinction, beyond our novel algorithm, that sets our software apart from static non-intelligent early warning systems; our software is action-based.
We believe that the importance of our results cannot be understated as we now practice in an era of chronic staff shortages but with increased demands on healthcare systems. In the face of this new reality, healthcare systems must find a way to ensure patient and staff safety while upholding standards of clinical practice. With this in mind and with the positive results of our preliminary work, we are midway through a follow-up study whereby we are retrospectively applying the programme and algorithm to all in-hospital deaths over a 1-year period. In this second study, we have been able to match patients based on age, gender, admission diagnosis and sofa score. In addition, we continue to develop our software to include additional input data that will further refine our software’s sensitivity and specificity to formulate a preliminary differential diagnosis as to the disease process at play. Finally, our ultimate goal is to introduce machine learning and develop artificial intelligence with our proven algorithm and data sets.
By standardising the clinical assessment, investigation and early response, software programmes such as ours will improve patient safety while facilitating best clinical practices and reduce staff workloads.
Data availability statement
Data are available upon reasonable request. Available upon request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by ARHCC Physician Engagement Society (local quality review board). (1) This was a quality improvement initiative, approved by our local quality improvement board. (2) This project was approved at the level of the health care region, hospital and medical ward.The treatment was to simply ensure that timely standard of care practices were provided to medical ward patients. The patient experience was the same in the control and treatment groups. This project was approved by health care region (Fraser Health Authority) and hospital (Abbotsford Regional Hospital and Cancer Center).
Footnotes
Contributors MK contributed to data acquisition, analysis and manuscript review. DM and TC contributed to data analysis and manuscript review. DW contributed to data and statistical analysis as well as manuscript review. JB contributed to study concept, design, analysis, interpretation of data and manuscript review. AN contributed to study concept, design, analysis and interpretation of data as well drafting of initial manuscript. JB and AN conceptualised and designed the software used in this study. The guarantor of this project is AN.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.