Article Text
PDF
Abstract
Background Hundreds of thousands of tests are performed annually in hospitals worldwide. Safety Issues arise when abnormal results are not recognized promptly resulting in delayed treatment and increased morbidity and mortality. As a result Singapore’s largest healthcare group, Singhealth introduced an electronic result acknowledgement system. This system was adopted by the Singapore National Eye Centre (SNEC) in February 2016. Baseline measurements show that weekly numbers of unacknowledged results ranged from 193 to 617. The current standards of electronic results acknowledgement posts a significant patient safety hazard.
Methods Root cause analysis was performed to identify contributory factors. Pareto principle was then used by the authors to identify the main contributory factors. We employed the rapid cycle improvement Plan-do-study-act (PDSA) strategy to test and evaluate implemented changes. Changes are implemented for 2 weeks and data collected prospectively. The data is analyzed the week after and the following PDSA actions are decided and instituted the following week. 3 PDSA cycles were undertaken in total.
Results The first PDSA cycle focused on raising awareness of the problem at hand, the number of unacknowledged results drastically decreased during the 1stweek of implementation of our PDSA from 617 to 254.The second PDSA cycle targeted the lack of knowledge of doctors involved in the electronic result acknowledgement process. There was a trend downwards near the end of the cycle which continued through the week after.The third PDSA cycle targeted individual doctors and provided individual remedial training. Second line doctors were also equipped to better handle abnormal results. There was significant improvement with the number of unacknowledged abnormal results dropping to <5 a week.
Conclusions Multiple factors were identified to contribute to the low compliance to electronic acknowledgement of results. The role doctors play in the issue at hand was paramount and required careful handling in a professional manner with multiple reminders and emphasis on the importance of acknowledging and acting on the results.A significant improvement in the rates of acknowledgement of abnormal results was demonstrated with clear benefits to patient safety. Interventions can be replicated when implementing similar systems to other areas of healthcare.
- PDSA
- root cause analysis
- healthcare quality improvement
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Problem
In most hospitals, there are instances where abnormal test results are not recognised or acted on in a timely fashion, thus resulting in morbidity or mortality for patients. Proper analysis of data to understand where the fault is and developing targeted strategies will mitigate future risks both to the patient and to the healthcare professional.1
We performed a root-cause analysis and found that three main contributory factors to the delay in acknowledging and acting on results include software-related issues, medical records office related and doctor related.
Three months after implementation of an electronic acknowledgement system, the results’ acknowledgement rate still lingered at about 90%–93%, representing a mean of 343 unacknowledged results in a week. As such we undertook a quality improvement project with an aim to increase the abnormal result acknowledgement rates to 100% within a 3-month period.
Background
Singapore National Eye Centre (SNEC) is the designated national centre within the public sector healthcare network and spearheads the provision of ophthalmological care. It manages an annual workload of 250 000 outpatient visits, 14 000 major eye surgeries and 13 000 laser procedures. Alongside a national move towards electronic medical records (EMRs), SNEC adopted the electronic acknowledgement system which displays test results ordered by a clinician and requires them to acknowledge electronically. A stipulated 48 hours timeline was also mandated for abnormal or critical results. Such a move is important to improve the quality of care for our patients.2
The flow chart (figure 1) illustrates the normal workflow from the time of ordering of a medical investigation to the time of result acknowledgement.
A flow chart illustrating the normal workflow from the time of ordering of a medical investigation to the time of result acknowledgement. Second-line doctors include designated medical officers during the weekdays and the on-call medical officer during the weekend and public holidays.
Data on electronic acknowledgement of results are monitored by the medical records office at SNEC. Concern was raised following surveillance data post implementation of the electronic system. Weekly data on the number of unacknowledged results ranged from 193 to 617.
Patient safety is one of the key goals in any healthcare system and it can be compromised in multiple ways. It is the duty of all healthcare professionals involved in patient care to actively promote and encourage a safety culture.3 Not acting promptly on critical abnormal investigation results can surely impact patient safety negatively. Reddy et al report settlements ranging from $127 837–2 021 887 for uveitis-related lawsuits.4
We initiated this quality improvement project aiming to enhance patient safety by optimising the acknowledgement of abnormal and critical results.
Measurement
The impetus for this quality improvement project was the surveillance data provided by the medical records office. The data provided both the absolute number of unacknowledged results and the rates of unacknowledged results. These could be further broken down into weekly and daily numbers if wanted.
Baseline measurements show that weekly numbers of unacknowledged results ranged from 193 to 617. We used this same measurement to track the progress of our project and subsequently further refined it to look specifically at the numbers of unacknowledged abnormal results.
Root-cause analysis was performed to identify contributory factors. Pareto principle was then used by the authors to identify the main contributory factors. The main factors are illustrated by the Ishikawa diagram (figure 2).
Ishikawa diagram showing the main contributory factors to the number of unacknowledged results including medical record office-related, software-related and doctor-related factors.
We hypothesised that the ‘Doctors’ group of factors was the most important in regard to their influence on overall results acknowledgement rates. Also, this group of factors was the most easily modifiable.
Design
The project team consisted of three doctors, one nurse, one member from the medical records office and one member from the information technology (IT) department.
The doctors were involved in management and training of clinical doctors to ensure timely electronic acknowledgement of results. The nurse was involved in management of case notes and patient flow to ensure that patients with abnormal results are flagged for early acknowledgement and review by the appropriate physician. Medical records office helped to facilitate the retrieval of physical case notes for patients with abnormal results. IT was involved in providing continued surveillance data on the number and nature of unacknowledged electronic results.
Strategy
We employed the rapid cycle improvement plan–do–study–act (PDSA) strategy to test and evaluate implemented changes. Changes are implemented for 2 weeks and data collected prospectively. The data are analysed the week after and the following PDSA actions are decided and instituted the following week.
PDSA cycle 1 (8–21 June 2016)
Out initial intervention aimed at raising awareness of the problem at hand. We emphasised the compromise on patient safety with missing abnormal results. This was done at a formal meeting setting. Doctors were also reminded via email and SMS to electronically acknowledge results in a timely fashion. Feedback received indicated that this was not sustainable and was in fact a source of annoyance to the doctors. Data on total number of unacknowledged results and total results were collected.
PDSA cycle 2 (29 June–12 July 2016)
To build on the first intervention, we targeted the lack of knowledge of doctors involved in the electronic acknowledgement process. Retraining by the IT department was done at specially allocated time slots with an emphasis on the work flow. We also arranged for case notes of patients who had undergone investigations to be kept in the treatment room for 24 hours to reduce the inconvenience of getting the medical records office to trace them. Due to the large volume of unacknowledged results, we decided to focus just on unacknowledged abnormal results>48 hours which is one of the hospital’s internal targets. There was a marked improvement with the number of unacknowledged abnormal results down to 70–80 a week.
PDSA cycle 3 (20 July–2 August 2016)
Despite having the appropriate training and making it more convenient in terms of having the patient’s case notes at hand, there were still results not being acknowledged. We hypothesised that a collective warning and reminder was less effective than individualised warnings. We identified doctors who persistently failed to acknowledge their abnormal results, approached them individually and provided remedial training. We worked also with the second-line doctors to help acknowledge remnant results; we realised that there was concern over acknowledging results without physically looking at the patient’s case notes and understanding what the tests were ordered for. Methods were taught to come to a sound medical decision based on previous tests results, which consultant the patient was reviewed by, when the subsequent follow-up appointment was, whether the results were already reviewed but just not acknowledged and the significance of the abnormality itself.
There was significant improvement with the number of unacknowledged abnormal results dropping to <5 a week.
Results
Our main outcome measure was the number of unacknowledged results which was monitored by the IT department.
With the changes implemented from each PDSA cycle, there was evidence of improvement. These data were plotted in a run chart. Post PDSA cycle 3, the number of unacknowledged results was significantly different from that of baseline data.
Table 1 shows the background data pre study in the first five rows, during PDSA cycle 1 (highlighted in green) and 1 week after PDSA cycle1 (highlighted in blue). Pre intervention, the weekly number of unacknowledged results varied between 193 and 617. We were only able to obtain the total number of unacknowledged results, including the normal results, as the appropriate algorithms to extract the unacknowledged abnormal results were not yet established and in place at this point in time. There was a significant number of backlogged unacknowledged results as reflected by the fact that the number of unacknowledged results surpassed the number of new results each week.
Background data, data pre and post plan–do–study–act (PDSA) cycle 1
It was promising that the number of unacknowledged results drastically decreased during the first week of implementation of our PDSA from 617 to 254. While we were not able to distinguish whether these unacknowledged results were from that particular week or from a backlog from previous months, the number of unacknowledged results decreased to less than the number of new results, suggesting a diminishing number of backlogged results to be acknowledged and a response to the first intervention.
Our second PDSA cycle (figure 3) ran for 2 weeks from 29 June until 12 July, where we focused on unacknowledged abnormal results>48 hours, a target set out by our hospital. There remained variability in the number of unacknowledged abnormal results; we attributed this to the delay in doctors receiving their retraining in the results acknowledgement system. There was a trend downwards near the end of the cycle which continued through the week after.
This graph shows the number of unacknowledged abnormal results during the second plan–do–study–act (PDSA) cycle.
During the third PDSA cycle (figure 4), the number of unacknowledged results had already reached close to zero for majority of days due to the implementations from the second cycle. There was however a surge in unacknowledged result on 31 July which fell on a Sunday. This was likely due to the long weekend with lesser manpower around.
This graph shows the number of unacknowledged abnormal results during the third plan–do–study–act (PDSA) cycle.
As demonstrated in figure 5, the decreased number of unacknowledged abnormal results was sustained after the study was done.
This graph shows the number of unacknowledged abnormal results during period after all interventions were in place showing sustainability of the implemented changes.
Lessons and limitations
The aim of this quality improvement project was to increase the rate of electronic results acknowledgement with a long-term goal of implementing a sustainable system rather than short-term gains. For this to be achieved, we needed to explore each contributory factor to the problem and address them. Therein lies the importance of the PDSA cycles which helped identify weak areas that could be worked on to optimise the final result.
Communication of not only the existence of but also the severity of the problem was a key factor which had to be dealt with. This was addressed not only corporately but also individually. While a corporate address got involved parties on the same page in understanding the issue at hand, the importance of the task was more acutely felt when individuals were notified personally of their shortfalls.
This project highlighted the challenges of implementing a new electronic system with more resistance from the senior doctors who were more used to a paper-based system. It also showed that with persistence and appropriate reminders old habits and routines can be changed but it needs to be done in a professional manner with due respect given.5
Work culture should change in the light of emerging evidence and technologies. At times, It unfortunately does not change because people will not change and are comfortable in their ways of doing things. While the onus is ultimately on the individual to recognise the need to change and take responsibility for it, the institution leaders and senior management need to highlight the shortfalls of the status quo, show the involved parties what can be achieved and ultimately champion the vision by setting clear examples to follow. This, the authors believe, would be the key to sustained change.
In terms of limitations, this project was completed within one rotation of doctors and the sustainability with appropriate handovers and education is an important aspect which was not assessed in this timeline. We were also not able to compare and account for natural process variation; the introduction of an electronic system will take time to adapt to and the improvement of the results could have been realised over time after initial teething issues have been dealt with. The authors believe that such a time would be prolonged and the interim period poses compromises to patient safety and is not ideal.
This new electronic system for results acknowledgement has reduced the risk of missed paper-based results and with the interventions through this project has improved the compliance and utility of the system. This system however is still dependent on the individual physician doing due diligence with checking and acting on abnormal results. For this to be improved further, we would need to work with the IT department to perhaps start the EMRs system with a popup reminder of the unacknowledged results before allowing the physician to continue using the system.
Conclusion
In conclusion, multiple factors were identified to contribute to the low compliance to electronic acknowledgement of results.6 The role doctors play in the issue at hand was paramount and required careful handling in a professional manner with multiple reminders and emphasis on the importance of acknowledging and acting on the results.
A significant improvement in the rates of acknowledgement of abnormal results was demonstrated with clear benefits to patient safety. Interventions can be replicated when implementing similar systems to other areas of healthcare.
Acknowledgments
The authors acknowledge the contributions from other members of the team including Ms Branda Lim who provided input with regards to logistics on the ground and how to better optimise it, Ms Ori Dwi Priyanthi from the medical records office for assisting with the tracing and tracking of physical patient medical records, Ms Fety Kie from the Information Technology department for invaluable input with the ongoing monitoring the numbers of unacknowledged results. Last but not least Professor Wong Tien Yin, Medical Director of Singapore National Eye Centre, for supporting this initiative.
Footnotes
Contributors VP, BA and YQS designed and conducted the study including data collection and data analysis. VP prepared the manuscript draft with intellectual input from BA, YQS and RH. All authors had access to the study data.
Provenance and peer review Not commissioned; externally peer reviewed.