Article Text
PDF
Abstract
The Oxford University Hospital NHS Trust (OUH) had introduced a policy to improve the timely endorsement of diagnostic tests. However, performance in the Oxford Kidney Unit (OKU) has been consistently below the OUH target of 85%. This project was undertaken to improve endorsement within the OKU. Weekly percentages of all diagnostic test results that were endorsed within 7 days of reporting were monitored as our main outcome measurement. During the intervention period, four plan–do–study–act (PDSA) cycles were undertaken each lasting 6 weeks. Introduced changes included interventions to develop a team-based approach and practical tools to enhance compliance, such as creating clinical worklists, a guidance document on endorsement and an endorsement newsletter. Data was monitored for a further 6 months beyond the intervention period to ensure improvement was sustained. There was a significant improvement in endorsement to above 85% by the end of the second PDSA cycle. This was maintained throughout the project and for a further 6 months beyond the intervention period. Our systematic approach to improving the endorsement of results is potentially transferable to other healthcare organisations using electronic healthcare records for clinical care.
- Clinical Audit
- Continuous quality improvement
- Electronic Health Records
- Process mapping
- Quality improvement
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Clinical Audit
- Continuous quality improvement
- Electronic Health Records
- Process mapping
- Quality improvement
WHAT IS ALREADY KNOWN ON THIS TOPIC
Studies show that the lack of robust endorsement system of medical investigations results in less than optimal care and current processes are not enough to ensure compliance with endorsement policy.
WHAT THIS STUDY ADDS
Our study shows that a team-based approach to endorsement alongside simple practical solutions can improve engagement and endorsement rates in the long term.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Its versatility means that it is directly transferrable to other healthcare systems using electronic patient record.
Problem
Oxford University Hospital NHS Trust (OUH) moved to use the electronic patient record (EPR)—a Cerner Millennium software for all patient care, including ordering diagnostic tests.
All tests are requested by healthcare professionals on EPR and the results of these diagnostic tests are then displayed within the diagnostic section of EPR, accessible to all clinical staff. The results are also sent directly to the EPR messaging centre inbox of the requester. Furthermore, results from radiological investigations are available on a separate platform outside EPR, which is more commonly used by clinical staff. There are therefore several ways in which diagnostic test results can be accessed by clinical staff. There is no clear audit trail of who accessed them and whether these results were acted on unless such actions are documented within patient records. Tracking individual patient’s records for audit purposes would not be practical or sustainable. Several serious incidents have been reported within our organisation, where the untimely endorsing of results and lack of an audit trail have compromised patient safety.
In response to this, OUH developed a ‘closing the loop on diagnostic test’ policy to improve the timely endorsement of results and action. It requires all requesters to ensure results are endorsed via their EPR messaging centre inbox within 7 days of them being issued, which enables a clear audit trail. There were no official rules to mandate the policy and the informal approach to encouraging staff to endorse results brought about only transient improvements. The performance of the Oxford Kidney Unit (OKU) was at 78% between August and December 2020. Within the OKU, the renal ward was the lowest performing area at 59%, whereas the day case unit performed at 93%. The specific aim of this project was to improve the endorsement of diagnostic tests within 7 days to 85% over a period of 6 months and to monitor the performance over a further 6 months after the project ended to ensure the sustainability of improvements. This project was limited to the renal day case unit and the renal ward.
Background
WHO has identified that failure to follow-up on test results is a critical patient safety issue and test result management is a priority area—World Alliance for Patient Safety Forward Programme.1 A systematic review reported that 20%–62% of diagnostic results are not followed for patients admitted to the hospital, especially for those patients being subsequently transferred to a different clinical setting.2 This contributed to missed or delayed diagnoses and delayed or failed patient care. They concluded that these issues should be tackled by interventions such as an online endorsement of diagnostic investigations. However, it has been suggested that in addition to the provision of electronic endorsement, a more robust clinical governance process with engagement from all stakeholders is required to bring the desired impact.3 This view is supported by another study, which suggested the results review may improve by streamlining access to the results endorsement platform and improving the handover of responsibility of the results review.4
Measurement
The data was provided by the central audit system called ORBIT. This system extracts data from EPR using standardised, trust-wide data extraction routines. The system also provides access to qualitative data describing compliance with endorsement requirements by each clinical area within the organisation for different groups of healthcare professionals such as nurses, junior doctors, specialty trainee doctors and consultants. Information on the identity of individual requesters is not available.
The following two outcomes measures were monitored weekly:
Percentages of all diagnostic test results that were endorsed on renal ward and renal day case unit within 7 days.
Percentages of all diagnostic test results that were endorsed by each staff group—consultants, registrars, junior doctors, nurses within 7 days.
As a process measure, feedback from team members was obtained as to whether the endorsement task by the clinical team was being completed as planned for the day. As a balancing measure, it was important to consider that not all endorsements may be followed up by appropriate clinical intervention. This could occur when junior members of the clinical team feel obliged to endorse test results requested by other healthcare professionals but may feel less confident or lack the knowledge to act on abnormal results. A ‘guide document for endorsing results’ (please see online supplemental material) was developed to support them in seeking help where required and made available to all clinical staff. It encouraged team members to use the current facility within EPR to endorse and forward results to senior clinicians overseeing a patient’s care if they needed advice regarding further action.
Supplemental material
Statistical process control (SPC) charts were created to establish a temporal relationship between the interventions and outcomes. This was generated at the start of the project and updated and analysed at regular weekly intervals to monitor the progress of the project, to analyse and to identify any barriers during the monthly team meetings.
All data regarding the endorsement of the results was anonymously exported from EPR to the ORBIT data warehouse and reporting system. Only this anonymous data was used in the analysis, therefore no patient confidentiality was breached.
Design
A results endorsement workflow was developed to identify and delineate all possible vital checkpoints at which diagnostic tests could be requested, where endorsing diagnostic test results is at risk, and how to overcome any barriers to endorsing them by introducing targeted interventions (figure 1A). The ‘five whys’ was used as a model to understand the root causes for why results are not being endorsed at each vital checkpoint to ensure interventions can be tailored towards these reasons. The five main root causes are:
Lack of continuity of care as doctors and nurses frequently change on the ward and day case unit due to shift working. Requesters may have moved to other clinical areas or may be off duty or on holiday and therefore no longer caring for those patients. This was tackled by creating a clinical work list and moving the responsibility to endorse from the requester to the team caring for the patients.
Requesters of diagnostic tests may not be responsible for acting on the results. There is a degree of apprehension from junior doctors or nurses to endorse results as they do not necessarily have sufficient clinical information regarding the patient. For example, when a nurse requests an obligatory investigation for the patient but the responsibility of acting on the results lies with the supervising physician. A guide to endorse with an explanation how to send the results to the responsible senior clinician for review and endorsement was designed as a response (see online supplemental material 1).
Results are uploaded onto the EPR after patients had been already discharged home or transferred to another clinical area or organisation and therefore, they are no longer under active follow-up of the team. A clinical discharge list was set up on EPR for the ward and day case unit separately, which automatically included patients who were discharged from these areas respectively and remained on the list for 2 weeks after discharge.
Endorsement is not considered a priority by staff. Healthcare professionals have access to results through the diagnostic section on EPR without the need to endorse them and therefore do not see any added benefit of endorsement in the day-to-day clinical management of the patients. There is also a lack of agreed time or responsible staff members in each clinical area to endorse the results during the daily work schedule. Furthermore, not all healthcare professionals are necessarily aware of the medicolegal consequences of lack of timely endorsement and of the need for a clear audit trail of such. We have therefore believed that education about the importance of endorsement during the induction of new doctors and fixing the time of endorsing during the day would address these issues.
Lack of adequate training on how to endorse results. Online training on this is included as part of induction into the organisation among several other topics and there is no further reinforcement of this training as staff rotate through the different clinical areas. This was again addressed by further guidance and training on endorsement during departmental induction.
(A) Work flow diagram of endorsement of results for a patient between admission and discharge as done prior to the project. All investigations are requested and results displayed within EPR. Solid blue arrows show the various approaches adopted by clinical teams to endorsing results for a patient between admission and discharge. The width of arrows reflects the frequency of the different approaches adopted. Red arrows indicate factors contributing to poor endorsement at various stages during the admission. (B) Work flow diagram of endorsement of results for a patient between admission and discharge as implemented during the project. Green arrows indicate interventions facilitating endorsement of results. Due to use of clinical worklist, individual patient record endorsement becomes less relevant. Discharge lists prevent results of discharged patients being missed and not endorsed. EPR, electronic patient record; OUH, Oxford University Hospitals.
Strategy
Both formal and informal means of rationalising the data were adopted. During meetings, progress on the SPC was shown and a brainstorming method of generating ideas and sharing knowledge was adopted where each member of the project team was encouraged to share their thoughts. Each meeting concluded with an overview of progress and a detailed plan for the next month.
The intervention phase of the project lasted for 6 months, starting on 22 March 2021 and had four consecutive plan–do–study–act (PDSA) cycles that were completed by 21 September 2021. The results endorsement workflow as proposed in the project is shown in figure 1B.
The first PDSA cycle ran from 23 March 2021 to 3 May 2021. During the first team meeting, it was established that the current endorsement of the results was mainly dependent on the consultants reminding junior staff groups to endorse. It was noted that most of the staff members were not aware of the importance of endorsing or did not know how to endorse the results.
On the basis of the gathered information, guidance documents for endorsing results were developed and shared with all relevant staff groups (see online supplemental material 1). A team-based approach was adopted to the endorsement of the results to overcome the issues of lack of continuity of care, and the fact that requesters may no longer be caring for those patients. Clinical teams in each clinical area were encouraged to use clinical work lists (figure 2) to endorse results rather than individual message centre inboxes. The clinical work lists provided a better display of all outstanding results to be endorsed in a particular clinical area and individual patient’s results could be accessed and endorsed directly from within these lists. This also meant that endorsing was no longer reliant on individual requesters who may have moved to other clinical areas or been off duty due to shift working. Once the results were endorsed by the team, these were cleared from the requesters’ inboxes and therefore avoided the need for additional endorsement by the requester.
Example of a clinical work list showing outstanding results that need endorsement.
To address the issue of missed results of discharged patients who were no longer under an active follow-up, ‘discharged patients EPR clinical lists’ were developed one each for the renal ward and the day case unit. Patients remained on these lists for 2 weeks after discharge, which allowed staff to chase and endorse the results that had not been reported at the time the patient was discharged. These lists were made accessible to all clinical staff within OKU so that it was available on their respective EPR logins.
In addition, the nursing shift coordinator was requested to remind the nursing and the medical teams to endorse results alongside other safety briefings during the morning multidisciplinary handover meeting.
The second PDSA cycle ran from 4 May 2021 to 14 June 2021. In that cycle, a champion was introduced among the junior doctors, whose role would be to nominate junior doctors in each clinical area to take responsibility for ensuring all results are endorsed by the end of the working day. We left it to the individual teams to decide on the appropriate time during the working day when it would be convenient for them to endorse the results for patients under their care. They could carry out endorsement either during ward rounds or at the end of the day, alongside other jobs such as electronic requesting of blood tests for the following day and updating the patient handover list.
Finally, an endorsement newsletter was created to share the results of the performance individually for each staff group. The aim was to encourage the groups with less-than-optimal performance.
The third PDSA cycle ran from 15 June 2021 to 26 July 2021. Our main intervention in that cycle was to reinforce the intervention from the second PDSA cycle regarding the timing of endorsing. Junior doctors were required to endorse before their daily ward round and afternoon handovers and that was monitored by the higher specialty trainee on the ward each day.
The fourth PDSA cycle ran from 27 July 2021 to 6 September 2021. In anticipation of the new rotational junior doctors, a session regarding endorsement was incorporated in all following departmental induction teaching to reinforce the training that was covered as part of online trust induction. All new staff were provided information on contacting the OUH IT team to request adding their names to the staff group and emailing list so that they have automatic access to current and discharged patient lists when they join the renal team.
Between 7 September 2021 and 14 March 2022, data on performance was collected and continued to be shared through internal mailing lists by the senior clinician within the team. No further interventions were undertaken during this period.
The four rules of SPC5 were used to assess special cause variations and whether changes in each PDSA cycle are resulting in improvement and whether this improvement is sustained. The rules applied were data points outside the control limits; a run of seven points above or below the central line (a shift) or a run of seven points all consecutively ascending or descending (a drift); any unusual pattern or trends within the control limits; the number of points within the middle third of the region between the control limits differs markedly from two-thirds of the total number of data points. SPC charts were produced using the NHS England SPC tool.6
Results
The overall percentage of results endorsed varied during the first PDSA cycle and while reaching 86 % at the beginning of the cycle it dropped to 67% at the end of it (figure 3). Overall, nurses and doctors endorsed 71% and 75% of those requested by each staff group, respectively. At the beginning of the second PDSA cycle, there was an improvement seen in the results endorsement among nurses (91%). The results for doctors still remained lower than target (69%) in the first week of the second PDSA cycle. Later, there was a significant positive shift in the endorsement of results to 85%, consistently reaching the trust’s target. In the end, during the second PDSA cycle, nurses endorsed 87% of results and doctors endorsed 97% of the results. The performance remained over 85% throughout the third PDSA cycle, with nurses endorsing 95% of reports and doctors 91%. The average performance in the fourth PDSA cycle was 93%, with nurses reaching 83% and doctors 94%.
Grey solid line indicates mean of endorsement of results prior to QIP and orange dashed line indicates Oxford University Hospital NHS Trust target of 85%. QIP, quality improvement project; PDSA, plan–do–study–act.
Data was monitored for further 6 months after the intervention period from 7 September 2021 to 14 March 2022. Endorsement performance was sustained above trust target. Overall performance was at 92%. Nurses’ performance dropped to 79% during this period but this did not have a significant effect on the overall performance which remained above the trust target at 85%. Requests made by nurses constituted only a small proportion of the total number of requests made in this period. Overall, we were able to maintain the trust target of 85% and above except for two isolated transient periods of low performance.
Several key processes were established which included a team-based approach to endorsement, using clinical patient work lists to help clinical staff easily identify outstanding results that required endorsement, creating a discharged patients list, assigning a champion in each clinical area to support and promote the project, regularly sharing the performance data among relevant staff groups and encouraging a fixed time in the working day to endorse results. We also recognised that due to the rotational nature of the training of junior doctors, sessions to reinforce the importance of results endorsement need to be incorporated as part of induction to each clinical area. Furthermore, involvement of a senior clinician who is permanently placed within the department provides an oversight and continuity to the project, when some of the team members leave the department due to the rotational nature of placements.
The first trend in overall improvement was observed following the second PDSA cycle interventions—championing in each area and sharing performance data. These improvements were facilitated by the changes introduced during the first PDSA cycle such as education on how to endorse via the endorsement guide through the clinical worklist. The clinical worklist with the endorsement tab showing all the patients grouped together with an overview of who needs endorsing made endorsing practical and less time-consuming than the process of checking individual patient record or endorsing through the requester’s inbox. A walk-through of the endorsement delineated in the endorsement guide allowed to address the issue of technical difficulties the staff might face, especially when using complex EPR software (see online supplemental material 1).
It was recognised that although the overall performance of endorsement of results within 7 days improved in the second PDSA cycle, the task of endorsement was not undertaken consistently every day. This was attributed to lack of time during ward rounds, competing priorities or forgetfulness of the staff. The latter usually occurred when the task was left for later in the day and subsequently forgotten or not handed over to incoming doctors at the end of the shift. We therefore encouraged a fixed time for endorsement at the beginning of the working day, straight after the morning multidisciplinary team handover and before the ward rounds. This also alerted the clinical team to any abnormal results from the previous day that required intervention during the ward rounds. When the topic of results endorsement was raised among other patient safety initiatives such as antibiotic reviews and venous thromboembolism prophylaxis, the clinical staff understood the importance of doing it consistently and accurately.
After the first PDSA cycle, the overall performance and doctors’ performance remained well above the trust target throughout and beyond the time of the project. Results endorsement among nursing staff improved significantly well ahead of other staff groups during the first PDSA cycle which probably related to intervention of daily briefing of the project during nursing handover by the nurse coordinators. This improvement among nursing colleagues was not sustained during the monitoring phase of the project. Lack of sustained change within a nursing team might be due to a shift of focus between our project and competing projects to achieve other trust targets.
There were other setbacks and periods of less than expected performance. For example, we spotted a transient drop in performance in December 2021 during the third wave of the COVID-19, when the NHS was affected by extreme staff shortages due to sickness. Another particularly important and common problem stemmed from junior doctors rotating between wards every four to 4–6 months. Having foreseen the possibility of deterioration of performance with a new group of doctors, we dedicated time during departmental induction for training around endorsement. However, the induction and its impact on staff attitudes and behaviour may not be evident until a few days after they have started working in the department resulting in a transient drop in performance in February 2022.
At the end of 12 months from the start of the project, in March 2022, data showed sustained overall improvement with endorsement rates above 85%. All but one project team member moved on to other clinical areas in August 2021. The only intervention that continued beyond the initial 6 months intervention phase was sharing of performance data through an email newsletter with the support of a senior clinician within the department. This helped maintain awareness of the project among incoming clinical staff.
Performance has been maintained at 92% according to the latest data in September 2022 at the time of writing this report. This has been achieved through peer support from current trainee doctors sharing existing good practice in team-based endorsement, departmental induction for incoming junior doctors and monthly newsletters on performance data which have all been now established as routine processes within OKU.
This project did not have any opportunity cost. It is thought that team-based approach to endorsing and using clinical worklists streamlined the already obligatory endorsement process—the responsibility to endorse was taken away from individual requester, who, if not knowing the patient very well, had to spend more time on endorsing, to usual team members who were able to endorse more efficiently.
Lessons and limitations
To our knowledge, this is the first quality improvement project to systematically address the barriers to follow-up on patients’ diagnostic tests within an electronic healthcare record. We were able to demonstrate sustained improvement in results endorsement to reach our trust target of 85% over a period of 12 months.
One of the strengths of this project is the involvement of a multidisciplinary team—nurses, doctors at various stages of training and a senior clinician, to facilitate staff engagement and mentorship. Change was therefore driven by all staff groups involved with patient care which routinely request investigations.
Second, the improvement of results endorsement is considered a trust-wide initiative, which ensured effective governance and senior support. Thanks to this, we had unlimited access to the already established central automatic ORBIT data, which streamlined the access to endorsement results platform data. The endorsement newsletter we created, and which is still active, enabled team members to monitor, reflect and share performance data on a regular basis.
The endorsement process that existed before our projects proved to be ineffective in achieving a required performance as it largely relied on senior clinicians’ individual efforts to periodically remind clinical staff of the importance of results endorsement.
In our project, we followed an agreed methodology to define the problem, understand the reasons behind it, and implemented interventions through small tests of change. We used the patient workflow and the ‘five why’ to establish the root causes for the lack of endorsement at various time points in the patient pathway.
It is perceivable that similar barriers to endorsing diagnostic test results are widely prevalent in other healthcare organisations and our systematic approach and some of the solutions identified are applicable to any healthcare organisation that uses electronic healthcare records for patient care. However, the process changes we introduced may not be reproducible in certain settings. For example, some specialities may have patients spread across the organisation in different wards and this would require engagement of a wider team outside a single specialty or clinical location. Furthermore, it would also not be possible to create a ‘discharged patient list’ based on location as done in our project. However, there may be options within electronic healthcare records to create such patient lists based on the consultant overseeing patients’ care.
One of the limitations of this project stems from the rotational nature of junior doctors within the NHS. Due to the changeover, we were challenged with recruiting new staff members to complete the project and this resulted in some discontinuity throughout the project. A preplanned commitment to the project through recruitment of incoming doctors needs to be considered to facilitate smooth transition of the project team.
Conclusion
In conclusion, our project had a positive impact on the safety of patients. A sustainable improvement in results endorsement was achieved through a team-based approach involving multiple healthcare professionals. Communication and collaboration within the whole multidisciplinary team allowed us to create a culture where endorsement of diagnostic tests is not considered a mundane, unnecessary task but an important step in ensuring patients’ safety.
This project could be further expanded by creating an automatic electronic prompt within EPR which would act alongside other patient safety initiatives which are already incorporated into EPR task prompts, such as the antibiotic review, and the venous thromboembolism or cognitive assessments.
Overall, our approach is directly transferable to other healthcare organisations which use electronic healthcare records for clinical care and could be implemented both within our trust and beyond it.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
This project was registered as a trust safety initiative priority and therefore did not require ethical approval.
Acknowledgments
We would like to thank our nursing colleagues and doctors who contributed to the project. We would like to specially thank Vicky Bint, Natalie Picken, Giada Donnini, Tinu Abraham, Vishani Loyola, Allie Thornley.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @jofilo55
Contributors This article was written by MW with support of JG and supervision of UPU. UPU is the guarantor for the overall content. The design of the project and plan–do–study–act cycles were developed by UPU and supported by MW and JG. JG collected the data from the Orbit system. MW conducted quality control of the data and performed analysis of it. MW and UPU created charts and illustrations in the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.