Article Text
PDF
Abstract
Ordering and protocolling CT scans after-hours from the emergency department (ED) at our institution previously required discussion between the ED physician and radiology resident, which led to workflow inefficiency. Our intervention consisted of creating an electronic list of CT requests that radiology residents would monitor. Radiology protocolled straightforward requests and contacted the ordering physician for more details when required. We aimed to improve workflow efficiency, increase provider satisfaction and reduce CT turnaround time without significantly affecting CT utilisation. Plan-do-study-act cycles were used to plan and evaluate the intervention. The intervention was initiated on weekday evenings and then expanded to weekend hours after an interim analysis. Qualitative outcomes were measured via electronic survey, and quantitative outcomes were collected from administrative data and analysed via control charts and other statistical methods. Survey response was high from ED physicians (76%, n=82/108) and radiology residents (79%, n=30/38). After the intervention, the majority of ED staff and radiology residents perceived improved workflow efficiency (96.3%, 73.3%), radiology residents noted a subjective decrease in disruptions (83.3%) and most ED staff felt that scans were performed more quickly (84.1%). Radiology residents received fewer pages per shift, adjusted for scan volume. There was a reduction in time from order entry to protocol on weekday shifts only, with no statistically significant effect on time from order entry to scan. Segmented regression analysis demonstrated a background increase in utilisation over time (0.7–2.0 CT/100 ED visits/year, p<0.0005), but the intervention itself did not contribute to an overall increase in CT utilisation. In conclusion, our intervention led to improved perceived workflow efficiency and reduced pages. Scans were protocoled more quickly on weekdays, but turnaround times were otherwise not significantly affected by the intervention. Background CT utilisation increased over time, but this increase was not attributable to our intervention.
- diagnostic services
- emergency department
- quality improvement
- healthcare quality improvement
- control charts/run charts
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- diagnostic services
- emergency department
- quality improvement
- healthcare quality improvement
- control charts/run charts
Problem
Ordering and protocolling CT scans after-hours from the emergency department (ED) at our institution previously required discussion between the ED staff and radiology resident. The ordering and protocolling process involved multiple steps, calls and pages, interruptions to workflow and duplication of work, which led to perceived workflow inefficiency. Figure 1A illustrates the previous after-hours CT ordering and protocolling workflow.
Process map flowchart illustrates CT ordering and protocolling workflow prior to intervention (A) and postintervention (B). Previous workflow (A) included several steps with high likelihood of interruption to radiology and ED workflow. Implemented workflow (B) involved the radiology resident monitoring a list (dotted lines in B) as a replacement for several steps in the previous workflow (dotted lines in A) for most cases. ED, emergency department; EPR, electronic patient record; MRN, medical record number.
The Joint Department of Medical Imaging (JDMI) is one of the largest medical imaging departments for teaching hospitals in Canada and serves three EDs (Toronto General Hospital, Toronto Western Hospital and Mount Sinai Hospital). These hospitals have a combined annual ED volume of over 182 000 visits, which is growing by 2%–3% annually. These EDs are staffed by board certified emergency medicine physicians and house an emergency medicine residency programme. During the after-hours period (17:00–08:00 hours on weekdays, 24 hours on weekends), the radiology resident serves as the point person for triaging imaging requests, protocols the studies and provides preliminary reports for inpatient and ED studies.
Increased ED volumes and imaging utilisation has led to increased time pressures for both ED staff and radiology residents. The previous after-hours CT ordering and protocolling process, which was introduced when imaging volumes were much lower, was felt to be unsustainable due the disruptive nature and time cost of pages and return phone calls. We streamlined the ordering and protocolling process by removing the requirement for direct discussion between ED and radiology for CT studies and creating an electronic list of ED requests that would be monitored by radiology. Radiology residents protocolled straightforward requests and called the ordering physician when a clinical discussion was required. The aim was to improve workflow efficiency, reduce the number of pages received by the radiology resident and improve ED CT turnaround time while not significantly altering CT utilisation.
Background
The use of CT in the ED has increased over time in North America and many other developed nations worldwide.1–5 This extends to the after-hours period,4 5 resulting in increased workload for both emergency and radiology departments after-hours. At most academic institutions, including our own, radiology residents perform most after-hours duties including protocolling studies and providing preliminary reports.6 Not surprisingly, increased utilisation has resulted in increased radiology resident workload.7
At our institution, CT scans ordered from the ED after-hours previously required a direct conversation between the radiology resident and ED staff prior to being protocolled. In our experience, this is the case in many academic institutions without a dedicated Emergency Radiology department. Given this requirement, increases in ED CT utilisation have naturally resulted in increases in pages and phone conversations. These interruptions are detrimental to the workflow of both ED staff and radiology residents. For radiology residents, telephone call interruptions lead to decreased diagnostic accuracy8 and increased report turnaround time out of proportion with time spent on the phone.9 Time waiting for radiology is a known bottleneck in the ED10 and shorter length of stay improves patient satisfaction.11 For these reasons, improving the CT ordering process in the ED is of great interest.
There are no quality improvement reports in the literature that, to our knowledge, address this issue adequately. Specifically, there have been no reports of an intervention in this setting followed by assessment of perceived workflow efficiency, CT turnaround time or number of pages. One group attempted to introduce a preauthorisation policy for after-hours CT requests, focusing on the effect on CT utilisation.12 The study reported an increase in utilisation postintervention, but background increases in utilisation over time were not accounted for.
Measurement
The intervention was evaluated through qualitative and quantitative measures. Qualitative measures included surveys of both ED staff and radiology residents to assess support for the change and perceived effect on clinical workflow efficiency. During the plan-do-study-act (PDSA) cycles, informal polling of the staff from both departments was done to identify any dissatisfaction or other issues that arose from the workflow change (ie, confusion regarding the new workflow, grievances with perceived workflow effects). A final analysis on the impact of the intervention on perceived workflow efficiency and satisfaction was performed via voluntary electronic survey.
Quantitative measures evaluated the effect of the intervention on volume of pages received by the radiology resident, CT turnaround time and CT utilisation. Data regarding the number of pages received by the radiology resident were collected from the switchboard from two of the three hospitals. The number of pages was collected for a 12-week preintervention control period (1 May–21 July 2017) and 12 weeks of paging data during each PDSA cycle (30 April 30–20 July 2018; 24 September–17 December 2018). Paging data were unavailable from one of the hospitals (Mount Sinai Hospital). The data also could not distinguish between pages from inpatient wards and the ED.
CT turnaround time and utilisation data were collected starting 30 January 2017 to establish a baseline and then monitored throughout 2018 as the new process was introduced. CT turnaround time was calculated using event timestamps for time of order entry, time of protocol and time of scan, all collected from the JDMI RIS (Radiology Information Systems). To assess utilisation, all ED CT scans performed during the time period were collected from JDMI RIS. ED visit volumes were collected from Decision Support, which oversees various administrative data, to control utilisation for ED volumes. These metrics were collected on an ongoing basis and analysed through statistical process control (SPC) charts. To establish baseline measurements, we calculated turnaround times and CT utilisation for a 12-week preintervention control period (1 May–21 July 2017). As with paging data, we planned to assess 12 weeks of data during each PDSA cycle (30 April–20 July 2018; 24 September–17 December 2018).
Preintervention, radiology residents received an average of 23 pages per weekday shift (1700–2000) and 79.2 pages per weekend shift (0800–2000). Average preintervention time to protocol and time to scan was 33.3 and 134.6 min on weekdays and 35.6 and 123.3 min on weekends. Preintervention weekday and weekend 24 hours utilisation rates were 6.3 and 5.5 CT scans requiring protocol per 100 ED visits, respectively.
Paging, turnaround time and utilisation data preintervention and during each PDSA cycle were first compared using two-tailed t-tests. To adjust for time-related and background effects on turnaround time and utilisation, we performed segmented regression analyses after each PDSA cycle. Given that CT utilisation rates are known to have increased over time, this would better assess whether the intervention itself had an effect on these variables.
Design
Our intervention involved creating an electronic list of ED CT orders that was monitored by radiology residents after-hours. Straightforward orders were protocolled directly from the list, and for requests that were unclear or more complex the radiology resident contacted the ordering physician to discuss the case further. ED physicians were instructed to include more clinical information in the computerised order entry (CPOE) system at the time of order entry. The ED physician paged the radiology resident if an order needed to be expedited or discussed directly. Figure 1B illustrates the implemented workflow, and highlights the steps replaced by the new process.
The proposed intervention was discussed with key stakeholders from both the Radiology Department and Emergency Department. Through a number of meetings, these discussions included: the chief radiology resident, chiefs of each of the EDs, the JDMI radiologist-in-chief, the radiology residency programme director and residency programme committee, selected ED staff, selected radiology residents and CT technologists. A final policy statement outlining the proposed changes was approved by both departments. The policy changes were disseminated to the radiology residents and the emergency physician group by email and departmental meetings.
The intervention was first piloted on weekdays between 17:00 and 20:00 hours starting on 30 April 2018. Following this intervention, feedback was solicited from both radiology residents and ED physicians. Data were continuously monitored to identify changes in CT utilisation and turnaround time. The intervention was expanded to include weekend daytime hours (08:00–20:00 hours) on 29 September 2018.
From discussions with ED physicians and radiology residents, we felt that the removal of the required phone call for each study would lead to improved workflow for both parties. However, we identified potential issues with the new process. A lack of clinical details on imaging requisitions might lead to mis-protocolled studies, inappropriate triaging of more emergent requests or a high proportion of calls required for clarification. This was addressed by emphasising the importance of a clear clinical indication in the CPOE. We also stipulated that emergent scans requiring expedited imaging still required direct communication between ED staff and the radiology resident. Given that radiology residents had to monitor a new electronic list of CT requests, we feared that requests may occasionally go unnoticed, leading to delays in care. We instructed residents to monitor the list of requests at least every 30 min, with the expectation that all CT scans be protocolled within 60 min of order entry.
Given our detailed discussions with impacted stakeholders, addressing of anticipated issues upfront, and continued monitoring for issues requiring adjustments, we believed that the intervention would have high uptake and sustainability among both the ED and Radiology departments.
Strategy
PDSA cycle 1
A stakeholder analysis was performed with leaders from the ED and radiology departments starting in January 2018. Discussions covered the scope of the initiative, hours that the process would apply to, new workflow procedures for each department and implementation strategy. Both departments agreed that the current process led to undue workflow disruptions for both the ED and radiology, and the benefits of having each study discussed prior to protocolling were limited to a minority of studies. It was decided that the intervention would be deployed in a phased manner. Phase 1 would have the process introduced for 3 hours every weekday, from 17:00 to 20:00 hours. This would give both departments time to familiarise themselves with the new workflow and to observe for any unintended negative consequences. Once the plan was finalised, each department introduced the proposal to their respective providers in March 2018. In the ED, this included a presentation at the monthly mandatory business meeting to solicit feedback, and discussion and reminders via email thereafter. The ED group was overwhelmingly supportive of this initiative. For the radiology department, a formal presentation was given at the Residency Program Committee meeting, where after discussion and feedback, it was unanimously supported by the committee. This was also presented at the monthly radiology operations meeting, where the change was discussed with technologists and operations managers. Multiple reminder emails were sent to key stakeholders prior to introducing the new system on 30 April 2018.
After the process was introduced, members of the project team tracked workflow issues (ie, study requests that were not acknowledged by the radiology resident, unnecessary paging to the radiology resident) and provider grievances with the new process. CT technologists noted an increase in calls received directly from the ED staff regarding the status of previously ordered studies. Periodic reminders were sent to the ED providers about the new process change, as some were still unclear about the new study request workflow. In the following 5 months, both departments strongly supported the new ordering process and felt that workflow efficiency had improved. Interim analysis of quantitative measures (volume of scans performed, time from order entry to protocolling, time from order entry to scan completion) did not demonstrate a significant change. Although the average time from order entry to protocol did not significantly change, occasional outliers were noted where time to protocol exceeded 60 min. The majority of these were instances where the radiologist felt further clinical information was required prior to protocol. To prevent delays due to missed requests, during the first PDSA cycle, a new process was implemented by which technologists would monitor requests and notify residents of studies that had not been acknowledged within 30 min. The project team then decided to proceed to extend the new process to the weekend days.
PDSA cycle 2
Given the success of the first limited implementation and the apparent lack of significant negative consequences, both departments agreed to proceed to a wider rollout of the process. The process was introduced on the weekends (08:00–20:00) starting 29 September 2018. Similar to the previous PDSA cycle, the project team monitored for any obvious negative effects to workflow or provider satisfaction, of which there were none. A more detailed evaluation of the intervention was performed to analyse both qualitative and quantitative measures 3 months after the deployment of the second phase.
Results
Qualitative measures
We created and distributed specialty-specific online surveys to ED attendings and radiology residents. We received survey responses from 76% of ED staff (82 of 108) and 79% of eligible radiology residents (30 of 38). The vast majority of ED staff and radiology residents supported the initiative (99% and 80%) and noted subjectively improved workflow efficiency (96% and 73%). Most radiology residents indicated decreased disruptions (83%) and increased time for imaging interpretation (70%). Most ED staff felt that scans were performed more quickly postintervention (84%).
Quantitative measures
Page volume
Radiology residents received fewer pages per shift postintervention on weekdays (23 preintervention; 19 postintervention, p=0.001), but not on weekends (79 preintervention; 75 postintervention, p=0.17). When adjusting for CT utilisation, page volume (pages/CT scan protocolled) was significantly lower postintervention on weekdays (5.3 pre-intervention; 3.1 postintervention, p<0.001) and weekends (7.2 preintervention; 4.1 postintervention, p=0.023).
CT turnaround time
Time from order entry to resident protocol (time to protocol) and time from order entry to the start of the scan (time to scan) were assessed. When comparing preintervention and postintervention time periods, there was a reduction in time to protocol on weekday shifts postintervention (33 min to 24 min, p<0.05), but not on weekends (36 to 33 min, p=0.47). Time to scan was not significantly different between preintervention and postintervention time periods on weekdays (135 to 125 min, p=0.053) or weekends (123 to 116 min, p=0.43). When the regression model was applied, the effect of the intervention on time to protocol and time to scan did not reach statistical significance on either weekdays or weekends. The regression results are shown in table 1A.
Parameter estimates and p values of the segmented regression models to predict daily CT turnaround times (A) and daily CT utilisation rate over time (B)
CT utilisation
SPC charts for weekday and weekend CT utilisation per 100 ED visits are shown in figure 2. Although utilisation appeared to increase postintervention for both weekdays and weekends, none of the data points show any patterns, trends or data points outside the upper and lower confidence limits to suggest that special cause variation is present.
Control chart of 24 hours CT utilisation for weekdays (A) and weekends (B). The average utilisation rate for each period is represented by the red line. The UCL and LCL are defined as 3 SD above and below the mean, respectively. LCL, lower confidence limit; UCL, upper confidence limit.
Without controlling for background changes in utilisation, comparison of preintervention and postintervention time periods revealed a higher number of CT scans protocolled per 100 ED visits in the postintervention time period. Weekday daily utilisation rates during the intervention hours (17:00–20:00) increased from 1.04 to 1.33 scans per 100 ED visits (p<0.001) and increased from 6.25 to 6.95 scans per 100 ED visits (p<0.001) over the entire day. Similarly, weekend daily utilisation rates increased as well during the intervention hours (08:00–20:00) from 3.34 to 4.13 scans per 100 ED visits (p<0.001), and increased from 5.54 to 6.58 over the entire day (p<0.001). Whether the increase in utilisation was due to the intervention itself or background increases in utilisation over time independent of the intervention was not clear based on this analysis.
Using segmented regression analysis of an interrupted time series and separate analyses for weekdays and weekends due to differences in workflows, we analysed the effects of background time (ie, trends in CT utilisation over time) and the intervention itself on CT utilisation. This methodology is powerful in this setting as it facilitates detection of both immediate and gradual effects of our intervention on utilisation. The regression results are shown in table 1B.
There was a background increase in CT utilisation over time for both weekdays and weekends independent of the intervention, with increases of 0.78 CT/100 ED visits/year on weekdays and 1.99 CT/100 ED visits/year on weekends. The intervention itself did not result in a statistically significant immediate or gradual effect on overall CT utilisation over 24 hours periods on weekdays or weekends. When analysis was isolated to the shift times directly affected by the intervention, the intervention led to increased utilisation on weekend day shifts (08:00–20:00) (0.732 CT/100 ED visits, p<0.05). This suggests that orders were concentrated during the time that the process was in place, with an artificial surge in orders during 08:00–20:00 but not an overall increase in a 24 hours weekend day.
Lessons and limitations
The strength of this project involved the use of minimally invasive interventions to achieve our targeted goals. The removal of a workflow process for both the ED staff and radiology residents on-call was met with overwhelming positive feedback when it was initially proposed. Furthermore, the creation of a dedicated on-call protocol list with the expectation of review every 30 min demonstrated only a minor change to workflow for the radiology resident. The simplicity of the proposed intervention was critical to the adoption, ongoing sustainability, and success of the intervention. This ultimately led to both departments formally agreeing to continue the process. We believe our intervention is sustainable, but continued monitoring will be required. Potential barriers to sustainability include increases in CT utilisation and workflow disruptions, which would necessitate further intervention.
We recognise that there are a number of solutions for ED-radiology department interactions (ie, hanging automatic protocol, radiology coordinators and so on). Our specific study is generalisable to academic institutions which commonly have resident trainees take on the after-hours responsibility and approve requests from the ED. Our study is less generalisable to community practice models.
There were a number of limitations to our study. We relied on a voluntary survey from providers to assess satisfaction and perceived workflow efficiency, which introduces response bias. However, given the high response rate overall, we feel that the survey results accurately reflect the satisfaction of each department as a whole. Although providers felt that clinical workflow efficiency improved postintervention (including decreased disruptions and increased time for imaging interpretation), not all parameters could be objectively measured. Such an undertaking would be too resource intensive for our purposes, and having assistants hovering over providers to track these events may lead to negative impacts in clinical operations. Quantitative analysis was limited by the quantity and quality of data available. Comparisons of preintervention and postintervention periods were based on relatively short time periods (12 weeks each), though utilisation and turnaround time data were analysed for a 23-month period with segmented regression analysis. Paging data were imperfect as we were unable to distinguish between pages from the ED and other inpatient services, and paging data were unavailable for one of three EDs.
Given the pragmatic nature of our study, we could not control for any additional systemic factors that may have influenced the outcome measures during the study period. To our knowledge, there were no significant relevant systemic changes to either the ED or radiology department during our study period. Finally, the relatively short follow-up period limits our assessment of sustainability of our intervention and further monitoring will be required.
Conclusion
Gradual increases in after-hours CT utilisation rendered our previous CT ordering and protocolling process unsustainable for workflow in both the ED and radiology departments. Our intervention was necessary and has proved beneficial for both departments.
Our intervention largely achieved its aims. Both radiology residents and ED staff indicated overwhelming support and improved workflow efficiency. Radiology residents indicated a decreased number of disruptions and increased time to interpret imaging studies. ED staff subjectively felt that CT scans were being performed more quickly postintervention.
Radiology residents received fewer pages per CT scan on both weekdays and weekends. Although there was a decrease in time to protocol on weekdays postintervention, segmented regression analysis demonstrated no change in turnaround times on weekdays or weekends. CT utilisation increased over time independent of the intervention, but the intervention itself did not result in an increase in overall utilisation on weekdays or weekends.
The simplicity of our intervention and benefits to ED and radiology workflow were critical to the adoption, ongoing sustainability and ultimately the success of the intervention.
Acknowledgments
We would like to thank Dr Heidi Schmidt, Dr Sam Sabbah, Dr Lucas Chartier, Dr Massimo Tarulli for help with implementation of changes to workflow.
Footnotes
Contributors RB and JC lead the quality improvement initiative, were the primary organizers of all efforts and contributed to all aspects of the study. CDW also helped with this process and helped collect and compile data. RB, CDW and JC were the primary contributors in writing the manuscript. RJM ensured data were accurately procured from the appropriate databases and provided statistical support. ESB helped with implementation of the initiative.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon request.