Article Text
PDF
Abstract
Importance Electronic health record (EHR) clinical decision support (CDS) tools can provide evidence-based feedback at the point of care to reduce low-value imaging. Success of these tools has been limited partly due to lack of engagement by busy clinicians.
Objective Measure the impact of a time-saving quality improvement intervention to increase engagement with a CDS tool for low back pain imaging ordering.
Design, setting and participants We conducted a quasi-experimental difference-in-differences analysis at (BLINDED), examining back pain imaging orders from 29 May 2015 to 07 January 2016. The intervention site was (BLINDED) Emergency Medicine/Urgent Care Center (n=5736) and control sites included all other (BLINDED) hospitals and clinics (n=1621). In May 2015, the Department of Health Services installed a CDS tool that triggered a survey when clinicians ordered an imaging test, generating an ‘appropriateness score’ based on the American College of Radiology guidelines. Clinicians often bypassed the tool, resulting in ‘unscored’ tests.
Intervention To increase clinician engagement with the tool and decrease the rate of unscored imaging tests, a new policy was implemented at the intervention site on 15 August 2015. If clinicians completed the CDS survey and scored an appropriateness score >3, they could forego a previously mandatory telephone call for pre-imaging utilisation review with the radiology department.
Main outcomes and measures We used EHR data to measure pre–post-intervention differences in: (1) percentage of unscored tests and (2) percentage of tests with high appropriateness scores (>7).
Results Percentage of unscored tests decreased from 69.4% to 10.4% at the intervention site and from 50.6% to 34.8% at the control sites (between-group difference: −23.3%, p<0.001). Percentage of high scoring tests increased from 26.5% to 75.0% at the intervention site and from 17.2% to 22.7% at the control sites (between-group difference: 19%, p<0.001).
Conclusion Workflow time-saving interventions may increase physician engagement with CDS tools and have potential to improve practice patterns.
- decision support
- clinical
- healthcare quality improvement
- quality improvement
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Reducing low-value care—defined as patient care that provides no net benefit in specific clinical scenarios and can harm patients—will reduce waste and decrease harms to patients, but is challenging to accomplish.1–3 One quintessential example of low-value care is diagnostic imaging for low back pain, accounting for 10% of primary care visits and costs approaching $100 billion per year.4 In addition to the financial burden, such imaging can lead to actual patient harm in the form of unnecessary surgeries and procedures.5–8 Despite published evidence-based consensus-driven guidelines detailing that imaging should be reserved for patients who display ‘red flags’ (ie, evidence of a possible serious underlying condition), low-value early diagnostic imaging for low back pain is a widespread problem and has increased substantially in recent decades.9–14
One potential solution to decreasing low-value imaging might be electronic health record (EHR) clinical decision support (CDS). Sophisticated EHR CDS tools can provide real-time, evidence-based feedback to ordering clinicians to inform them when an order is low value, with the intended result that the clinicians cancel low-value orders.15 Unfortunately, CDS has largely failed to live up to its original promise as a cost-effective tool to change physician behaviour; most studies of these CDS tools show only modest reductions in low-value imaging, in part due to their poor implementation into clinician workflows and the perception by busy clinicians as being time-consuming.6 16–20 One systematic review of CDS imaging interventions found that CDS tools are typically only effective when incorporating ‘hard-stops’ that prevent clinicians from over-riding the tool.17 However, such ‘hard-stops’ also slow down workflow, and can contribute to a perceived lack of autonomy, alarm fatigue and clinician burnout that carries over to other clinical situations.17
The goal of this study is to measure the impact of a time-saving workflow quality improvement (QI) intervention on clinicians’ engagement with a CDS tool for low back pain imaging ordering. We hypothesised that a QI intervention that would make workflow easier for clinicians—specifically one without a ‘hard-stop’ and that was instead time-saving—would incentivise clinicians to engage with the CDS tool and improve ordering behaviour and physician practice patterns. The specific aims of this study were to examine the association of the time-saving workflow QI intervention on the rate of unscored imaging tests and the rate of tests that had high appropriateness scores.
Materials and methods
Study design and setting
We conducted a quasi-experimental difference-in-differences (DinD) analysis to evaluate the impact of a time-saving workflow QI intervention initiated on 15 August 2015. The study took place at (BLINDED) Department of Health Services (DHS) network of hospitals and community-based clinics. (BLINDED) is the largest municipal safety net healthcare system in the nation and serves the county’s million residents, with 26% of hospital outpatients being uninsured.21 The intervention site was (BLINDED) Emergency Medicine and Urgent Care Center (n=5736) and control sites included other (BLINDED) departments and all other DHS hospitals and community-based clinics (n=1621).
Participants and data
Study participants included all patients who received imaging orders for low back pain between 29th May 2015 and 7th January 2016 (3 months before and 4 months after the QI intervention). Our team collected and analysed EHR data to quantify the CDS score (see below) on all orders.
Intervention
In May 2015, prior to the QI intervention, DHS implemented a new CDS tool intended to reduce low-value imaging for low back pain. (see online supplemental appendix). The CDS tool consisted of a survey with 2–3 items that prompted clinicians at the time of ordering to answer questions about the patient. Using the CDS survey, an appropriateness score was generated by the EHR using the American College of Radiology guidelines and reported to the ordering clinician at the point of care, with the goal of clinicians cancelling orders with low appropriateness scores. Scores ranged from 1 to 9, with a score of 1–3 considered low, a score of 4–6 considered medium and a score of 7–9 considered high. A high score was generated if the patient had indications of a ‘red flag’ (eg, a patient with active cancer), while a low score would be generated for a patient without ‘red flags’ (eg, a young healthy person with no concerning physical examination findings). If clinicians bypassed the survey (for example, by entering free text) the study would be unscored, undermining the intended impact of the CDS tool.
Supplemental material
With the goal of increasing engagement with the CDS tool and reducing the large (~70%) proportion of unscored tests, the (BLINDED) leadership implemented a QI intervention designed to make using the CDS tool time-saving for busy clinicians. As described in detail in the next paragraph, the intervention consisted of a policy change eliminating a burdensome utilisation review phone call and a potentially challenging conversation with a radiologist.
Prior to the QI intervention, clinicians were required (except in the case of emergency trauma or stroke) to make a phone call to the radiology department to obtain approval for all imaging utilisation orders regardless of whether the ordering clinician had completed the CDS survey. Starting on 15 August 2015, hospital leadership implemented a new policy in which clinicians in the (BLINDED) Emergency Medicine and Urgent Care Center could bypass this phone call requirement if they completed the CDS tool survey, obtained a score, and the score was >3.
Patient and public involvement
We did not directly include patients or the public in this study, but the study protocol was reviewed by the institutional review board of (BLINDED) which includes patient representatives and community members.
Primary outcome measures
We used EHR data from the CDS tool to measure: (1) percentage of imaging tests which were unscored and (2) percentage of imaging tests with high appropriateness scores (>3).
Statistical analysis
Patient characteristics were summarised for the intervention and pooled control sites using EHR data from the evaluation period.
A quasi-experimental (non-randomised) DinD design was used to evaluate the impact of the intervention. The analysis was performed comparing the intervention site and 15 pooled control sites. Marginal logistic regression models were used to determine whether the intervention was associated with: (1) greater CDS tool use (lower rates of unscored tests) and (2) higher rates of appropriateness scores >7. Models were fitted using a generalised estimating equations approach, with clustering at the level of the practice site. The primary model terms included study group (intervention vs control sites), study period (pre-intervention vs post-intervention) and the interaction of these terms. Models also adjusted for patient age, gender and physician specialty. A test of the interaction between study group and period was used to evaluate the intervention effect. A significance threshold level of 0.05 was used throughout.
All analyses were performed using SAS V.9.4 (SAS Institute).
Results
Mean age of participants in the intervention group was 49 years and 51 years in the control group (n=1621). The intervention site had fewer female patients compared with the control sites (38.4% vs 51.7%).
As shown in figure 1, the percentage of unscored surveys decreased from 69.4% to 10.4% at the intervention site, compared with a decrease of 50.6% to 34.8% at the control site, with a between-group difference of −23.3% (p<0.001). As shown in figure 2, the percentage of high scoring studies increased from 26.5% to 75.0% at the intervention site, compared with an increase from 17.2% to 22.7% at the control site, with a between-group difference of 19.0% (p<0.001).
Percentage of unscored orders during the study period. 
Intervention site (n=5736); 
Control sites (n=1621). DinD, difference-in-differences; QI, quality improvement.
Percentage of high scoring orders (>3) during the study period. 
Intervention site (n=5736); 
Control sites (n=1621). DinD, difference-in-differences; QI, quality improvement.
Discussion
This simple time-saving workflow QI intervention was associated with large improvements in clinicians’ rate of using an EHR CDS back pain imaging tool. Clinicians caring for patients with back pain reduced rates of unscored tests and increased rates of tests with high appropriateness scores. The improvements were far greater at the intervention site than at control sites, providing empirical support for our hypothesis that this QI intervention that made workflow easier for clinicians successfully incentivised clinicians to engage with the CDS tool. Though this study does not provide evidence that the intervention reduced rates of utilisation (low value or otherwise), these findings suggest that time-saving can serve as a powerful tool to increase physician engagement with CDS tools and improve practice patterns.
Some might call this QI workflow intervention a ‘nudge’ because it consisted of a small change in the presentation of choices without restricting freedom of choice22; clinicians were still free to bypass the CDS tool after the QI workflow intervention policy change, but would then miss out on the time-saving benefit of avoiding a burdensome call to the radiology department for approval. Since these calls frequently include having to justify to a busy radiologist the need for the test, conflict aversion may have also driven clinicians to engage with CDS. How and when to effectively use nudges to induce physicians to change behaviour is an area of active research23 24; to our knowledge this is the first empirical study to show that a time-saving intervention can markedly change physician behaviour for test ordering.
Future work should examine whether these changes in ordering patterns correspond to actual lower rates of low-value testing and become cost-effective. While unlikely, it is possible that some clinicians were ‘gaming’ the system and inducing high scores for patients for whom testing was still inappropriate. Other limitations of this study should also be noted. The non-randomised observational nature of the study makes it impossible to eliminate the possible role of residual confounding, such as differences between operational workflows at different DHS sites. Clinicians at outpatient settings (making up much of the control group) were not required to telephone the radiology department for approval. Due to the different working schedule, one cannot assume that personnel in the emergency/urgent care centre would have the same response to this QI intervention as the personnel in an outpatient department. The institutional culture at (BLINDED)—the second largest safety net health system in the country—may be ideally suited for a time-saving workflow intervention; whether this type of intervention would work at other systems with different incentives for busy clinicians warrants further study. We did not measure sustainability. Future studies should also examine whether time-saving interventions such as this one are sustained over longer time periods.
Despite these limitations, given the well-documented, relatively disappointing impact of CDS on rates of low-value care to date,18–20 QI teams might use our study findings to inform potential workflow time-saving interventions at their own institutions to increase engagement with CDS tools and improve clinical practice patterns. CDS will never become cost-effective if clinicians do not use it. On a larger scale, we suggest considering whether the labour-intensive process of insurance company-driven utilisation review/prior authorisation processes that are so disliked by most physicians (and patients) might also be informed by these findings.25 We can envision an improved healthcare system where physicians could be guaranteed to bypass burdensome utilisation review barriers if they used a well-designed, seamlessly integrated CDS tool to ensure compliance with guidelines for appropriate ordering. Time-savings might be a powerful enough motivator to induce doctors to provide high-quality care, while also saving insurance companies utilisation review resources that could be passed back to patients. While policymakers place much emphasis on financial incentives to modify physician behaviour,26 leveraging one of medicine’s most precious resources—time—may be an under-recognised and promising strategy to change physician behaviour and improve the quality of care.
Conclusion
This QI workflow time-saving intervention was associated with a decrease in unscored imaging tests and an increase in imaging tests with high appropriateness scores at a large urban safety-net health system. Workflow time-savings may serve as a powerful lever to increase physician engagement with CDS tools and improve clinician practice patterns.
Acknowledgments
The authors thank Drs Laura Sarff and Chase Coffey for their leadership and support of this collaborative project. The authors also thank Drs Chi-Hong Tseng and Katherine Kahn for their help on an early version of the analytical model.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors BL, JM, MKP, AS, SV, EW and CS contributed to the planning/design of the work. JM, MKP, EW and CS contributed to data collection. BL, JM, AS, SV and CS contributed to data analysis and interpretation, and wrote the paper. MKP and EW critically revised the paper. CS is responsible for the overall content as a guarantor.
Funding American Board of Internal Medicine Foundation/Choosing Wisely grant from the Robert Wood Johnson Foundation, National Institutes of Health (NIH)/National Center for Advancing Translational Science (NCATS) Institute (KL2TR001882), NIH/National Institute on Aging (NIA) Midcareer Investigator Award in Patient-Oriented Research (1K24AG047899-01), NIH/NIA UCLA Resource Center for Minority Aging Research/Center for Health Improvement of Minority Elders (2P30AG081684), NIH/NCATS UCLA Clinical and Translational Science Institute (UL1TR001881).
Competing interests The authors have no competing interests to report.
Patient consent for publication Not required.
Ethics approval The study protocol was approved by the institutional review board of the University of California, Los Angeles (UCLA) and the Los Angeles County Department of Public Health.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are avaliable upon request.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.