Article Text
PDF
Abstract
Background Repetitive inpatient laboratory testing contributes to waste in healthcare. We evaluated an intervention bundle combining education and multilevel social comparison feedback to safely reduce repetitive use of inpatient routine laboratory tests.
Methods This non-randomised controlled pre-intervention post-intervention study was conducted in four adult hospitals from October 2016 to March 2018. In the medical teaching unit (MTU) of the intervention site, learners received education and aggregate social comparison feedback and attending internists received individual comparison feedback on routine laboratory test utilisation. MTUs of the remaining three sites served as control units. Number and cost of routine laboratory tests ordered per patient-day before and after the intervention was compared with the control units, adjusting for patient factors. Safety endpoints included number of critically abnormal laboratory test results, number of stat laboratory test orders, patient length of stay, transfer rate to the ICU, and 30-day readmission and mortality.
Results A total of 14 000 patients were included. Pre-intervention and post-intervention groups were similar in age, sex, Charlson Comorbidity Index and length of stay. From the pre-intervention period to the post-intervention period, significantly fewer routine laboratory tests were ordered at the intervention MTU (incidence rate ratio=0.89; 95% CI 0.79 to 1.00; p=0.048) with associated costs savings of $C68 877 (p=0.020) as compared with the control sites. The variability in the ordering pattern of internists at the intervention site also decreased post-intervention. No worsening was noted in the safety endpoints between the pre-intervention and post-intervention period at the intervention unit compared with the controls.
Conclusions Combination of education and multilevel social comparison feedback significantly and safely led to cost savings through reduced use of routine laboratory tests in hospitalised patients.
- health care waste
- low value testing
- routine laboratory testing
- social comparison
Statistics from Altmetric.com
Introduction
Unnecessary, repetitive laboratory testing is a marker of low-value care.1 A 15-year meta-analysis estimates the rate of overutilisation of laboratory tests to be 20%,2 much of which occurs in the form of repetitive testing.3 A prior study that explored reasons for ordering laboratory tests found that more than half of laboratory tests were ordered to reassure patients in the context of low clinical suspicion.4 The practice of repetitive use of routine blood tests in hospitalised patients without consideration of pre-test probability limits the predictive value of the tests, thus generating false-positive results which may subject the patient to further inappropriate diagnostic and therapeutic procedures.5–7 Hence, although laboratory services account for only around 5% of hospital budgets, they leverage over two-thirds of downstream medical decisions.8 In addition, excessive diagnostic phlebotomy is also associated with adverse patient outcomes such as hospital-acquired anaemia,9 10 which is subsequently associated with increased blood transfusions, prolonged hospitalisation and mortality.11 Therefore, mitigation of low-value diagnostic testing presents an ideal opportunity for cost-savings and also for potential improvement of patient care.
Although multiple medical societies recommend against ordering repetitive laboratory testing,12–14 recommendations alone often do not lead to behavioural changes.15 16 Successful reduction of unnecessary laboratory investigations requires addressing root causes of laboratory test overuse including insufficient knowledge of costs, absence of feedback on testing practices, fear of missing diagnoses and the utilisation culture of the practice setting.17–19 Prior studies have shown that higher numbers of routine laboratory tests are ordered in teaching hospitals where testing is ordered by rotating medical learners at different stages of training.20 21 To optimise laboratory testing, interventions that have attempted to address the above causes generally fall into three main categories: education, audit and feedback on ordering practices, and restrictive electronic medical record (EMR) mediated changes; with multifaceted interventions being associated with greater success.22–30
In our health zone, laboratory test volume increased between 6% and 8% annually from 2004 to 2014, compared with the annual increase in population of only 2.2%.31 In this study, we aimed to evaluate the impact of an intervention bundle, consisting of multimodal education and multilevel social comparison feedback on the number and costs of routine laboratory tests ordered per patient-day in hospitalised patients.
Methods
Design, setting and study population
Our health zone includes four adult tertiary care hospital sites, each with its own medical teaching unit (MTU). On all MTUs, learners (ie, medical students and resident physicians) are integral to patient care delivery and rotate in 4-week blocks, supervised by an attending internist, who rotates every 7 days. Laboratory tests can be ordered by all resident physicians and attending internists through the same EMR system (Sunrise Clinical Manager; Allscripts, Chicago, IL). Orders entered by medical students require verification by a resident or attending physician prior to being processed.
We conducted a non-randomised controlled study using a pre-intervention post-intervention experimental design that compared our intervention bundle with no intervention for reducing use and costs of target laboratory tests in the MTU of the intervention site between 13 March 2017 and 13 March 2018. The study was divided into three phases: baseline or pre-intervention period (23 October 2015 to 23 October 2016), intervention development phase (24 October 2016 to 12 March 2017) and post-intervention period (13 March 2017 to 13 March 2018). The study participants included attending physicians and learners (which include medical students and resident physicians) who provided care to patients on the intervention MTU. The remaining three MTUs at the other sites served as controls.
Development of intervention bundle
Identification of target laboratory tests
We identified target laboratory tests as those that were top contributors to expenditure through review of local data on laboratory test utilisation. These were complete blood count, electrolytes (comprising sodium, potassium, chloride and CO2), creatinine, urea, calcium, magnesium, phosphate, international normalised ratio and partial thromboplastin time (see online supplementary appendix table 1). Although not within the top 10 cost contributors, we also included creatine kinase (CK) in our study on the basis that this expensive investigation has very few indications for use and therefore may be a good target for utilisation reduction interventions.
Supplemental material
Development of local consensus on appropriate use criteria of target laboratory tests
During an in-person meeting, internists (n=10) at the intervention site drafted frequency recommendations for common indications for use of the 10 tests. These recommendations underwent minor modifications over two rounds of review through email. All internists at the intervention site approved of the final list of indications and associated frequency recommendations (see online supplementary appendix table 2). These criteria and frequency recommendations formed the basis of the educational material developed with the intervention including posters, pocket cards and case-based online learning module.
Iterative implementation of intervention bundle
The multicomponent intervention bundle was developed and implemented iteratively between 24 October 2016 and 12 March 2017 (see online supplementary appendix figure 1), with the sequential introduction of educational interventions, followed by social comparison feedback. Discussions from our stakeholders following this led to two main suggestions: (1) education and social comparison for learners should be provided online to facilitate adequate in-depth learning and consistency of delivery during all blocks; (2) the social comparison should include individual attending physicians. Incorporating these recommendations, our final intervention bundle (implemented from March 2017 to March 2018) included the following two components:
Education: In addition to the posters and pocket cards, we also developed an online case-based educational module for the learners (10 cases) followed by an 8-item knowledge assessment test. Before or during the first week of the block, learners on the intervention MTU were contacted via email from the study team to generate awareness about the initiative and the upcoming social comparison, and provide a link to the online educational module and the electronic pocket card. The posters were placed in the medical unit team meeting rooms. On the first day of the block, our project nurse champion provided an overview of the project and distributed the pocket cards to the learners. The learners were reminded with two additional follow-up emails at mid-block and end of block to complete the module. We tracked the rate of module completion during the study period.
Social comparison feedback: The intervention medical unit consisted of three learner teams, each headed by a different attending internist. Through the above reminder emails, the learners also received social comparison feedback on the average per patient-day usage of the target laboratory tests and the associated expenditure, relative to the other two teams within the intervention MTU. Attending internists on the intervention medical unit received via email a report of their individual utilisation and costs at intervals of 3 to 4 months (see online supplementary appendix figure 2). The report allowed the physician to compare his/her utilisation to the median as well as highest and lowest users among the general internists at the intervention site. Any laboratory test that was ordered on a patient under the attending physician’s care was attributed to the physician, as the attending physician was the most responsible practitioner and the ‘quarter back’ of the patient’s care. This was done to help generate a sense of responsibility within the physicians to provide oversight into laboratory test ordering by the learners working with them.
Only the attending internists and learners on the intervention medical unit received the intervention bundle. Local factors in our health zone where we are undergoing a provincial change in our EMR system precluded the inclusion of EMR changes as part of our intervention bundle.
Measures and data collection
Our primary outcome was the number and costs of 10 target laboratory tests ordered per patient-day. Our secondary outcomes included safety endpoints to look for unintended negative consequences of our intervention. These included number of ‘stat’ tests ordered, number of critically abnormal test results as defined by our laboratory services (see online supplementary appendix table 3), patient length of stay, ICU transfer rate, 30-day readmission and 30-day mortality rate.
We collected data on laboratory tests ordered from the EMR. Costs of each target laboratory test were obtained from our laboratory services provider to our provincial health system. These costs reflect the cumulative price paid by the hospital to the laboratory service provider every time an individual test is ordered on an inpatient. Expenditure on laboratory testing was calculated by multiplying the total number of each target test ordered with the cost of each individual test charged by the provider. Data on patient demographics and patient-oriented outcomes were obtained from our provincial health zone data repository system.
Data analysis
During the study period, X-bar control charts and run charts were used to track the number of target laboratory tests ordered per patient-day and the expenditure on those tests, respectively, on a weekly basis at intervention and control sites (figures 1 and 2 and online supplementary appendix figure 3). For our final analysis, random-effects Poisson regression was used to model the primary outcome of total laboratory tests ordered per patient-day accounting for the clustering of laboratory tests within patient. The data from the intervention development phase were completely excluded as the washout period. An indicator variable for the time period (pre-intervention=0, post-intervention=1) was created to assess whether the impact of the time period was different for those at the control and intervention sites. The model was structured as a traditional difference-in-difference analysis by including an interaction term for the interaction between an indicator for post-intervention time and an indicator for the intervention site. The same approach was used for the safety endpoints. For the analysis of the cost data, after log transforming the outcome of total cost, a random-effects linear regression model was fit using the same modelling approach. All models included adjustment for patient factors: patient age, patient sex and Charlson Comorbidity Index. The robust Huber/White/Sandwich estimator was used for the covariance matrix. P values less than 0.05 were regarded as statistically significant for primary and safety outcomes. To account for multiple testing, tests of individual labs were considered significant for p values less than 0.01. All statistical analyses were performed using Stata SE V.15.2 (Stata Corp, College Station, TX, USA).
X-Bar control chart of the mean number of target laboratory tests per patient-day on a weekly basis during study period at Intervention and control sites. Pre-intervention period: 23 Oct 2015 to 23 Oct 2016. Intervention development phase: 24 Oct 2016 to 12 Mar 2017 (please see online supplementary appendix figure 1 for detailed sequential introduction of intervention components). Post-intervention period: 13 Mar 2017 to 13 Mar 2018. The entire intervention bundle was simultaneously implemented at the beginning of the post-intervention period, ie, 13 Mar 2017. LCL, lower control limit; UCL, upper control limit.
Run chart of the mean expenditure on target laboratory tests per patient-day on a weekly basis during study period at intervention and control sites. Pre-intervention period: 23 Oct 2015 to 23 Oct 2016. Intervention development phase: 24 Oct 2016 to 12 Mar 2017 (please see online supplementary appendix figure 1 for detailed sequential introduction of intervention components). Post-intervention period: 13 Mar 2017 to 13 Mar 2018. The entire intervention bundle was simultaneously implemented at the beginning of the post-intervention period, ie, 13 Mar 2017.
Results
Patient characteristics
A total of 14 000 patients were included in our analysis (pre-intervention period n=6693 and post-intervention period, n=7307). Patient characteristics were similar between the two time periods and among the four sites (see table 1).
Study sample characteristics
Intervention effect
The mean percentage of learners who completed the educational module per block was 34% (SD 14%) with the mean score on the knowledge assessment test for all those that completed the module being 72% (SD 13%). We were unable to formally assess the rates of email opening for the social comparison feedback reports. However, with respect to the attending physicians, there was informal discussion of the reports at one of their quality improvement (QI) rounds. With respect to the intervention MTU team-directed reports, attending physicians were encouraged to review the results during the team rounds with their team.
The X-bar control charts and run charts used to track the number of target laboratory tests ordered per patient-day and the expenditure on those tests, respectively, on a weekly basis at intervention and control sites, are displayed in figures 1 and 2. In figure 1, we note that at the control site, the mean number of target laboratory tests was 3.89 in the pre-intervention period, which subsequently came down to 3.52 during the intervention development phase, and down to 3.03 in the post-intervention period. At the intervention site, we see a greater overall reduction from 3.61 in the pre-intervention period to 3.16 in the intervention development phase, and down to 2.53 in the post-intervention period. In figure 2, we note that at the control sites, the mean expenditure on target laboratory tests was $C24.65 in the pre-intervention period, which subsequently came down to $C22.55 during the intervention development phase, and down to $C19.00 in the post-intervention period. At the intervention site, we see a greater overall reduction from $C22.81 in the pre-intervention period to C$20.10 in the intervention development phase, and down to $C15.95 in the post-intervention period.
From the pre-intervention period to the post-intervention period using the difference-in-difference analysis, significantly fewer routine laboratory tests were ordered at the intervention MTU (incidence rate ratio=0.89; 95% CI 0.79 to 1.00; p=0.048) as compared with the control units. Hence, an 11% reduction was noted at the intervention site from the pre-intervention to post-intervention period beyond any reduction noted at the control sites during the same time period. Significantly lower cost was observed for the intervention with a total cost reduction of $C68 877 from the pre-intervention to post-intervention period. Before the intervention at the control sites, the average cost per patient for laboratory testing was $C74.45 (95% CI $C69.51 to $C79.73). At the intervention site, this was slightly higher at $C75.42 (95% CI $C70.26 to $C80.96). There was significant reduction (p<0.001) in the cost of laboratory testing per patient over the intervention period at the control sites to $C64.90 (95% CI $C60.61 to $C69.50). At the intervention site, this reduction was also significant and larger than at the control sites (p=0.020) with the average cost per patient for laboratory testing being $C61.42 (95% CI $C57.17 to $C65.98) at the intervention site after the intervention. The mean number of tests ordered per physician per patient-day at the intervention site was 4.8 (SD 5.0) in the pre-intervention period and 4.0 (SD 3.4) in the post-intervention period. Among the internists at the non-intervention site, the mean number was 5.0 (SD 4.0) in the pre-intervention and 4.8 (SD 12.2) in the post-intervention period.
During the study period, the target ‘routine’ laboratory tests in this study were ordered through 17 different test order combinations on the EMR. Electrolytes could be ordered as an entire panel or through specific ordering of each component of the panel. In our results, we included orders of either the entire panel or individual components with their associated individual costs. Of the target tests, Chem7 included urea, electrolytes and creatinine while Chem8 included calcium and phosphate. Using a p value cut-off of 0.01, significant declines were observed in calcium and urea with no significant change in the remaining tests (table 2). The incidence rate ratio in table 2 reflects the change in laboratory test orders from pre-intervention to post-intervention period at the intervention site beyond what was noted in the same period at the control site (ie, the change attributable to the intervention itself).
Laboratory test reduction at the intervention site from the pre-intervention period to the post-intervention period as compared with control sites
Safety endpoints
No worsening was noted in the safety endpoints between the pre-intervention and post-intervention period (table 3) at the intervention site compared with the controls. Again, the incidence rate ratio in table 3 reflects the change in safety end-points from pre-intervention to post-intervention period at the intervention site beyond what was noted in the same period at the control site (ie, the change that could be attributable to the intervention itself).
Safety endpoints before and after intervention at intervention site compared with control sites
Discussion
Our study using a controlled pre-intervention and post-intervention experimental design showed a 11% reduction in number of routine laboratory tests ordered per patient-day associated with cost-savings of $C68 887 without compromising patient safety. This reduction was noted at the intervention site beyond the secular trend noted at control sites and hence may be most directly attributable to the intervention itself. Using a conservative p value cut-off for individual tests, we noted the most significant reductions occurred in tests of calcium level and urea. This may suggest that some laboratory tests are more elastic in terms of utilisation management than other tests. We looked for a broad array of critically abnormal results (see online supplementary appendix table 3) and did not find any major concerns with the intervention. We also saw a decrease in variability in the ordering patterns of internists working in the intervention site.
The magnitude of total reduction is within the range of reduction that has been seen in other studies17 22–27 29 32–38 looking to intervene on utilisation of low-value laboratory testing and the estimated range of overutilisation.2 Studies that used education alone around costs (through fee-display) and indications showed non-significant39 40 reductions, small reductions in utilisation34 or poor sustainability.17 A recent randomised trial that evaluated social comparison alone also did not find a significant reduction in laboratory test utilisation.41 A systematic review of the literature suggests that multicomponent initiatives are more likely to succeed in reducing low-value care.42 Our intervention bundle combined the education around appropriate use criteria and costs with timely team and individual directed feedback on utilisation that allowed for social comparison, allowing for synergy between education and social comparison. The appropriate use criteria developed collaboratively with local stakeholders on specific target tests helped build early stakeholder engagement in the project. This intervention is imminently scalable because the criteria and educational modules are freely available. Both components of our intervention are automated and free of cost which makes the intervention sustainable. This helps mitigate against the problem of weakening gains over time often seen in QI projects. The problem of low-value care is most acute among physicians immediately after they complete training, as studies have shown that more experienced physicians practice at lower cost.43 Teaching hospitals prepare the next generation of physicians and are fertile grounds for introducing cost-conscious decision-making.
Several studies have confirmed no increase in adverse patient outcomes when laboratory testing is reduced.22 33 Variables examined include readmission rates,22 33 hyperkalemia,33 hyponatremia33 and mortality.22 We additionally included number of stat tests ordered, ICU transfer rate and again found no concerns. According to the Institute of Medicine report, variability in the delivery of healthcare is one of the greatest opportunities to improve quality and reduce costs through process improvement and standardisation.44 Similar reduction in variability has been mentioned in prior studies.25 41 Hence, interventions like this help streamline healthcare delivery and reduce variability.
Limitations
The findings of this study should be interpreted with the following limitations in mind. First, our intervention was performed at a single hospital medical teaching unit and may not be representative of all hospitals, limiting the generalisability of our findings because local factors likely played an important role. In addition, our intervention unit was not selected randomly but based on early engagement of that site with this project which further limits the generalisability of the impact of our intervention bundle. However, over-use of inpatient routine laboratory testing is common and our educational intervention materials are freely available for use. Second, the timely social comparison feedback report component of our intervention requires a robust electronic health record system which may not be available at all places. Third, this was a non-randomised observational study that precludes causal inference. However, we used a control group which allows us to separate intervention effect from time trends alone. We also adjusted for patient factors that may impact laboratory test utilisation in our analysis. Fourth, we could not determine which elements of this multicomponent initiative were most important in reducing low-value care. Fifth, unfortunately given the upcoming change in our hospital EMR system, we were unable to use EMR-based tools to curb unnecessary repetitive laboratory test utilisation. However, while studies employing EMR-enabled restrictive ordering have shown significant reductions,24 32 the magnitude of reduction is similar to that seen here. In addition, EMR-mediated changes may be difficult, expensive and sometimes impractical to implement. Sixth, our learner rate of educational module completion is low at 34%. However, these learners may still have been exposed to other components of the educational arm including the introductory email containing study overview as well as the posters and pocket cards. For the learners that did complete the module, they did well on the skill testing quiz with the average score being 72%. Seventh, although attending physicians at the intervention sites were confined to the intervention site during the study period, our residents and medical students rotate through all four hospitals which would have led to some contamination of intervention to control sites. However, the extent of contamination would have been restricted to the education retained from the educational module as the other educational and social comparison feedback components were not available to residents on control units.
Conclusions
A multipronged QI initiative was associated with substantial reductions in low-value inpatient laboratory testing among hospitalised patients on medical units as well as cost savings for the health system with no concerning safety outcomes for the patients. In the future, we will add EMR-mediated restrictive ordering to our intervention bundle to assess for the potential of further safe reductions in routine laboratory test utilisation.
Acknowledgments
We would like to acknowledge the general internists working at the intervention site for their engagement with the project and their efforts in deriving the appropriate use criteria.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @aambasta1
Presented at Early part of this project including plan was presented at the Canadian Society of Internal Medicine conference in 2017 in Toronto, Ontario, Canada.
Contributors All authors contributed materially to the inception, design, conduct, analysis, and write-up of the manuscript.
Funding This study was funded by Alberta Health Services.
Disclaimer The funding body played no role in the design of the study; collection, analysis and interpretation of the data; and the decision to approve publication of the finished manuscript.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval This study was approved by the Conjoint Health Research Ethics Board in our institution and University (REB17-1215). As this was a study of a health system QI initiative, the review board waived participant informed consent.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. All deidentified data from this study are available on request from the corresponding author (aambasta@ucalgary.ca).