Background Laboratory overutilisation increases healthcare costs, and can lead to overdiagnosis, overtreatment and negative health outcomes. Discipline-specific guidelines do not support routine testing for Vitamin D and thyroid-stimulating hormone (TSH) in the inpatient rehabilitation setting, yet 94% of patients had Vitamin D and TSH tests on admission to inpatient rehabilitation at our institution. Our objective was to reduce Vitamin D and TSH testing by 25% on admission to inpatient Stroke, Spinal Cord Injury, Acquired Brain Injury and Amputee Rehabilitation units.
Methods A fishbone framework for root cause analysis revealed potential causes underlying overutilisation of Vitamin D and TSH testing. A series of Plan-Do-Study-Act (PDSA) cycles were introduced to target remediable factors, starting with an academic detailing intervention with key stakeholders that reviewed applicable clinical guidelines for each patient care discipline and the rationale for reducing admission testing. Simultaneously, computerised clinical decision support (CCDS) limited Vitamin D testing to specific criteria. Audit and feedback were used in a subsequent PDSA cycle. Frequency of Vitamin D and TSH testing on admission was the primary outcome measure. The number of electronic admission order caresets containing automatic Vitamin D and/or TSH orders before and after the interventions was the process measure. Rate of Vitamin D supplementation and changes in thyroid-related medication were the balancing measures.
Results After implementation, 2.9% of patients had admission Vitamin D testing (97% relative reduction) and 53% of patients had admission TSH testing (43% relative reduction). Admission order caresets with prepopulated Vitamin D and TSH orders decreased from 100% (n=6) to 0%. The interventions were successful; similar to previous literature, CCDS was more effective than education and audit and feedback interventions alone. The interventions represent >$9000 annualised savings.
- quality improvement
- evidence-based medicine
- decision support, computerised
- clinical practice guidelines
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- quality improvement
- evidence-based medicine
- decision support, computerised
- clinical practice guidelines
Laboratory overutilisation increases healthcare costs, and can lead to overdiagnosis, overtreatment and negative health outcomes.1 2 Studies have estimated that up to 95% of laboratory investigations may be unnecessary, or do not add value to patient care.1 3 Tackling laboratory overutilisation is a crucial target of healthcare quality improvement (QI) initiatives, improving the quality of care while potentially lowering costs.4 In the inpatient rehabilitation setting, discipline-specific guidelines do not support routine testing for 25-hydroxy Vitamin D and thyroid-stimulating hormone (TSH).5–9 Clinicians at Parkwood Institute (PI), a university-affiliated academic rehabilitation centre, aimed to reduce admission testing of Vitamin D and TSH by 25% as a laboratory utilisation QI initiative. At this institution, the costs of these tests are $C15.00 and $C5.20, respectively. Data availability: all data relevant to the study are included in the article or uploaded as supplementary information. Educational materials used for PDSA Cycle #1 are available on request.
PI is a university-affiliated academic rehabilitation centre with 90 inpatient rehabilitation beds distributed across the following units: Musculoskeletal, Stroke, Spinal Cord Injury (SCI), Acquired Brain Injury (ABI) and Amputee Rehabilitation. Patients within each unit are medically comanaged by a hospitalist and a consulting university-affiliated Physical Medicine and Rehabilitation specialist as part of a interdisciplinary inpatient rehabilitation team consisting of physicians, physical therapists, occupational therapists, social workers, speech-language pathologists, dietitians, pharmacists and psychologists. PI uses PowerChart (Cerner Systems) for electronic medical records (EMR) and implemented computerised order entry (CPOE) in 2014. Although an independent institution, PI shares laboratory services with London Health Sciences Centre, where the Division of Pathology and Laboratory Medicine (PaLM) runs >7 million laboratory tests annually, and it shares an EMR with 14 of 33 hospital systems in PI’s catchment area for inpatient rehabilitation (population >2 million people), making laboratory results, clinical notes and other investigations readily available to clinicians receiving persons for inpatient rehabilitation after their acute care stay at different sites. This shared system has many advantages for patient care, including ensuring appropriate follow-up for identified clinical problems, improving communication between acute care and inpatient rehabilitation teams, and potentially reducing unnecessary investigations by improving access to previous results.
Patients admitted to inpatient rehabilitation often have acute medical issues that may require diagnosis, treatment and monitoring. In addition, specific rehabilitation populations are at higher risk of certain conditions; for instance, persons with SCI, stroke and amputations are at higher risk of osteoporosis than the general population,5 6 10–13 and persons with moderate-to-severe acquired brain injury are at much higher risk of hypopituitarism.14 15 Clinical practice guidelines and evidence-based medical reviews relevant to these patient populations support routine supplementation of Vitamin D,5 6 8 9 16 and recommend clinical assessments for pituitary dysfunction in inpatient rehabilitation7; they do not, however, support laboratory screening for Vitamin D deficiency or for thyroid dysfunction in asymptomatic patients.5–9
In systematic reviews, a variety of interventions targeting laboratory overutilisation showed promise for reducing testing, particularly when combinations of interventions are used.2 17 Educational, academic detailing interventions, audit and feedback interventions and computerised clinical decision support (CCDS) interventions have been shown to be efficacious in improving laboratory utilisation.2 17 This article describes the development and impact of a series of Plan-Do-Study-Act (PDSA) cycles using these methods aimed at changing Vitamin D and TSH ordering practices in an inpatient rehabilitation setting.
Based on clinical experience in inpatient rehabilitation, the project leads (EAB and HMM) brainstormed putative areas for improved knowledge translation in this setting.
From this hypothesis generation exercise, two domains for improvement were identified for further characterisation: Vitamin D and TSH. The project leads selected the following inpatient populations for assessment: Stroke, SCI, ABI and Amputee Rehabilitation. The rationale for this was two-pronged. First, clear clinical guidelines applicable to these patient groups are available for the relevant tests (Vitamin D and TSH).5–9 Second, the project leads had adequate agency to introduce a knowledge translation initiative based on their clinical roles within these inpatient rehabilitation units. Cross-sectional quantification of the rates of Vitamin D and TSH ordering on admission to the chosen inpatient rehabilitation units validated the hypothesis. Herein, we also examined which clinicians were ordering these tests, and identified three key clinicians (two hospitalists and one physiatrist) as targets for subsequent interventions. The extent of the problem was further quantified through baseline data collection using interval time sampling over a 4-month period (September–December 2017, inclusive). A fishbone framework was used for root cause analysis, and to identify possible avenues, facilitators, and barriers to intervention. We identified five domains (patient, physician, team, institution and system) with a total of 16 contributing causes to admission Vitamin D and TSH orders (figure 1). Patient factors include clinically appropriate reasons for testing Vitamin D and/or TSH. The remaining domains encompass causes of admission Vitamin D and TSH orders that are outside the scope of clinical practice guidelines relevant to the study population, and likely lead to unnecessary or inappropriate testing.
Using the key factors identified from root cause analysis, a series of interventions were planned. For PDSA Cycle #1, we devised an academic detailing education intervention to target two physician factors, possible knowledge gap for best practices and lack of appreciation of potential harms of unnecessary testing, and one institutional factor, lack of awareness of the scope and cost of the problem. Simultaneously, CCDS was used to restrict Vitamin D orders to target two physician factors, habit and possible knowledge gap for appropriate indications for measuring Vitamin D, as part of a broader PaLM initiative across all institutions. The user interface for the CCDS is shown in figure 2. For PDSA Cycle #2, we devised an audit and feedback intervention to target two physician factors, automatic ordering in ‘favourite’ admission caresets and habitual ordering of Vitamin D and/or TSH, as well as two system factors, CPOE-enabled favourite caresets and CPOE-related ease of ordering inappropriate tests. The authors felt the intent of the interventions aligned with principles of QI and knowledge translation, namely maintaining or improving quality of patient care.4 Based on systematic reviews of QI interventions,2 17 an a priori goal of a 25% absolute reduction in admission TSH and Vitamin D testing was chosen. A third PDSA cycle was devised to solicit feedback and disseminate results.
Data collection and measures
Anonymised data were collected at regular intervals from the EMR to capture all patients admitted to the target units; data were collected using a standardised form in Excel. Variables of interest were defined prior to baseline collection and included patient factors (unit of admission, ie, stroke, SCI, ABI or amputation; patient age; history of dialysis; history of thyroid dysfunction); any admission bloodwork, admission TSH, admission Vitamin D; TSH result available on EMR from 90 days before admission; Vitamin D available on EMR from 90 days before admission; non-admission TSH testing during inpatient stay; non-admission Vitamin D testing during inpatient stay; prescription for Vitamin D supplementation during inpatient rehabilitation. Thyroid dysfunction was defined a priori as any patient with a prescription for thyroid hormone supplementation, propylthiouracil or methimazole; on medications known to perturb thyroid function (amiodarone, lithium, iodine and dopamine); or, other documented thyroid dysfunction in the EMR.
The primary outcome measures were the frequency of Vitamin D and TSH testing on admission to inpatient rehabilitation. The primary process measure was the number of electronic admission order caresets containing automatic Vitamin D and/or TSH orders. Additional process measures included documented exposure to education intervention and follow-up audit/feedback interventions, and number of patients with Vitamin D or TSH testing during the 90 days prior to inpatient rehabilitation admission. The 90-day cut-off was selected based on recommendations for testing frequency.19 20 The rate of Vitamin D supplementation and new or altered thyroid medications were the primary balancing measures. Secondary balancing measures included frequency of admission bloodwork to ensure changes in Vitamin D and TSH ordering patterns were not attributable to unintended decreases in admission bloodwork, and non-admission testing for Vitamin D and TSH, to ensure providers were not delaying screening to improve audit results without changing practices and/or that our intervention had not inadvertently eliminated testing altogether.
In the initial audit of admissions over a 4-month period (September–December 2017, inclusive), 174 patients were admitted. Nearly half of patients were admitted to Stroke Rehabilitation (47.8%), and the average age of admitted patients was 64.0±18.7 years (range 18–98). Patients’ medical history included dialysis (0.6%) and thyroid dysfunction (9.6%) for a minority of patients. 95.6% of patients had laboratory testing on admission. 94.4% of all admitted patients had Vitamin D tested and 93.7% had TSH tested. Vitamin D and TSH test results were available in the EMR from the preceding 90 days for 10.8% and 47.3% of patients, respectively. Although these test results are available in the EMR, the clinician would be required to check the patient’s 3-month history for this test result. Subsequent thyroid testing was performed in 7.7% of patients later during their rehabilitation inpatient stay, separate from their admission blood work. 90.5% of patients were prescribed a Vitamin D supplement. 100% of key clinician “favourite” admission caresets (n=6) for the units being studied included automatic orders for Vitamin D and TSH. One key clinician did not use any favourite admission caresets.
PDSA Cycle #1: education (academic detailing) and CCDS
We designed an in-person educational intervention targeting clinical team leaders (hospitalists and physiatrists) that reviewed applicable clinical guidelines for Vitamin D and TSH testing for each patient care discipline, targeted to this audience. The aim of the QI project and rationale for reducing admission testing was also presented. We hypothesised that the academic detailing intervention would reduce admission ordering by improving clinician awareness of the inappropriateness of population-based screening of Vitamin D and TSH and reaffirming the applicability of the Choosing Wisely Canada (CWC) national quality assurance guidelines to the majority of patients admitted on these units, and that the CCDS order restriction would reduce ordering by making appropriate clinical scenarios explicit with each order. The in-person academic detailing was held on 8 January 2018 and run by the project leads (EAB and HMM) during quarterly Quality Assurance rounds. All Physical Medicine and Rehabilitation department members were invited to attend. Attendance by hospitalists and Physical Medicine and Rehabilitation specialists was documented; two hospitalists and three physiatrists were absent from the initial intervention session. To ensure that the key clinicians received exposure to the academic detailing intervention, one-on-one presentations were arranged with the absent hospitalists and took place within 10 days of the initial intervention. In these meetings, clinicians were invited to provide feedback regarding their concerns and rationale for current practice patterns with the goal of identifying additional balancing measures to monitor unintended consequences. Simultaneously, CCDS limited Vitamin D testing to specific criteria based on CWC recommendations:8 chronic renal failure, metabolic bone disorder, malabsorption syndrome, perturbed calcium homeostasis, prescription for medication known to alter Vitamin D level (phenobarbital, carbamazepine, phenytoin and valproate) or known to increase risk of metabolic bone disorder (steroids and antiretrovirals). This CCDS intervention introduced a CPOE forcing function requiring clinicians to choose the indication for testing from this list prior to placing the order (figure 2).
PDSA Cycle #2: audit and feedback
In-person follow-up using an audit and feedback approach took place after the first postintervention data collection (approximately 1 month after PDSA Cycle #1). The audit provided information on the rate of admission Vitamin D and TSH orders after the intervention, as well as information on balancing measures and process measures, such as whether or not clinicians continued to have automated Vitamin D or TSH orders selected in their admission caresets. Root cause analysis identified that admission caresets with automated Vitamin D and TSH testing were a major contributor to unnecessary, population-based testing. We hypothesised that these meetings would reduce unnecessary testing by targeting clinician habit by inducing them to alter the caresets. The in-person follow-up took place with previously identified key clinicians. In these meetings, clinicians were again invited to provide feedback regarding their concerns and rationale for current practice patterns with the goal of identifying additional balancing measures for the project team to monitor unintended consequences.
PDSA Cycle #3: results dissemination and feedback request
We disseminated the results of the QI interventions using a brief, narrative description of the outcome measures and balancing measures in an email summarising 5 months of postintervention data including a run chart. In this email, we also solicited feedback on the process and associated outcomes of this QI project.
We performed a prepost analysis comparing the 4-month preintervention baseline to a 6-month postintervention period (ending 5 months after PDSA Cycle #2). Outcome, balance and process measures are described in table 1 and summarised in a run chart (figure 3). The baseline and postintervention cohorts were similar in their demographic characteristics (table 2). In the baseline group, the average patient was 64.0 years of age (range 18–98) compared with 64.7 (range 17–98) in the postintervention group. Approximately half of all patients were admitted to Stroke Rehabilitation (47.8% in the baseline group vs 52.4% in the postintervention group). The prevalence of thyroid dysfunction as defined by the study authors was 9.6% and 12% for the preintervention and postintervention groups, respectively, similar to previously described rates in the general population.21 After the intervention, 93% of patients received a Vitamin D supplement, which was similar to the preintervention period (90.5%). This aligns with recommendations for Vitamin D supplementation in our Canadian population at risk for Vitamin D deficiency.8 After the intervention, 1.7% of patients had a new, altered or discontinued thyroid medication prescription, compared with 1.3% pre-intervention.
The frequency of Vitamin D testing on admission decreased from 94.4% in the baseline period to 2.9% after the interventions, which reflects a 91.5% absolute decrease in the rate of admission testing for Vitamin D (relative reduction 97%). The frequency of TSH testing decreased from 93.7% in the baseline period to 53.1% after the interventions, which reflects a 40.6% absolute decrease in the rate of admission testing for TSH (relative reduction 43.3%). This decrease represents a cost savings of approximately $C17.06 per admission or $C9172 annualised. The rates of admission bloodwork were similar before and after the interventions (preintervention 95.6% vs postintervention 97.8%), suggesting that the QI initiatives did not inadvertently reduce admission laboratory testing outside of the planned test targets (Vitamin D and TSH). TSH testing was performed at some point during a patient’s inpatient rehabilitation stay for an average 6.8% of patients in the 5 months after the intervention, similar to the rate in the 4 months prior (7.7%). Rates of TSH testing within 90 days prior to admission to inpatient rehabilitation remained relatively constant (47% prior to the intervention vs 50.9% after), suggesting that there was no inadvertent exposure of acute care clinicians to our rehabilitation-directed QI academic detailing intervention. The number of patients with redundant TSH testing on admission to inpatient rehabilitation declined from 44.1% prior to the intervention to 19.1% after the intervention.
After PDSA Cycle #1, there was a substantial decrease in the rate of Vitamin D testing, from 94.4% in the preintervention period to 14.5% after 1 month, and a modest decrease in the rate of TSH testing (from 93.7% to 72.2%). The discrepancy between the testing rates is most likely attributable to the CCDS forcing function requiring an appropriate clinical indication for Vitamin D, which was not required for TSH. In this interval, the number of key clinician favourite admission caresets including automatic orders for Vitamin D declined to zero and for TSH declined by a third (from n=6 to n=4).
After PDSA Cycle #2, in which key clinicians received audit feedback, further declines in Vitamin D and TSH orders were observed: Vitamin D testing declined to an average of 0.6% and TSH declined to an average of 52.1% over the subsequent 4 months. During this interval, no key clinician favourite caresets included automatic Vitamin D or TSH orders. The success of this intervention may be attributable to recommending the clinicians remove remaining preselected ordering of TSH in their favourite caresets, thus targeting physician habit.
Although we solicited feedback at each stage of our QI initiative, in PDSA Cycle #3 key clinicians targeted by PDSA Cycles #1 and #2 were afforded an additional opportunity to provide feedback to the project leads. The key clinicians provided predominantly informal, in-person feedback. Overall, feedback was generally positive, with clinicians reporting they felt they continued to make appropriate care decisions despite changing their practice habits. We did not receive feedback on the preference for type of intervention (ie, CCDS forcing function or not). From this feedback, the key clinicians voiced concerns about the possibility that the CCDS deterred them from ordering Vitamin D in clinically indicated scenarios. Because our data collection parameters were set out a priori, data collected for the present study did not provide information to adequately answer this question in retrospect. To address this question, we performed cross-sectional audits of 2 months (n=90 patients) selected at random during the postintervention period using a random number generator. In this sample, four patients (2%) had an indication that may have warranted vitamin D testing, and zero patients were tested. All of these patients received Vitamin D supplementation during their inpatient stay.
Lessons and limitations
Overall, our interventions were successful, exceeded our laboratory utilisation reduction goal and resulted in significant cost savings. We learned a number of valuable lessons from our QI process. The use of a CCDS forcing function in conjunction with academic detailing and audit and feedback for Vitamin D was more effective than academic detailing and audit and feedback alone. Unlike the Vitamin D CCDS tool, the TSH interventions did not infringe on a physician’s autonomy in clinical decision-making (ie, there were no extrinsic barriers to ordering the test if he or she felt it was clinically appropriate to do so). This may explain the high rate of testing in the context of nearly half of patients already having a recent TSH value available. In PDSA Cycle #2, we specifically targeted automatic TSH orders saved in the key clinicians’ favourite admission caresets, which was effective in further reducing TSH testing. As previous studies have shown, electronic embedding of orders is a major contributor to repeated and/or unnecessary testing that should not be overlooked in laboratory overutilisation initiatives.22 It also emphasises the need to revisit electronic order caresets regularly to ensure they remain current with best practices as they evolve.
Compared with existing studies using sequential academic detailing and audit and feedback approaches,23–25 this TSH intervention produced a more robust relative reduction in the frequency of testing. Although the precise reasons for this are beyond the scope of the present study, there are several potential reasons. First, our academic detailing intervention was clinician-focused and addressed predominantly physician-related root causes. It provided clinical decision-making support relevant to each physician’s patient population, and therefore appealed to intrinsic motivators for providing optimal patient care within the appropriate clinical sphere. It also equipped clinicians with up-to-date guidelines-based information with which they could educate other members of the interdisciplinary team, such as dietitians and pharmacists. Second, the in-person academic detailing and audit and feedback allowed us to monitor actual exposure to the QI interventions, follow-up with missed key clinicians to ensure exposure and provided an opportunity for two-way dialogue to understand practice behaviours and concerns of clinicians. This collaborative atmosphere may have increased buy-in. Future directions will include consideration of implementation of CCDS to reduce redundant TSH orders.
Drawbacks and limitations to our methods include manual data collection, single-centre design, one-on-one in-person engagement and subspecialised academic detailing. To be sure we introduced this QI initiative responsibly, we used balancing and process measures that required manual data collection. This was feasible for a small team given that the number of monthly admissions to the Stroke, SCI, ABI and Amputation Rehabilitation units at this institution totaled 42.1 on average (n=421 over 10 months). Such a labour-intensive strategy requiring chart review for all newly admitted patients may not be feasible for broader implementation within our own institution, and conceivably not for implementation on a multi-institution or multisite scale. As such, using automatically measured balancing and process measures may be preferable. Moreover, the use of one-on-one in-person engagement for key clinicians, while likely more effective than other, less personal means of academic detailing or audit and feedback, was also a labour-intensive process. This was achievable by a small but passionate team in part because there were three key clinicians driving laboratory utilisation. In settings with more barriers, such as more key clinicians or more sites/institutions, such an approach might not be feasible. Moreover, our use of tailored, subspecialised academic detailing was a likely contributor to our success, but our educational materials and evidence base may not be useful for others looking to adopt this strategy in a related or unrelated field. Moreover, the project leads had a high degree of credibility within the clinical areas targeted in this QI initiative and with the key clinicians therein. This also likely increased buy-in. Having project leads with credible clinical involvement on the targeted units who were able to engage key clinicians in person may not be reproducible across all contexts and may therefore limit the generalisability of this intervention.
Laboratory test overutilisation is a multipronged problem, contributing to high healthcare costs, non-evidence-based and non-patient-centred care delivery, and potentially to overdiagnosis, overtreatment and/or adverse patient outcomes. Factors contributing to laboratory overutilisation may be physician-driven, team-driven, institution-driven or system-driven. Using a combination of educational intervention, CCDS, and audit and feedback methodologies, this report outlines successful targeting of inappropriate Vitamin D and TSH orders in an inpatient rehabilitation setting. Our aim was to decrease admission orders for Vitamin D and TSH by 25%. We exceeded this goal, producing a 91% absolute reduction in Vitamin D tests and a 41% absolute reduction in TSH tests, yielding a cost savings of approximately $C17.06 per admitted patient, or $C9172 annualised.
The authors wish to thank the Department of Physical Medicine and Rehabilitation, St. Joseph’s Health Care London, and London Health Sciences Centre for making quality improvement a departmental and institutional priority.
Contributors All authors fulfill the authorship recommendations outlined by the ICMJE. This project was undertaken as part of a quality improvement curriculum through the Schulich School of Medicine & Dentistry and London Health Sciences Centre. EAB and HMM collaboratively identified areas of potential knowledge translation and quality improvement applicable to inpatient rehabilitation at Parkwood Institute, St. Joseph’s Healthcare London, performed root cause analysis and designed Plan-Do-Study-Act (PDSA) cycles targeting underlying causes of laboratory overutilisation to be targeted by this project. EAB was responsible for data collection and analysis, and preparation of this manuscript. All authors contributed to PDSA cycle design, analysis and interpretation of the data. AG and ICY provided feedback and expertise in quality improvement and laboratory utilisation, respectively, throughout the project. EAB drafted the manuscript, to which all authors contributed revisions. All authors reviewed, provided feedback and approved of the content of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Institutional review board approval was not sought as this study met criteria for exemption based on the Tri-Council Policy Statement, Article 1.1 stating that quality assurance studies should be exempt from review.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.