Problem

This study was performed at NewYork-Presbyterian Brooklyn Methodist Hospital, a large urban teaching hospital in Brooklyn, New York, with 651 beds and approximately thirty thousand annual discharges. The study was performed using data for the in-patient population admitted to the Department of Medicine.

The problem stems from ordering wasteful laboratory tests that do not comply with evidence - based medicine. “Waste” can be differentiated from “Value,” which is defined as “relative worth, utility, or importance 2.”

A ZEUS (Zero Unnecessary Studies) committee was formed with representatives from the Department of Medicine, Endocrinology Division, the Clinical Laboratory, Information Technology, and the Division of Quality and Patient Safety. If a problem was identified, the committee was charged with designing and implementing the improvements to mitigate the problem. The goal of the committee was to establish if the ordering habits were in accord with the best practices for TFT ordering. The American Association of Clinical Endocrinologists (AACE) recommends that T3 and T4 only be measured if the TSH is abnormal.20 However, the problem identified was that Pre-Intervention, TSH, fT3, and fT4, were all ordered simultaneously through the EMR.

Background

The cost of our nation's healthcare has been on the rise since its inception. In 2014, 4.1% ($971.8 billion) of the Federal budget was attributed solely to hospital care, increased from 3.5% in 2013.1 The American College of Physicians, “High Value Care,” recommends for physicians to give the finest, yet most efficient, care to patients by eliminating unnecessary costs.2 The American Medical Association, “Choosing wisely,” also believes in the same ideology of “more care is not always better care.3” Thus, in accord with these ideologies along with one of the bedrocks of the Toyota Production System (TPS) Lean Methodology, eliminate muda (waste), our hospital embarked in a Zero Unnecessary Studies (ZEUS) initiative to identify and eliminate waste.

It has been shown that Thyroid Function Tests (TFTs) are ordered far more frequently than the number of patients with symptoms of thyroid dysfunction.4 TFTs are ordered more often than just a TSH level alone,5 even though the TSH level is substantial enough on admission to determine if future studies should be pursued.6 The clinical symptoms of thyroid dysfunction are very sensitive yet less specific, and thus, a majority of symptomatic patients will have normal TFTs.7 ,19 Furthermore, non-symptomatic patients with established hypothyroidism and compliance with treatment may not need fT3 and fT4 levels measured upon admission, as treatment can be adjusted as an outpatient if the TSH is appropriate during the hospital course.

The literature has illustrated that it is not effective to order TFTs for in-patients due to high false positives associated with Non-Thyroidal Illnesses.11^–13 During the stress of an illness, there are changes on the pituitary-thyroid axis, changes in the metabolism,14 and clearance of thyroid hormones.15 ,16 Small changes in T4 can create large changes in TSH. T3 is difficult to accurately interpret due to its short half-life and it can remain normal in severe cases of thyroid dysfunction.19 The most sensitive test for thyroid dysfunction is a TSH level. If the TSH level is abnormal, it is then appropriate to measure fT3 and fT4.

Specifically regarding TFTs orders, each facility would have their own ordering capabilities and regulations. As per literature searches, there has not been any attempts to solve the problem of only ordering a fT3 and/or fT4 level after the TSH is abnormal, through changes in an EMR. Thus, there was no evidence found regarding which changes would benefit or inhibit the issue of waste in an inpatient setting.

Baseline measurement

The committee determined that fT4 was the most frequently ordered TFT, after TSH, for the Department of Medicine in-patients by performing analysis of the TFT ordering practices recorded in the EMR.

The committee measured the frequency of TSH orders per total admissions, fT3 orders per TSH orders, and fT4 orders per TSH orders. fT3 and fT4 were measured per TSH orders due to the current AACE guidelines that TSH is the best initial screening test to detect thyroid dysfunction. The pre-intervention order frequencies were accessed through the EMR from May 1, 2014 until August 1, 2014, where there were 11,948 Department of Medicine admissions.

Pre intervention, TSH was ordered for 21.9% of Department of Medicine in-patients. Free T3 was ordered for 6.2% of TSH orders and Free T4 was ordered for 44.6% of TSH orders. Reference Table 1.

Table 1: Pre-Intervention Measurements of TSH, Free T3, and Free T4 (2014).

Design

The goal was to decrease the costs associated with wasteful tests which initially existed due to standardized order sets in the EMR which allowed TSH, fT3, and fT4 to be ordered simultaneously. The committee was driven to solve this problem in order to follow well studied guidelines, provide in-patients with “High Value Care,” and save costs when allowable10 ,17 through the SMART aim. Their specific aim was to improve appropriate use of TFTs by reducing waste attributed to the ordering of unnecessary fT3 and fT4 and then measuring the decrease in fT3/TSH or fT4/TSH one year post-intervention as per changes in the EMR. The committee also obtained IRB (Institutional Review Board) ethics approval.

The committee designed a change to eliminate all TFTs ordered for in-patients upon admission. The committee constructed a best practice algorithm which implemented real time clinical decision support (CDS) and hard stops (HS) into the EMR. House staff were forced to use CDS to decide when TFTs were necessary since it was no longer part of the standardized admission order set. If TSH was normal, the HS would not allow fT3 and fT4 to be ordered. If TSH was abnormal, a “reflex” fT3 and fT4 was added on. A reflex is an “if/than” decision rule built within the EMR. In this case if the TSH is abnormal, than the fT3 and fT4 are ordered from the same vial of blood, without human intervention, and is tested for a follow up laboratory value. This algorithm was incorporated into the EMR on April 10, 2015.

The committee decided this was an effective initial change because it was automated, did not require additional orders by the house staff, and was easy to execute through the EMR. The committee had members of each division affected to include Medicine, Endocrinology, the Laboratory, Information Technology, and Division of Quality and Patient Safety; but did not consult other organizations. The only problems predicted at the initial change was the ease of transition from ordering a full TFT panel to only a TSH, upon admission to the in-patient Department of Medicine.

One week prior to the intervention, emails and house staff meetings were required in order to provide ordering details, rationales for the changes, and implement long term sustainability. There was also a “Tip of the Day” built into the EMR login screen which ran for one week from the initiation of the intervention. These rules were implemented to provide physicians with clinical decision support,18 which helps improve efficacy, avoid errors, and reduce unnecessary care.10 ,17

Strategy

The original aim was to eliminate TSH, fT3, and fT4 from all being ordered simultaneously for the Department of Medicine in-patient admissions. The plan was to allow only TSH to be ordered initially, and if abnormal, the physician could then order fT3 and fT4. The committee predicted that this would save unnecessary care and unnecessary laboratory tests because an initial TSH is enough to determine if further studies should be pursued. However, within three days post intervention, physicians stated it was more time consuming to check for an abnormal TSH level and then order fT3 and fT4. Thus, the original intervention was able to save costs; however, physicians consumed more time.

The first improvement to the original intervention was to create an automatic reflex order of fT3 and fT4 in the EMR, if the TSH was abnormal. This was done in order to alleviate the physician's concern of having to place an additional order for fT3 and/or fT4 if the initial TSH was abnormal. Our Information Technology division was able to implement the improvement through the EMR so that once an abnormal TSH was noted; an automatic order of fT3 and fT4 was sent to the laboratory, without any additional physician orders. The committee predicted that this algorithm would save unnecessary care, costs, and physician's time. The physician's time was saved more so compared to the original intervention; however, it still took longer for a complete TFT panel than pre-intervention. There has not been a solution to overcome this issue, however, the house staff is in agreement that the benefit of saving costs outweighs the slight increase in extra time awaiting TFTs.

The second improvement took place three weeks post intervention. There are specific medical conditions where a full set of TFTs are useful: this includes symptomatic patients who have previously been diagnosed with hyper/hypothyroidism, patients prescribed thyroid medication, patients with atrial fibrillation, the geriatric population, psychiatric patients, and in AML.8 Thus, in these patients, a full TFT panel is needed. Moreover, only fT3 and fT4, without a TSH, are needed in cases of primary hypothyroidism.20 Thus, certain exceptions could be granted with an Endocrine approval or a justification dropdown order screen in the EMR. This improvement allowed for clinical decision support and ensured the physician acknowledged if they needed a TSH versus fT3 and/or fT4. This improvement was successful in accounting for the rare exceptions when TSH is not the best initial laboratory test.

With these changes, most attending physicians and house staff acknowledged the benefits due to the current guidelines. Any questions were addressed by reviewing the guidelines and the goals of “High Value Care.” Initially, the endocrinologists had difficulty with patients with central hypothyroidism, however, after the second improvement, those issues were also resolved.

Results

The intervention was implemented on April 10, 2015. Total orders of fT3, fT4, and total of fT3 plus fT4 orders per TSH orders were calculated for 3 months prior (May 1, 2014 - August 1, 2014) and 3 months post (May 1, 2015 - August 1, 2015) the EMR changes. (Reference Table 2).

A follow up analysis of fT3, fT4, and Total orders per TSH was conducted for one year Post-Intervention (May 1, 2016 - August 1, 2016) as well and similar results were determined (Reference Table 3).

In order to demonstrate sustainability, a variation over time graph illustrates the decrease in fT3 orders per TSH, fT4 orders per TSH, and fT3 plus fT4 orders per TSH from Pre - Intervention, Post - Intervention, and one year Post - Intervention. (Figure 1)

The fT3/TSH ratio was 6.2% pre-intervention and 2.9% post- intervention, a 53.2% reduction (P Value <.0001). Pre-Intervention, fT4 was the most ordered test upon admission. The fT4/TSH ratio was 44.6% pre-intervention and 28.9% post-intervention, a 35.2% reduction. (P Value <0.0001).

The Overall TFT/TSH pre-intervention ratio was 52.2%, which decreased by 39.1%, to 31.8% post-intervention. (P Value < 0.0001).

The numbers of fT4 orders were specifically looked at before and after the implementation in correlation with abnormal TSH values. The percentage of fT4 ordered due to abnormal TSH increased from 36.8% to 83.2%. The remaining 16.8% of fT4 orders were attributed to the exceptions.

The decrease in the number of fT3 and fT4 ordered were statistically significant. By adding clinical decision support to when TFTs can be ordered, there has been a decrease in the number of unnecessary tests ordered at our institution. There were no barriers or bias found.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Number of locations and average number of hits per location per month

Lessons and limitations

The guidelines are clear for when a fT3 and fT4 are needed in a patient,8 ,9 and thus, it was relatively easy to obtain a consensus among the house staff users and experts. The changes in ordering these labs initially caused confusion in the hospital, particularly for the patients already diagnosed with panhypopituitarism as an outpatient. Another limitation faced was that the physician had to follow the patient's laboratory values for an abnormal TSH and then the automated follow up fT3 and fT4. These challenges were overcome by adding the reflex order, justification dropdown order screen, and educating the house staff with the reasoning for these interventions as per the AACE guidelines and the importance of “High Value Care.”

If the project was repeated, it would be ideal to have a smoother transition. Perhaps there could be a video illustrating the changes in the EMR and how to tackle the extra steps needed in exceptions. The committee could also reiterate the importance of these changes and share the guidelines for why these changes will save unnecessary care.

The project did not necessarily implement PDSA cycles but more so improvement cycles as problems were identified. Both improvements (the reflex order and justification dropdown order screen) were able to be implemented within one month of the intervention, and thus, data collected post intervention was not biased by the improvements.

We believe that the rules were simple enough that they can be reproduced in any inpatient setting. Minimum resources were needed to include IT, CMIO, and lab personnel for approximately ten to fifteen hours, thus making this a very cost effective intervention.

There are direct and indirect costs associated with unnecessary fT3 and fT4 orders. The direct cost is $7.00 per test. The indirect costs include additional consults, follow ups, and the patient's burden of having a thyroid disease. The committee does not believe there were any major limitations due to sample sizes or the length of the study. In the future, due to its cost effectiveness, the committee can continue the study in other medicine settings by acknowledging recommendations set by National Associations, creating algorithms, and executing changes in the order set. Sustainability was attempted by bringing awareness to these changes, which will allow for better compliance, teach residents how to successfully treat thyroid dysfunction, and allow for clinical decision support on behalf of the entire house staff.

Conclusion

Physicians have the ability to order any laboratory tests they deem necessary for patient care. However, many admitted patients will have false positive elevations in TFTs due to Non-Thyroidal Illnesses and stress on the body. Thus, for in-patients, Free T3 and T4 are only indicated when the TSH is abnormal. Exceptions include diagnosed hyperthyroid, hypothyroid, atrial fibrillation, or panhypopituitarism. By providing clinical decision support through the EMR ordering capabilities, we have met our goal and seen a reduction in tests associated with TFTs at NYMH. Post intervention, we saw a reduction in fT3/TSH by 55.2% and fT4/TSH by 35.2%, within a 3 month span. The measures were appropriate since they were compared to the TSH, which is the initial TFT that should be ordered. Use of real time electronic clinical decision support (18) and hard stops significantly decrease unnecessary orders and costs. By making small changes and saving costs, we can target our energy and resources toward effectively treating patients.