Problem

Virginia Mason Hospital and Medical Center, in the Pacific Northwest, is a group practice with a 336 bed hospital with 16,500 annual admissions. In 2005, the organization implemented hospital computerized order entry along with the use of electronic order sets. Order sets are a series of orders focused around a diagnosis, condition, or treatment that are built sequentially into a single electronic format to support efficient, evidence based, standardized care. We started our implementation with 200 order sets and there are currently 550 active hospital order sets. However, we lacked a systematic method of reviewing and updating order sets, meaning that single requests for changes were submitted by providers, sections, or departments. These requests were put through various approval processes and placed in a queue to be worked on by analysts. At Virginia Mason, in addition to the Information Technology (IT) analysts that are trained to work with our Electronic Health Record (EHR) vendor tools, the laboratory and pharmacy department also staffed analysts to build the laboratory and medication orders respectively. This staffing model required coordination of resources from three departments, with the lack of a reliable method of prioritization, and contributed to great variability in the time required to process these requests.

Background

In the last 10 years, many healthcare organizations have adopted the use of electronic health records. This transition can be largely attributed to the HITECH Act and the United States government's creation of Meaningful Use incentives for use of the electronic health record (EHR).1 ,2 The United States federal government website to guide EHR implementation (HealthIT.gov) contains six steps on “How to Implement EHRs”. Step 6 discusses the need to “continuously evaluate implementations to assure that they are producing workflows that meet intended goals while leveraging functionality”.3 Publications have addressed the best practices for EHR implementation and the financial and non-financial costs involved, but there are few that discuss the challenges and the resources required to adequately maintain order sets in an EHR after implementation.4–8 Herein, we describe our challenges and the application of Lean production methodology to improve efficiency of management and quality of electronic order sets.

The use of computerized provider order entry (CPOE) within the EHR has been shown to improve safety in ordering.8 ,9 The use of order sets can reinforce best practice, help eliminate outdated practice, provide clinical guidance at the right point of decision making, and makes the right thing to do, the easy thing to do.6 Order sets have been shown to improve patient care by facilitating the ordering of evidence based treatment options and by reducing costs through reduction on length of stay10 and decreased mortality with the use of an order set to treat acute myocardial infarction.11

However, order sets require maintenance. Medication and other order types within the order set need to be updated regularly based on changes in evidence, changes in supply, or changes in processes.5 ,6 ,12 The implementation of an electronic health record creates a new type of supply chain within these facilities to support computer workflows that were previously performed on paper. Paper order sets could often be modified by anyone who had word processing document proficiency. The skillset required for EHR build and modification is very specialized; order set requests can only be completed by analysts who are trained to use the tools that can make these changes to the EHR.

The Institute for Safe Medication Practices (ISMP) has published “Guidelines for Standard Order Sets” that includes best practice recommendations for design and maintenance of order sets, but there is not a “recipe book” for how to create a maintenance process. ISMP recommends that order sets be reviewed and approved every two years, with some order sets requiring more frequent review.13 There is potential for much variation in the way a maintenance process would need to be designed for an organization based on its size, its culture of collaboration and standardization, and amount of resources that can be dedicated to maintenance.

Baseline measurement

We identified that there were over 200 order sets that had not been reviewed for over nine years. In addition, the process did not support review of evidence to create or modify content and did not encourage the collaboration of subject matter experts with other key stakeholders. The process also lacked transparency so requestors often did not know when to anticipate getting their request processed.

Processing time for order set build duration prior to the review process was a mean of 79.6 days (N=78, SD=68.0), with high variability (Figure 1) demonstrated with statistical process control charts. Because of this, many requestors were discouraged from submitting lower priority order set requests or evaluating all of the order sets in their sections or departments at one time.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Statistical Process Control Chart showing variability in processing days over time, before and after the intervention

Design

Root cause analysis of the delays and variability in the process revealed three main causes:

1. The priority of the request was not visible to the analysts.

2. The work needed to be coordinated by three teams and so required the availability of three individuals at the same time. Order sets often sat in the queue for days/weeks waiting for the next team to perform the work.

3. The request often required additional clarification from the requestor before build could be completed. This clarification process could sometimes take weeks or months.

Additional challenges were the large number of order sets that had not been reviewed, and redundant and obsolete order sets.

A team was put together comprised of a project manager, who worked with analysts from the IT, Pharmacy, and Lab departments, and nurse informatics specialists. To address order sets, we sought to leverage our institutional focus on the Lean methodology to improve quality, reduce waste, and ultimately, increase value.14 One of the principles used to remove waste is the use of root cause analysis to discover source of defects and delays. Another key principle is the concept of putting a flow into the production process where workers can efficiently hand off work to the next worker.15 Flow reduces time wasted between workers by optimizing the hand-off.

We determined that a systematic, cyclical order set review process would address our challenges with un-reviewed and obsolete order sets, and would also provide the consistency and transparency needed for the users of the order sets. Use of Lean tools to design such a system could improve efficiency, and decrease the time for order set review and production. We sought to improve the build duration time of the order set review process (Fig. 2) by measuring pre and post processing times. We also hoped to remove variability and lack of transparency around the process.

Strategy

In March of 2014, we implemented a systematic, cyclical order set review process to review 550 order sets for evidence and other changes (Figure 2). A production schedule was created based on approximately 15-25 order sets per month grouped by section, department, and/or topic. These groupings were prioritized in the schedule based on their alignment with organizational goals, pay for performance quality measures, and/or relationship to patient safety. Single order set requests continued to be submitted by users, but these were grouped appropriately into the production schedule for processing, except for high priority requests related to urgent regulatory changes or high risk patient safety incidents. High priority requests would be processed as a single request as soon as resources were available. This eliminated most single order set requests and allowed the build team to focus resources more efficiently on the production schedule needs.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Order set review process

We targeted our process toward specific quality and safety improvements (Table 1), and defined specific roles for necessary contributors, including primary content experts, secondary content owners, and process owners were identified prior to the review. Primary content experts were defined as the group, section, or department that were the experts for the main topic of the order set and were deemed the “owners” of the order set(s). Responsibilities included evidence review, reaching consensus with other content experts or process owners, design, and communication. Secondary content experts were those that provided oversight for specific orders within the order set such as laboratory or radiology orders, or clinical committees sections that provided oversight for specific orders such as the Antimicrobial Stewardship Committee for use of antibiotics. Process owners were those individuals such as provider groups that would also use the order set, or nurses, respiratory therapists, etc. that might be recipients of the orders.

The process was designed to have an eight week cycle of review by all content experts and process owners (Figure 2). Order sets were kept in shared network drive in a modifiable Word document with tracked changes and the path to the drive was sent out to all primary content owners, secondary content owners, and process owners. Initially, these were done through email but that presented many issues with version control. The content owners reviewed the evidence and updated the documents, reached consensus, and confirmed that changes were final. The entire group was notified of the final drafts so all could review and make final responses to the changes. A review meeting was scheduled approximately eight weeks from the start of the cycle so that the primary content owners could meet with the analysts responsible for the build to assure that all specifications were clarified and technically feasible. The build duration cycle was designed to take 4-6 weeks.

In addition to content changes within the order set, we created a style guide with guidelines for creating order descriptions, consistency of text and build within order sets, and order set naming conventions. We applied the principles from these guidelines to each order set review. We also reduced the number of synonyms that were used for each order set to reduce confusion and to ease future maintenance. For example, our Acute Coronary Syndrome order set(s) previously had a total of 19 synonyms and providers were not aware that these 19 order sets contained essentially the same content.

Results

Data from 2013 was used to assess order set processing times prior to implementation of the order set review process. The time to build a new order set (build duration) was defined as the time from submission of request to build completion. After the intervention, because order sets were identified proactively for review, the build duration was defined as the time from review meeting to build completion. The change in mean processing time prior to and after the intervention was analyzed using STAT MP v.12 using a T-Test for two independent samples. Data was displayed graphically using statistical process control charts.

We also assessed the number of order sets that were obsolete or redundant through the intervention. These were inactivated, or renamed.

The dates of analysis post-intervention was from March, 2014 to August, 2014. Prior to starting this process, 89 order sets which were infrequently used were identified and inactivated. For the time period studied, a total of 142 order sets went through the new review process, of which 38 were newly created order sets, 21 were reviewed but did not require revision, and 20 were inactivated (Table 2).

Processing time for order set build duration prior to the review process was a mean of 79.6 days (N=78, SD=68.0), and decreased to 43.2 days (N=101, SD= 22.9), an absolute decrease of 36.4 days (p<.001, CI=22.1, 50.7) following the intervention (Figure 1).

Lessons and limitations

The production schedule allowed for the planning of this work and the allocation of resources at the appropriate time. It created an efficient production line for one product, a set of order sets for one section or department instead of the processing of fragmented, sporadic, individual requests. The 4-6 week build cycle allowed each department of IT analysts to plan for the necessary resources required thus level loading work demand with demands from “non-order set” requests. The building of the order sets by three teams “together” allowed for an effective hand-off from one team to another and eliminated the wasted wait time that was present in previous processes. The review meeting scheduled prior to the build assured that change specifications requested for the order sets were understandable and reduced the time wasted in “clarifying specifications” after the request was submitted. Previously, a single clarification could result in a chain of 8-10 emails that occurred over the course of 2-4 weeks. All of these changes in production contributed to reduced amount of time utilized for processing order set requests.

The 8 week review cycle allowed time for the review of changes to evidence as well as collaboration with subject matter experts to share knowledge and gain consensus, and supported a number of changes that improved the quality of orders within order sets. Since content owners were aware of these processing times, it allowed them to plan for communication of order set changes to colleagues and other stakeholder prior to deployment.

We discovered that by creating a process that looked at order sets from a section, department, or topic all at one time allowed content owners to make substantial changes to all order sets at once, such as restructuring the way orders were presented or aligning changes across many order sets. These changes sometimes spanned across departments as adjustments were also made to Emergency Department or Critical Care order sets as necessary to reflect consistent practice in the assessment or treatment of acute coronary syndromes, stroke, or patients with diabetes. This resulted in improved consistency, quality, and efficiency in the use of these orders throughout the organization.

The collaboration between primary content owners and other key stakeholders increased the quality of the content of the orders, improved design, and improved acceptance of the order set changes. Osheroff et al. in “Improving Outcomes with Clinical Decision Support” discusses the need for increased stakeholder engagement as a key to a successful clinical decision support program.[16] The collaborative process also increased provider satisfaction as many felt that their feedback was valued and often incorporated into the content or design.

One of the challenges that we encountered during this review process was the lack of resources given to providers to review order sets. Providers were not always given administrative time to review order sets, so they often spent off work hours reviewing literature or working on changes. Many commented that the process for stakeholder engagement and getting to consensus was much more time consuming than anticipated.

The review meeting helped substantially to reduce downstream defects, but there were situations where changes were requested after that review meeting that required additional clarifications and created delays. Improvements could also be made to the testing and implementation process. We received frequent feedback that it is difficult for providers to adequately assess order sets until they are being used in the real workflow. We had only sufficient resources to offer to make minor adjustments to order sets after implementation but lacked resources to respond to larger requests unless they posed a risk to patient safety.

As more organizations have adopted EHRs and the use of order sets and other forms of clinical decision support, it is imperative that these organizations understand the maintenance needed and resources required to assure that these interventions are updated regularly to reflect evidence updates and changes to workflow to meet intended goals.5 Engaging key stakeholders, including the institutional Antimicrobial Stewardship Committee and other clinical groups, was time consuming but necessary to align changes within the governance and goals of the organization.

This approach to order set management may not be completely generalizable to large multi-facility organizations, organizations that have a different model for processing IT requests, or do not employ Lean principles. Our model or aspects of it may be more applicable to smaller organizations, with limited resources to outsource or purchase order set maintenance services, in order to design efficient methods for order set maintenance.

Conclusion

We found that applying lean production principles to an order set review process resulted in significant improvement in processing times and increased quality of orders. We believe that our process can be sustained indefinitely because it allows for planning based on the projected available IT resources. We reviewed 12 anticoagulation-related order sets the month following this project with a build duration of 38 days, which also resulted in the creation of three new order sets (total 15). For the subsequent four months, the process was put on hold while IT resources were diverted to other implementations. The process was then started again the with the review of 35 Emergency Department order sets with a build duration of 23 days.

Since there is a lack of literature that imparts specific advice or guidance on how healthcare organizations can manage and maintain the build within their electronic health records, we hope that the results of this project might allow others to glean ideas or to apply the lean methodology principles that were presented.