During the last 15 years, several systematic studies concerning medical errors have been published. The Harvard Medical Practice Study estimated that, in New York State in 1984, 3.7% of hospitalized patients experience an adverse event during their hospital admission.¹ Based on this and other studies, in 1999 the US Institute of Medicine estimated that 44 000–98 000 deaths occur annually in US hospitals, at least partly due to preventable adverse events.² A significant number of the adverse events are related to medication use. In the Harvard Medical Practice Study, medication errors accounted for approximately 19% of all events.³

Medication errors are common in the pediatric inpatient setting. Fortunately, very few result in patient injury (adverse drug events, ADEs).^4–6 In a prospective cohort study by Kaushal et al⁵ a medication error rate of 5.7% of medication orders was reported. However, the potential for medication related patient injury has been found to be three times higher for pediatric inpatients than for adults, especially in neonates.⁴ (Potential) ADEs occurred most commonly at the stage of drug ordering. Most errors appeared to be preventable by physician computer entry with clinical decision support or full time, ward based clinical pharmacists.⁵ Several causes have been hypothesized. Pediatric patients need weight based dosing which results in more calculations than for adults. Stock solutions often have to be diluted. In addition, the consequences of dose errors can be worse in young children. One of the reasons for this is that they cannot communicate about adverse effects they may experience. Furthermore, all children, especially neonates, may have more limited internal reserves than adults with which to buffer errors: an absolute small dosage error in an adult patient may have major consequences in small children. Due to its toxic nature and possible irreversible effects, the above listed causes are of even more significance in the use of chemotherapy in children.

Until now, most risk assessment methods in health care have analysed adverse events retrospectively. In 1970 it became mandatory by law for every hospital in the Netherlands to have a Committee for (Near) Incidents in Patient Care. In every hospital this committee registers and evaluates incidents and near incidents based on spontaneous reporting. Although having a committee is mandatory, reporting itself is not. Experts estimate that in self-reporting systems probably 90% of errors go unreported.⁴ Moreover, these committees only have an advisory function and are not involved in the development of actual patient safety improvement measures.

In August 2003 a chemotherapy near accident on the pediatric oncology ward was reported to the Committee for (Near) Incidents in Patient Care of our hospital. The chemotherapy treatment schedule that the pharmacist had received had been unclear, and this nearly caused an error in the preparation of a chemotherapeutic agent. The pharmacist who reported the near incident suggested a major change in the working procedure to prevent this incident from recurring. Before implementing this major change, the management wanted to know to what extent implementation would contribute to the safety of the process as a whole. It proved impossible to answer this question because there was no description of the whole process, let alone an overview of the risks of each step. The hospital management asked the hospital’s patient safety coordinator to analyze the chemotherapy process on the pediatric oncology ward. To keep the analysis manageable, it was decided to focus on one chemotherapeutic agent, vincristine. Vincristine is commonly used in pediatric oncology—for example, in the treatment of acute lymphoblastic leukemia. Several catastrophic medication errors with vincristine in children have been reported in other hospitals in the past.^7–9 The process from prescription up to and including administration of vincristine was therefore deemed a representative process.

We sought a method to proactively evaluate processes in patient care. In the literature, prospective studies most often concern medication error rates^5,6,10 rather than process analyses.^11,12 The engineering community uses the Failure Mode and Effect Analysis (FMEA) technique to accomplish proactive risk analyses. Although widely used in industrial design and quality management for many years, FMEA has only recently been used in health care.^4,11 The National Center for Patient Safety of the US Department of Veterans Affairs adjusted FMEA to the health care setting, resulting in the Health Care Failure Mode and Effect Analysis (HFMEA).^13,14 HFMEA is a five-step process which uses a multidisciplinary team to proactively evaluate a healthcare process. The team uses process flow diagramming, a hazard scoring matrix, and a decision tree to identify and assess potential vulnerabilities. In this way, the vulnerabilities are not only judged by the likelihood of occurrence but also by the potential severity and the ease with which they might be detected and intercepted before causing harm. A worksheet is used to record the team’s assessment, proposed actions, and outcome measures.¹³

The aim of this study was to investigate whether the Veterans Affairs’ HFMEA is a valid proactive tool to evaluate a circumscribed healthcare process like prescription up to and including administration of vincristine in the pediatric oncology inpatient setting with a high detection rate of potential adverse events.

METHODS

Setting and period of study

The University Medical Center Utrecht (UMC Utrecht) is a merger of a university hospital, a children’s hospital, and a medical faculty. With 9000 employees, it is the second largest health care organization in the Netherlands. The children’s hospital is located in a separate building and has a total of 143 beds. The pediatric oncology ward had 251 admissions in 2004 with an average length of stay of 14.9 days. The age range of admissions is 0–19 years.

A computerized physician order entry system (CPOE: Mirador, iSoft) is used. This system has a medication interaction alert system, a maximum dose alarm, and is linked to the pharmacy department; it is not linked to patient data such as laboratory data. After the physician has ordered chemotherapy via this computer system, the pharmacy department and the nursing staff are notified automatically by a print out of the medication order. The children’s hospital has its own pediatric pharmacy department where all ordered medications are checked—for example, dose-weight/body surface area, interactions, and for consistency with personal chemotherapy schedule. Furthermore, everyone who consults the system is informed about medication history, current medication, and possible planned medication such as chemotherapy. The chemotherapy orders have to be authorized by a registered pediatric oncologist via a separate system before the pharmacy department delivers the medication. The nursing staff consists of registered pediatric oncology nurses.

The HFMEA was performed between January and April 2004. The recommendations were implemented in the second half of 2004. No approval from the hospital research ethics board was asked since this study was part of an ongoing safety program and no patients were involved in the study. The analysis was asked for by the hospital management, thereby giving its permission. No financial resources other than those of the hospital were used.

Composition of HFMEA team

According to the HFMEA guidelines, a multidisciplinary team was assembled.¹³ The team had nine regular members and two advisors. In order to accomplish complete multidisciplinarity, we thought that it would be profitable to ask a patient (in this case, the mother of a patient) to be part of the team. This parent representative was chosen because her child (the patient) was on chemotherapy treatment for approximately 1 year, thereby ensuring that she had enough experience with the process. Also, this parent was asked because the oncology team felt that she was able to contribute to an analytical process and give critical comments without feeling restrained by her child being under treatment with some of the team members. When she was asked, it was emphasized that her child would never have any disadvantages or benefits from her participation.

The team leader was the hospital’s patient safety coordinator (PSC), a medical doctor with no pediatric oncology experience. The PSC reports directly to the board of directors of the UMC. He had no previous experience with proactive risk assessment. He had learnt the HFMEA procedure by using the US Department of Veterans Affairs’ HFMEA toolkit, consisting of an article,¹³ a flipbook and a video. A student in public security and safety assisted him. The head of the pediatric oncology department and the head of the pediatric pharmacy department had an advisory function. They participated in the team meetings concerning their specialty or department. Three groups of subject matter experts participated in the team: pharmacy staff, nursing staff, and medical staff. The pharmacy was represented by a senior employee and a hospital pharmacist, the nursing staff by two nurses, and the medical staff was represented by a pediatric oncologist who supervises the ward and a pediatric resident working at the pediatric oncology department.

HFMEA procedure

As described above, the process of prescription up to and including administration of vincristine to pediatric oncology inpatients was analyzed using HFMEA. The procedure was performed as described in the HFMEA guidelines (box 1).¹³ The team first developed a flow diagram of the process by identifying the main processes and sub-processes. Then, for each process, potential failure modes were identified. Next, the severity and the probability of each potential failure mode were determined by consensus of the team based on the definitions published by DeRosier et al.¹³ The hazard score was determined using the hazard scoring matrix (table 1). A hazard score of 8 or higher was considered relevant enough to analyze this failure mode further (“high risk”). A decision tree was then used to determine whether the failure mode warranted further action on the basis of criticality, absence of effective control measures, and lack of detectability.¹³ Detectability measures whether the entire system will fail if this part of the process fails. For the failure modes which scored positive on these three items, all potential causes were listed. Finally, recommendations were made in which outcome measures and persons responsible for implementation and evaluation were appointed. After presenting the recommendations to the Chief Executive of the Children’s Hospital, he and his management team decided which recommendations would be implemented on the basis of required structural man hours, financial consequences, judicial consequences, and possible consequences for other departments outside the pediatric oncology department. After 6 months the medical staff of the pediatric oncology department evaluated whether the implemented recommendations were still functioning. The medical staff have a weekly meeting and patient safety is on the agenda on a regular basis. Thus, the HFMEA recommendations were put on the agenda 6 and 12 months after their implementation. Also, a satisfaction survey was performed among all the team members. This survey was anonymous and concerned general items such as usefulness of the procedure, expectations of implemented recommendations, time planning satisfaction, one question about whether or not one would recommend a parent/patient or a colleague to participate in an HFMEA, and enough space for general remarks.

RESULTS

Time schedule

An introductory session of 1 hour was held to explain the features of the HFMEA procedure. The team leader supervised the systematic HFMEA route and kept the team on target. The optimal meeting duration was 1.5 hours; meetings of less than 1 hour were less efficient. The whole process took seven meetings and a total of 140 man-hours, including writing the report. The hospital management obliged the different departments to enable the team members to join every meeting.

HFMEA procedure

The process was divided into three main phases: prescription, processing by the pharmacy department, and administration by the nursing staff (fig 1). Since there are two ways of administering vincristine (via a peripheral or central intravenous access), administration was divided into nine and 11 sub-processes, respectively.

Sixty one potential failure modes were recognized, from which 14 were classified as high risk by the hazard scoring matrix (fig 1). Ten of these 14 were already sufficiently covered by the current protocols, as determined by the decision tree. For the other four failure modes, six potential causes were found which resulted in five recommendations (table 2). Four additional recommendations were made concerning non-high risk failure modes.

HFMEA analyzes a process very precisely in a stepwise matter and can therefore encounter failure modes which do not qualify as high risk. However, the HFMEA team can still benefit from this and decide to give recommendations for these non-high risk failure modes. This was done in relation to chemotherapy treatment schedules (fig 1, sub-process 1A) where the HFMEA team made two recommendations (table 2, recommendations 1 and 2). Although these failure modes were not classified as high risk, they were deemed unacceptable by the team and therefore warranted recommendations. For the process of administration (fig 1, sub-process 3.1E and 3.2E), two recommendations were made in relation to (non-high risk) potential failure modes because they were considered unacceptable by the HFMEA team (table 2, recommendations 8 and 9).

Failure modes and recommendations (table 2)

Failure mode 1

The only person who is entitled to change a chemotherapy treatment schedule is a pediatric oncologist. However, once he or she has made this decision, it is essential that the new schedule is placed in the patient’s chart and that the old schedule is marked as invalid. Also, the new schedule has to be sent to the pharmacy department (second check point). The pediatric oncologist has to take care of both actions in person.

Failure mode 2

Since all physicians understand English sufficiently but not all are familiar with other languages, all chemotherapy treatment schedules in languages other than Dutch or English have to be translated.

Failure mode 3

It has been reported that reducing work hours prevents medical errors.^15,16 It has also been reported that the prescription error rate is related to new doctors joining a team.⁶ It was therefore recommended that a minimum number of residents to be scheduled on the pediatric oncology ward should be determined, and also not to change all residents at the same time. Because pagers cause a lot of distraction,^12,16 it was recommended to turn them off after sign out, forcing the nurses to call the resident on call. To maximize concentration while prescribing chemotherapy, a separate room was allocated for this purpose.¹²

Failure mode 4

Changes in prescriptions which are already authorized can go unnoticed by the nursing staff, so the resident has to inform the nursing staff. The most effective control measure would be to install an electronically controlled administration system which is always up to date.

Failure mode 5

No recommendation was made since the problem was already solved.

Failure mode 6

To prevent the pharmacy department from delivering the wrong chemotherapy after a change has been made in an already authorized order, they have to be informed by telephone because the order change is possibly not processed or seen by the pharmacy department employee in time.

Failure mode 7

When it is impossible to withdraw blood from a central venous line, the line could be positioned extravascularly. In this case, the line must always be evaluated by a physician before administering chemotherapy. A contrast radiograph should be considered.

Failure mode 8

Since vincristine is only rarely administered via a peripheral intravenous access, nurses and residents do not know the risk of extravasation by heart. Vincristine is therefore only administered via a peripheral intravenous access by a pediatric oncologist.

Implementation by the management

Recommendations 1, 2, 5, 7, 8, and 9 were implemented by the hospital management (table 2). For now, a minimum number of residents were scheduled in the oncology ward but the management could not guarantee this for the future (failure mode 3, recommendation 3). Recommendation 4 was tested but not implemented. Recommendation 6 affects the whole process and requires a major financial investment. Electronically controlled medication administration could not be installed on a short term basis.

Evaluation of recommendations

Recommendations 1, 2, 3, 7, 8, and 9 were maintained. Recommendations 4 and 5 required an attitude change. To change working habits takes time and effort. Concerning recommendation 6, the pharmacy department is investigating which electronically controlled medication administration system is suitable for our hospital in the future.

Evaluation of the HFMEA procedure

After the HFMEA, the team members received a satisfaction survey. The response rate was 56%.

Usefulness: all respondents considered the procedure to be a useful method for analyzing this precarious process prospectively.

Expectations: all respondents expected the process to become safer after implementation of the recommendations. Also, they expected other chemotherapy procedures to benefit from this HFMEA.

Time planning: all respondents emphasized that, along the procedure, the team became more decisive.

Parent/patient participation: the parent of the patient reported that, by gaining more insight into the hospital procedures, she was more aware of possible risks when her child received chemotherapy. This made her somewhat more anxious, although she is very positive about the attention patient safety gets. Asked what she would remember most, she answered: “The honesty with which the team members discussed failure modes in the presence of a parent”.

Colleague participation: all respondents would advice a colleague to participate in an HFMEA because it results in an increased awareness of potential safety risks.

DISCUSSION

We have shown that HFMEA is a valid tool for proactive analysis of the prescription up to and including administration of chemotherapy—in this case vincristine—in a pediatric oncology setting. One circumscribed process was analyzed, thereby allowing the HFMEA team to evaluate the process very thoroughly. It was found that the reported incident was not related to the weakest link in the medication process. In fact, many other failure modes in the process were shown to have a higher risk of leading to a medication error. This illustrates the multifactorial nature of most errors. A prospective approach of a system permits a more complete evaluation of vulnerabilities (failure modes) before adverse events occur (in contrast to retrospective analyses). Doing this HFMEA prevented the management from allocating resources to a relatively safe part of the process. With the recommendations it was possible to make the process safer as a whole. The recommendations from this HFMEA are also expected to have a favorable effect on a large number of other medication processes.

Limitations of the study

A limitation of the outcome of our analysis is that we have not measured actual failure rates. As mentioned above, medication errors are common but unfortunately most are not reported. To compare failure rates realistically before and after HFMEA or to perform a cost benefit analysis is therefore impossible. However, our study uncovered previously unacknowledged system errors.

Every team member is expected to be biased by his personal position in the hospital. However, bias in the HFMEA is minimized by the multidisciplinary composition of the team. Because the team leader was not related to the pediatric oncology department, he had independent judgement.

Inadvertent intrathecal administration of vincristine is one of the most feared fatal disasters in pediatric oncology.^7–9 However, this failure mode was not scored as high risk by HFMEA since the probability of occurrence was remote. This is due to the fact that very strict safety procedures according to international guidelines are already in place.^8,9,12 A patient never receives intrathecal chemotherapy and intravenous vincristine on the same day, and a closed system for administration of multiple chemotherapeutic intrathecal agents is used.¹⁷

HFMEA procedure

HFMEA has now been performed in several healthcare processes.^4,11,18–21 In contrast to the HFMEA performed in our hospital, some authors report very large HFMEAs analyzing large processes at once requiring, a large amount of personnel resources.^20,21 This can result in more general failure modes and thus also in more general recommendations. Others, like us, have chosen a more detailed analysis of a smaller process, resulting in more specific recommendations.^4,11,18 We believe that when the whole medication process is already very streamlined (CPOE use, standard chemotherapy treatment schedules) and provided with double safety checks (fig 1), it is more useful to concentrate on a specific part of the process. By analyzing a specific circumscribed process, the team is protected against an overload of failure modes. Moreover, this HFMEA showed that the recommendations made for vincristine also apply to many other medication processes on the ward, making the investment even more rewarding.

Our HFMEA team was unique because a parent of a patient participated. She made a valuable contribution but became personally more aware of all possible failures. Future studies should consider selecting a patient or parent for whom the chemotherapy treatment has been completed.

Strong support from the hospital management is needed for facilitating time and implementation of recommendations. HFMEA does not take into account the financial consequences of its recommendations. The hospital management has to evaluate which recommendations can be implemented regarding their financial situation.

Future directions

Currently, the board of directors is launching HFMEA in other departments of the UMC (outside the Children’s Hospital) using the experiences from our analysis as a blueprint. Every division is obliged to perform at least one HFMEA in one or more of their departments each year. Processes will be selected by questioning staff which process they think has the highest risk of leading to adverse events. In addition, our patient safety department is investigating how to optimize the error reporting rate in our hospital.