Article Text
PDF
Abstract
There is mounting evidence that communication and hand-off failures are a root cause of two-thirds of sentinel events in hospitals. Several studies have shown that non-standardized hand-offs have yielded poor patient outcomes and adverse events. At Stony Brook University Hospital, there were numerous reported adverse events related to poor hand-off during the transfer of patient responsibility from one resident caregiver to the next. A resident-conducted root cause analysis identified lack of a standardized hand-off process and formal training on safe and efficient hand-off among caregivers as key contributing factors.
This quality improvement project used the PDSA methodology to test the use of a standardized method, the IPASS mnemonic, and compare it to our conventional hand-off method in our internal medicine residency program. The main goals of this study were to test the feasibility and effectiveness of a standardized I- PASS hand-off and to create a robust sustainability model that includes 1) integration of I-PASS handoff in the Electronic Medical Record (EMR), 2) direct observation of the hand-off process by faculty and senior residents, and 3) surveillance and reporting of hand-off compliance scores.
Compared to hand-off with a conventional method, the use of the I-PASS method resulted in significantly fewer reported adverse events (χ2=4.8, df=1, p=0.04). I-PASS was successfully integrated into our EMR system and residents were mandated to use this as the sole method of hand-off. An EMR audit conducted six months after implementation revealed poor compliance, which ultimately led to the creation of a sustainability model that improved overall compliance from 60% to 100%.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See:
Statistics from Altmetric.com
Problem
Stony Brook University Hospital is a tertiary care center on Long Island, New York. The internal medicine residency consists of approximately 96 trainees from Post-Graduate Year (PGY) one to three. On numerous occasions, residents brought to our attention issues related to poor quality hand-off. Poor hand-off was found to be the source of many morbidities and mortalities at our institution. A root cause analysis conducted to analyze the issue of poor quality hand- offs with a resident focus group found that the hand-off format employed by our residents was extremely variable and error prone. The format and content was left to the resident's discretion without a standardized approach. Review of the current hand-off methods employed revealed that only 14 of 50 (28%) hand-off sheets were without error. The most common errors on the hand-off sheets were omission of code status and medication discrepancies. To address the lack of a standardized hand-off method at our institution, we trialed the use of I-PASS - a standardized, evidence-based physician hand-off mnemonic, and successfully incorporated an electronic version into our Electronic Medical Record (EMR). Additionally, we designed an implemented informatics technology and simulation education on the I-PASS hand-off. Six months following its implementation, an audit revealed a significant drop in compliance with I-PASS. A follow up study was conducted in order to identify barriers to the sustainability of I-PASS and measure the effectiveness of targeted interventions to maintain compliance.
The aim of our project was to first test and compare the quality and rates of errors of I-PASS hand-off with our conventional variable methods of hand-offs over a three-month period. After successful demonstration of feasibility of the I-PASS method at our institution, our goal was to pilot EMR integration, measure compliance six months following its integration, create a robust sustainability model, and test the effectiveness of targeted interventions to maintain compliance.
Background
The Joint Commission has identified improving hand-offs as a national patient safety goal, citing problems with communication as a frequent cause of medical errors.1 Similarly, the Agency on Healthcare Research and Quality (AHRQ) has identified improving handoffs in care as a priority in nationwide efforts to improve patient safety.2 Furthermore, the Accreditation Council of Graduate Medical Education Clinical Learning Environment Review Program (ACGME CLER) mandates the use of a standardized hand-off process for all residents and faculty members.3
In a recent study of 321 trainees surveyed on their perceptions of adverse patient events, 70% reported that an improved sign-out would lead to a reduction in these events.4 Another study identified the importance of patient hand-offs, citing that night interns referenced the written or verbal sign-out 56% of the time in order to answer questions that arose during the night.5 When a standardized method of hand-off is instituted, the risk of missing critical data is minimized. In fact, following standardization of hand-off, night interns have been shown to be significantly less likely to report missing data.6 ,7 Hand-offs represent a critical transition point in patient care, highlighting the need for a standardized process.
Several tools for patient hand-off have been studied.8–10 A specific format known as I-PASS is an evidence-based, consensus-driven, and standardized method of hand-off. It is widely accepted and considered a gold standard for effective hand-off communication.2 In a pilot study, implementation of I-PASS was associated with a 40% reduction in medical errors and a significant decrease in verbal and written miscommunication rates.11 In a large multi-center follow up study of a resident hand-off improvement program, the medical error rate decreased by 23% and the rate of preventable adverse events decreased by 30%.12
Typically, quality improvement (QI) initiatives are a celebrated endeavor; however, maintaining sustainability remains a significant challenge, particularly in the healthcare industry.13 The need for ongoing strategies to sustain change is crucial in order to avoid the “improvement evaporation effect” of QI. A standardized system, which ensures formal structure to the transmission of pertinent information, is paramount to providing continuity of care. Additionally, to maintain hand-off effectiveness, ongoing efforts such as direct observation and active surveillance are necessary to ensure sustained compliance.
Baseline measurement
Prior to the intervention, we conducted direct observation of the hand-off process. We noted large variability in hand-off methods such as use of the written admission note, written daily progress note, physician work list, and blank sheets of paper with written patient information. In November 2014, faculty review of these variable hand-off methods and comparison to the requirements outlined in I-PASS showed that only 14 of 50 (28%) hand-off sheets were completely without error. In order to analyze the hand-off process at our institution, we conducted a root cause analysis using a resident focus group (Figure 1). The root cause identified extreme variability and lack of a standardized process of hand-off as key contributors to poor hand-off.
Based on our initial measurements, we identified hand-off variability, lack of formal structure, and poor knowledge as critical issues in need of being addressed. Following a thorough literature search, the intervention agreed upon was use of I-PASS as the standardized hand-off method.
Design
A QI team was formed comprising six internal medicine residents, a patient safety officer from the institution, quality nurses, and the associate program director of the internal medicine residency. Study participants included postgraduate year one (PGY-1) internal medicine (IM) residents. We employed the Plan-Do-Study-Act (PDSA) method of QI for this study.
The study was conduced at Stony Brook University Hospital within the department of medicine residency from December 2014 - February 2015 (PDSA 1), March 2015 - June 2015, (PDSA 2), and July 2015 - January 2016 (PDSA 3) (Figure 2).
The data from baseline measurement along with results of the root cause analysis highlighted the issue of lack of standardization and whether a standardized hand-off process would be feasible and sustainable on our busy general medicine service. The use of a standardized I-PASS hand-off was compared to the conventional method in the medicine residency program. This was chiefly because we hoped to establish proof-of-principle that I-PASS could work as an effective and feasible hand-off method in our program.
Strategy
PDSA Cycle 1
Our initial intervention involved the trial of a paper printed version of the I-PASS hand-off format in the internal medicine residency program. In order to assess effectiveness, we compared the use of the standardized I-PASS format with the conventional methods of hand-off in terms of quality of hand-off and difference in error rates.
We anticipated that the use of I-PASS would result in decreased error rates and better quality of hand-offs. We planned a quasi- experimental design to measure the effect of our intervention (i.e., implementation of I-PASS) on pre-specified outcomes, and compared it to the conventional methods of hand-off. Overall comfort level and quality of hand-offs were also measured using a survey tool administered at the end of each PGY-1 resident shift. We selected only PGY-1 night float residents as they are primarily responsible for receiving and transmitting hand-off to one another in our internal medicine residency program.
A total of 50 residents (n=50) participated in the study over an 11-week period. Residents were split into 2 arms: 22 (46%) received the conventional, non-standardized method while 25, (53.2%) received an I-PASS standardized hand-off. Residents completed post hand-off surveys based on their experience for both groups. The intervention included an educational didactic for participating PGY-1 residents (N=25). Participating residents were instructed to either use the I-PASS protocol (N=25) or the conventional method (N=25) of sign-out during hand-off to the night float PGY-1 resident. Control and intervention groups were compared simultaneously. Hand-offs were assessed through survey-based data collected from hand-off recipients. We measured pre- and post- intervention data, specifically the quality of hand-offs, as well as the combined rate of data omission errors and frequency of medical errors.
Following a two-month study period, we noted significant improvement in overall quality and decreased medical error rates with the I-PASS intervention. However, feedback from the residents indicated difficulty keeping track of the paper version of the I-PASS template. Residents also reported misplaced hand-off templates and difficulty in the copying and printing of the hand-off paper documents.
PDSA Cycle 2
The aim of the second cycle was to address the issues identified through use of the paper format of the I-PASS hand-off. We presented the data from PDSA cycle 1 to our institutional leaders, demonstrating feasibility of I-PASS in our residency program and enabling us to prove effectiveness of the I-PASS method. This facilitated transition of the paper format to an electronic version and acquisition of institutional support for EMR integration of the I-PASS method for all patients.
Following EMR integration of IPASS, an I-PASS hand-off didactic simulation was created as part of the patient safety simulation curriculum.14 All medicine residents received formal informatics training on I-PASS in addition to hands-on simulation training as hand-off transmitters and receivers.
Following educational exercises and didactic simulation on I-PASS hand-off, we hypothesized that there would be sustained compliance with the I-PASS hand-off method. Comprehensive use of the I-PASS method on at least 90% of patients was expected. However, six months after I-PASS EMR integration and educational training, we conducted I-PASS surveillance and noted a significant drop off in usage of the I-PASS hand-off method. We then focused our attention on identifying barriers to hand-off compliance and creation of a sustainability model to avoid “improvement evaporation effect”. After we noted very poor compliance rates and high I-PASS omission rates, we surveyed the residents and created another focus group to analyze the barriers. On analysis of the surveys, the majority of the residents reported two main reasons for poor compliance with I-PASS: 1) Lack of feedback on I-PASS and 2) Lack of surveillance.
PDSA Cycle 3
In order to address the barriers to compliance with I-PASS, the aim of the third cycle was to create a robust sustainability model for hand-off at our institution. Based on the results of a ten-question survey and focus group discussion, a series of hand-off interventions were implemented over the course of six months: 1) We mandated a once-weekly direct observation of hand-offs by a senior resident and supervising attending using hand-off Direct Observation Tools (DOT) created based on I-PASS components, 2) We sent out weekly reminders for daily attending oversight of I-PASS during teaching rounds, and 3) We posted weekly I-PASS compliance team scores during morning report. This cycle enabled us to analyze barriers to I-PASS use and create sustainability.
We used a pre- and post-intervention design to measure hand-off compliance on the same cohort of residents after EMR integration and education. Data were measured using the well-studied validated I- PASS faculty observation tool for written EMR.15 A scoring scale from one (<20% of the time) to five (100% of the time) was used to assess compliance with each component of I-PASS.
Results
PDSA cycle 1 served as an assessment of feasibility and effectiveness, the purpose of which was to facilitate the incorporation of an EMR I-PASS hand-off at our institution.
Chi-squared tests yielded statistically significant differences between the conventional non-standardized hand-off group and the standardized I-PASS intervention group in terms of the quality of the hand-off received at the end of their shift (χ2=15.6, df=2, p=.001), the number of adverse events that occurred during the shift (χ 2=4.8, df=1, p=.04), code status omission (χ 2=9.4, df=1, p=.003), and the number of events that should have been anticipated or discussed during hand-off but were not (χ 2=9.6, df=1, p=.003). There was no difference between the groups in regards to the number of rapid responses called for patients based on the resident's census (p=.39) and whether or not the sickest patient on each team covered was conveyed to the resident (p=.12). Although residents reported higher quality hand-off with the I-PASS hand-off intervention, we received feedback regarding the difficulty in using a paper template of I-PASS.
Once feasibility was achieved, we moved onto PDSA cycle 2 which lead to the discontinuation of the paper format and institution wide use of I-PASS, coupled with formal Informatics Technology and simulation educational training on the use of I-PASS. This process change was also combined with a well-structured guideline on mandated use of I-PASS hand-off on all patient encounters on the general medicine teams. We then used the I-PASS faculty observation tool for surveillance and assessment of hand-off compliance in a random sample of 40 patients on a weekly basis on the general medicine-teaching wards over a six-month period. A total of 12 scores were averaged to determine overall pre-intervention compliance with each component of I-PASS hand-off: I = 3.92 (50-69%), P = 4.67 (70-99%), A = 3.58 (50-69%), S = 3.16 (50-69%), S = 5 (100%). Rates of compliance generally decreased over time with the lowest levels in October 2015: I = 2.67 (20-49%), P = 4.33 (70-99%), A = 2 (20-49%), S = 2 (20-49%), S = 5 (100%). Overall compliance was 50-60% (N=320). Analysis of the ten-question survey and focus group discussion revealed lack of routine review and supervision as the two major barriers to hand-off compliance. Many residents commented “no one is watching or giving feedback on the I-PASS hand-off.”
The results of PDSA cycle 2 lead to the development of a series of hand-off interventions that addressed the aforementioned barriers. PDSA cycle 3 addressed the barriers to compliance and lead to the implementation of a sustainability model for hand-off compliance. A repeat audit performed on the medicine teaching teams three months following the intervention revealed a significant improvement in compliance with EMR I-PASS. All components of I- PASS compliance increased with the exception of synthesizing the hand-off, which remained at 100% compliance pre- and post- intervention. Overall hand-off compliance improved to 90-100% range N=45, p=0.0001) (Figure 3).
Lessons and limitations
As with many QI initiatives, one of the greatest difficulties we faced was engaging the residents and faculty members in adopting the change. A major learning point was the identification of barriers from the residents and obtaining their feedback and input regarding change. Although I-PASS is a well-known published tool that has been shown to reduce errors, it must be proven to be feasible and perceived as a valuable tool by the stakeholders at different institutions. Furthermore, we learned that EMR integration and mandated use of a system is not enough to sustain effective change. We showed that repeated targeted interventions are needed to sustain change even when using a validated and published tool such as I-PASS. One of the particular strengths of the project is the use of interventions to sustain change; sustainability testing is often not a centered focus in quality improvement or an integral part of the cycles in the PDSA model.
One of the limitations of this study is the lack of direct measurement of the time it takes for I-PASS completion. We relied on overall resident perception of the I-PASS method but did not measure time to completion of I-PASS hand-off compared to the conventional method. We assessed resident perception of errors related to patient care by using self-administered surveys. Our aim was to focus on process and structure measures related to hand-off in the residency program. We did not assess outcome measures related to direct patient care.
Another limitation of the study is the use of self-reported data for PDSA cycle 1. Although this was the easiest and most feasible way to collect data directly from the participants, the data might be subject to self-reporting bias. Ideally, we would have directly observed the hand-offs and collected the data in-person.
A challenge towards implementation of the I-PASS hand-off may be the cost associated with the EMR integration. Our EMR integration of I-PASS was associated with a Cerner software package upgrade with no additional cost. An estimated cost to build I-PASS into the EMR can range from approximately $10000 - $12000.
Conclusion
Standardized hand-off has proven to be a feasible method that improved overall quality of hand-off at our institution. This, coupled with a robust sustainability model, enabled us to not only adopt the change, but to also sustain it over time. Despite the initial success of implementing a standardized hand-off method and integrating it into the EMR, a significant decrease in compliance was observed with time. This “improvement evaporation effect” is a common drawback of quality improvement. It is clear that instituting real and lasting change requires elimination of old behaviors and encouragement of those that support new evidence-driven processes. Reinforcement and active surveillance are paramount to making the change the “new normal” and ensure compliance.
As the learning environment becomes more complex, medical errors continue to be one of the leading causes of death. A significant body of evidence has arisen over the last decade regarding medical errors related to poor communication. A standardized approach to communication during hand-off is also now well studied in the literature. Our study demonstrated that simply adopting a hand-off process from the literature might not be enough to succeed. The process entails feasibility testing in order to adopt the change and subsequent testing and planning to sustain the change.
Acknowledgments
The authors would like to thank the residents of the Stony Brook University Internal Medicine Residency Program, with special mention to members of the patient safety quality council, namely Dr. Matthew Joseph and Dr. Konstantin Peysin. The authors would also like to thank Dr. Joseph Decristafaro, Dr. Matthew Tharakan, and Dr. Gerald Kelly, leaders of the institution and informatics team.
Footnotes
Declaration of interests The authors of this study have no conflict of interest to declare.
Ethical approval The Institutional Review Board at Stony Brook University Hospital, NY deemed this study as exempt as an educational initiative. IRB acknowledgement # 640907-2.