You are here

  • Home
  • Archive
  • Volume 22, Issue 9
  • The Housestaff Incentive Program: improving the timeliness and quality of discharge summaries by engaging residents in quality improvement

Article Text

Article menu
Original research
The Housestaff Incentive Program: improving the timeliness and quality of discharge summaries by engaging residents in quality improvement
  1. Kara Bischoff1,
  2. Aparna Goel1,
  3. Harry Hollander2,
  4. Sumant R Ranji1,
  5. Michelle Mourad1
  1. 1Department of Medicine, Division of Hospital Medicine, University of California San Francisco, San Francisco, California, USA
  2. 2Department of Medicine, Division of Infectious Disease, University of California San Francisco, San Francisco, California, USA
  1. Correspondence to Dr Michelle Mourad, Department of Medicine, Division of Hospital Medicine, University of California San Francisco, 505 Parnassus Ave, M1287, Box 0131, San Francisco, CA 94143, USA; Michelle.Mourad{at}ucsf.edu This article was a poster presentation at the 2011 AAMC Integrating Quality Meeting

Abstract

Background Quality improvement has become increasingly important in the practice of medicine; however, engaging residents in meaningful projects within the demanding training environment remains challenging.

Methods We conducted a year-long quality improvement project involving internal medicine residents at an academic medical centre. Resident champions designed and implemented a discharge summary improvement bundle, which employed an educational curriculum, an electronic discharge summary template, regular data feedback and a financial incentive. The timeliness and quality of discharge summaries were measured before and after the intervention. Residents and faculty were surveyed about their perceptions of the project; primary care providers were surveyed about their satisfaction with hospital provider communication.

Results With implementation of the bundle, the average time from patient discharge to completion of the discharge summary fell from 3.5 to 0.61 days (p<0.001). The percentage of summaries completed on the day of discharge rose from 38% to 83% (p<0.001) and this improvement was sustained for 6 months following the end of the project. The percentage of summaries that included all recommended elements increased from 5% to 88% (p<0.001). Primary care providers reported a lower likelihood of discharge summaries being unavailable at the time of outpatient follow-up (38% to 4%, p<0.001). Residents reported that the systems changes, more than the financial incentive, accounted for their behaviour change.

Conclusions Our discharge summary improvement project provides an instructive example of how residents can lead clinically meaningful quality improvement projects.

  • Quality improvement
  • Information technology
  • Financial incentives
  • Graduate medical education
  • Transitions in care

https://doi.org/10.1136/bmjqs-2012-001671

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Traditionally, residency training has focused on delivering direct clinical care, and residents have not been integrated into quality improvement (QI) efforts. Recognising the front-line role of residents in healthcare delivery and the growing importance of QI in the practice of medicine, the Accreditation Council for Graduate Medical Education established ‘practice-based learning and improvement’ and ‘systems-based practice’ as core competencies for all accredited residency programmes in 2006.1 Many residency programmes have since incorporated QI and patient safety didactics within their curricula, but it remains difficult for residents to lead and engage in meaningful QI projects given the clinical demands of residency.1 ,2 There are only a handful of published examples of successful resident-led QI projects, and there are few models of how to successfully involve residents in influential and sustainable QI projects.3–5

    Despite the abundance of evidence-based guidelines that are available to clinicians, implementation of best practices remains challenging and imperfect.6–9 The completion of hospital discharge summaries is an area where clinical practice lags behind current evidence.10 Timely completion of high-quality discharge summaries can both improve the quality of care and demonstrate achievement of training milestones such as to ‘effectively communicate with other caregivers in order to maintain appropriate continuity during transitions of care.’11 However, despite the evidence-based standards for discharge summary content,12 discharge summaries completed by residents often lack critical information10 ,13 ,14 and do not reach the correct outpatient provider in a timely manner.10 ,15

    Prior to our intervention, discharge summaries at our institution were completed an average of 3.5 days after patient discharge from the hospital and were therefore commonly not available during follow-up appointments or in the event of readmission (see results section for data). They were also time-consuming and a recognised contributor to duty-hours violations.16 With input from important stakeholders (including the Department of Medicine QI leadership, residency programme leadership and primary care physicians), residents set the goal of improving the timeliness and quality of their discharge summaries by implementing a discharge summary improvement bundle that embodied many of the characteristics of previously published, successful QI projects.11 ,14 ,17 The project fit within the Medical Center's overarching goal of improving transitions of care and the residency programme's goals to involve residents in QI. The primary aim of our project was to increase the percentage of discharge summaries completed on the day of discharge to at least 75%. We studied how the project impacted timeliness as well as quality of discharge summaries.

    Methods

    Setting

    Our project involved internal medicine residents at the University of California at San Francisco Medical Center (UCSFMC), a 600-bed academic medical centre. The residency programme includes 170 internal medicine residents, although only 123 rotated at the intervention hospital during the study period and were able to voluntarily participate in the QI project. During their inpatient medicine rotations at UCSFMC, residents work on one of eight teaching teams. Each team is comprised of one senior resident (post graduate year (PGY)-2 or PGY-3) and two PGY-1 residents. On average, 15 patients are discharged from the medicine teaching service daily and all patients hospitalised for more than 48 h require a discharge summary to be completed. Although the hospital had an electronic medical record (EMR) that was used for most clinical documentation, discharge summaries were completed via telephone dictation and transcription into the EMR prior to this intervention.

    In 2007, UCSFMC partnered with the Office of Graduate Medical Education to introduce the Housestaff Incentive Program.2 Each year the Program gives recognition and financial incentive payments to all residents in each department based on the achievement of three quality goals specified by the Medical Center, such as improved patient satisfaction, pain control and hand hygiene. In 2009, the Program expanded to allow residents within each department to also specify a unique QI goal for their department, subject to the approval of the Medical Center and Graduate Medical Education Committee. For each quality goal met in an academic year, every eligible resident received a bonus of $300. The eligibility of residents depended on whether they rotated at the study hospital during the intervention period.

    Ethical issues

    We considered the ethical issues around involving residents in this QI initiative. All residents were invited to participate in the selection of the goal and the design of the project. All elements of the discharge summary improvement bundle were optional; alternative systems for completing discharge summaries (ie, transcription system) remained unchanged. Performance feedback was reported anonymously by team and no individual data were provided. Surveys were optional and anonymous. The authors had no conflicts of interest.

    Planning the intervention

    A multifaceted intervention was planned that had features of previously published, successful QI projects including stakeholder involvement, educational efforts, system changes, frequent performance feedback and a financial incentive.11 ,14 ,17

    Residents, as the principle stakeholders, were invited to participate in every stage of this project. Two residents took a leadership role, while nearly 50 residents participated to a more limited extent, for instance, through brainstorming the goal, analysing data and refining the project through PDSA (plan-do-study-act) cycles. All residents rotating at the site received biweekly updates on incentive progress.

    At monthly sessions, resident champions educated fellow residents about best practices in transitions of care. Attendance was not monitored at these conferences, but residents were encouraged to attend as they are for all didactics during their inpatient wards rotation. A mandatory half-day session about discharge summary best practices was held for PGY-1 residents, facilitated by the chief residents.

    A template-based electronic discharge summary (EDS) was created within the existing EMR platform through collaboration between resident champions and members of the Division of Hospital Medicine. It contained both free text and EMR-populated fields to document all recommended discharge summary elements, including medication changes, functional status, follow-up plans, pending tests and discharge instructions for patients.12 To facilitate workflow and minimise redundancy, the EDS could be edited throughout the hospital admission by any member of the physician team and was unified with the last hospital day's progress note. No fields were mandatory in the discharge summary template, as residents elected to have their peers responsible for entering relevant patient information. Through a test of change, the EDS was piloted on three of the eight medicine teams for 3 months and modified based on their feedback.

    Data on discharge summary timeliness were audited biweekly and fed back to residents by team (no individual provider data were presented). These data were analysed by the resident champions as well as the 48 PGY-1 residents who participated in a QI rotation. When barriers to meeting the prespecified goal were identified, they were addressed in tests of change that often involved targeted educational efforts and provider feedback.

    The Medical Center offered a financial incentive of $300 to each resident if the overall goal of having 75% of discharge summaries completed on the day of discharge was achieved over the course of the year-long intervention. Individual performance data were not used to determine which resident would obtain the monetary award, but rather collective departmental performance was used to determine if all or no residents would receive the bonus.

    Planning the study of the intervention and methods of evaluation

    To study the efficacy of this project, we measured the change in timeliness and quality of discharge summaries over time. Surveys of residents, attendings and primary care doctors were used to reveal balancing measures and generate hypotheses about what aspects of the multifaceted intervention were most important.

    Timeliness

    The Billing and Compliance Office at UCSF provided weekly information about the time of each patient discharge, discharge summary completion and attending signature of the discharge summary. Timeliness was measured in two ways: (1) time from patient discharge to resident completion of the discharge summary and (2) percentage of discharge summaries completed on the day of discharge (the Incentive Program goal). A total of 563 discharge summaries from the pilot period (April 2010 through June 2010) were audited for baseline timeliness. After full implementation of the EDS, the overall and team-specific rates of discharge summary completion on the day of discharge were calculated biweekly. The two resident champions presented these results to residents at biweekly conferences, through posted flyers and via email. Though resident feedback stopped in June of 2011, we measured timeliness monthly for 6 months after the Incentive Program ended to determine the lasting impact of the intervention. There was no financial incentive associated with achieving our goal after June 2011.

    Quality

    A discharge summary evaluation rubric was created based on prior literature about the quality of discharge summaries.12 ,18 Attending reviewers were asked to assess whether each of the criteria was present, absent or not applicable in a sample of discharge summaries. Two reviewers scored groups of 10 duplicate summaries until 100% rating consensus was reached. The final scoring rubric consisted of 13 quality criteria (see online supplementary appendix 1).

    The quality of 80 randomly selected discharge summaries from the preintervention period was compared with the quality of 80 randomly selected discharge summaries from two time points in the postintervention period: (1) during the first 2 months of the intervention and (2) 6 months after the start of the intervention. To provide a balanced comparison, the samples were limited to unique, living patients with a length of stay more than 48 h.

    Surveys

    At the end of the academic year, residents and faculty who had worked on the medical service during the intervention period were asked to provide feedback regarding their experiences with the intervention. The survey focused particularly on their opinions about this project, which aspects of the intervention motivated their behaviour change and unintended consequences of this project (see online supplementary appendix 2A and B for resident and attending surveys).

    Outpatient provider satisfaction was evaluated using an existing survey of referring outpatient physicians. The physician database of UCSF was used to identify outpatient physicians (both primary care physicians and outpatient-based specialists) who had at least three of their patients admitted to UCSF in the prior year. An initial survey was conducted in June 2010, just prior to implementation of the QI project, and a similar survey was conducted in July 2011 (see online supplementary appendix 3 for referring provider survey).

    Analysis

    We used χ2 tests to compare the proportion of discharge summaries completed on the day of discharge before and after the intervention. The proportion of discharge summary quality criteria met before and after the intervention was also compared using χ2 tests. We used Student t tests to compare the time between discharge to resident and attending discharge summary signatures during the preintervention and postintervention period. The UCSF institutional review board, the Committee on Human Research, approved this study.

    Results

    Nature of the improvement intervention

    A timeline of our multifaceted intervention is provided in figure 1. Due to the constraints of achieving our aims within one academic year, many elements of the discharge summary improvement bundle were implemented simultaneously. Residents performed recurrent PDSA cycles to facilitate improvement in our discharge summary timeliness. For instance, one audit revealed that discharge summaries were likely to be completed late when PGY-1 residents on a team were off duty on the patient's day of discharge. This led to an educational campaign emphasising the importance of initiating discharge summaries early in anticipation of a patient being discharged. Residents also found that deceased patients were more likely to have discharge summaries completed late, which led to interactive discussions about the importance of timely chart completion even for deceased patients.

    Figure 1
    Figure 1

    Timeline showing the year-long rollout of the multifaceted intervention.

    Discharge summary timeliness

    There were 563 discharge summaries completed during the 3-month preintervention period and 2560 summaries completed after the intervention. Voluntary adoption of the EDS template was high, with 77/80 (96%) discharge summaries being done via the EDS template in early audit and 80/80 (100%) in the late audit, even though the telephone dictation system remained available.

    With the intervention, average time from patient discharge to completion of the discharge summary fell from 3.5 to 0.61 days (p<0.001). The percentage of discharge summaries completed on the day of discharge rose from 38% to 83% (p<0.001). Improvements in timeliness were achieved rapidly after implementation; the prespecified goal of completing greater than 75% of summaries on the day of discharge was achieved in the first month of the intervention (figure 2). The timeliness of discharge summaries was maintained throughout the duration of the study period, which included the academic year during which the Incentive Program applied as well as 6 months after the incentive ended.

    Figure 2
    Figure 2

    The timeliness of resident completion of discharge summaries during the preintervention, intervention and postintervention periods.

    Discharge summary quality

    The percentage of discharge summaries that included all recommended quality criteria increased from 5% to 88% (p<0.001) after the intervention (table 1).

    View this table:
    Table 1

    Quality of discharge summaries before and after project implementation

    Regarding discharge summary elements most desired by primary care doctors,12 the rate of documentation of medication reconciliation (detailed information about medications stopped, new medications started and medications continued) increased from 10% to 88% (p<0.001) and the rate of documentation of pending tests and follow-up plans each increased from 34% to 98% (p<0.001).

    Resident and attending surveys

    Of the 123 residents who completed a wards rotation at the study hospital during the intervention, 46 (37%) completed a survey about their perceptions of the project. These limited survey data suggest that systems changes were likely the most powerful motivators for residents. In all, 91% of residents reported that the unification of the last hospital day’s progress note with the discharge summary motivated their behaviour change and 87% said the ability to start the note ahead of time motivated them. Overall, 88% of residents said they were motivated by a belief that timely discharge summaries can improve continuity of care for their patients. In all, 73% of residents felt that the effort required to complete discharge summaries on the day of discharge was worth the gain in quality of care rendered. Residents reported that the financial incentive was a less powerful motivator than other factors (figure 3). When asked about balancing measures, only 17% of residents felt the goal led them to prioritise discharge summaries over providing direct clinical care.

    Figure 3
    Figure 3

    Resident perceptions on motivating factors to participate in the timely completion of discharge summaries.

    A total of 36 faculty members, representing 73% of the faculty who attended on the inpatient medicine service during the study period, completed the attending survey. Their opinions were largely congruent with those of the residents. All surveyed attendings believed that the effort required to complete discharge summaries on the day of discharge was worth the gain in quality of care. Only 12% felt that the pressure to complete summaries on the day of discharge led to premature summaries.

    Referring physician survey

    In the year prior to project implementation, 55 of 88 (62.5%) referring providers responded to a survey. During that time, 38% reported that they often or always see a patient in clinic following discharge before the discharge summary is available. In all, 11% reported that discharge summaries were often ‘missing information critical for patient care’. In the year following project implementation, 48 of 80 (60%) surveyed providers responded to the survey. At that time, only 4% of providers reported that they often or always see a patient in clinic before the discharge summary is available (p<0.01); only 4% reported that discharge summaries are often missing critical information (NS).

    Discussion

    Summary

    We describe a novel, resident-led QI project that improved discharge summary timeliness and quality. After implementation of the discharge summary improvement bundle, discharge summaries were completed in one-sixth as much time as previously; the large majority were available on the day of discharge. The rates of documentation of elements important to patient safety, such as medication reconciliation and pending tests,12 improved significantly. Though a financial incentive programme supported this project, our results exceeded the goal required to receive the incentive and were sustained after the incentive. Survey responses suggest that numerous aspects of the multifaceted intervention contributed to the observed behaviour change.

    Relation to other evidence

    Our project is congruent with previous literature that has shown greatest success from multifaceted QI interventions that include stakeholder involvement, educational efforts, system changes, frequent performance feedback and a financial incentive.7 ,11 ,14 ,17 These principles may be instructive to those working to engage residents in QI projects, and in this way we add to the scarce literature about involving residents in meaningful QI.5

    Limitations

    Our project also demonstrates some of the challenges of incorporating QI projects into the busy training environment. Most notably, under pressure to effect change within a single academic year, our multifaceted intervention was rolled out in a bundle rather than a stepwise fashion. Although our survey results generate hypotheses about which aspects were most influential, the bundled approach limits our ability to determine the effects of each aspect of our intervention. Future projects that implement each element of the intervention sequentially could provide more information.

    The response rate to the resident survey was low and so all conclusions extrapolated from this survey should be considered preliminary and hypothesis generating. The low response rate may be in part because the survey was sent at the end of the academic year and at a time when residents were being surveyed heavily in anticipation of changes to duty-hours requirements, and so ‘survey fatigue’ may have occurred. Some PGY-3 residents may not have responded because they had graduated from the residency.

    Although improvements in discharge summary timeliness and quality were of large magnitude and temporally associated with our intervention, this observational study does not conclusively exclude the possibility of other factors contributing to the improvements.

    Further, while our discharge summary quality rubric was based on previous research, it is not a validated measure. Our quality reviewers were not blinded, as they were able to determine which discharge summaries were done by dictation versus electronically via formatting differences.

    Finally, while all members of the residency were invited to participate in all stages of the project, two resident champions led the project from start to finish. This project did not replace the residency QI educational curriculum, but rather provided all residents rotating through the site with practical experience participating in a QI initiative.

    Balancing measures

    We were concerned that residents would prioritise discharge summaries over more important patient care, write premature summaries or feel excessively stressed by this intervention, but based on the data we have from the resident and faculty surveys, we found little evidence for such consequences.

    Generalisability

    We believe that resident-led QI projects are most likely to be successful in a setting where residents have protected academic time, strong QI faculty support, departmental backing and an adaptable EMR system. This project would have been difficult to execute at an institution without these qualities. Resident champions used protected academic time, totalling about one full day per month, to execute the QI project. Faculty members who are experienced in QI advised and supported the residents at each stage of the project including selection of the goal, project design, analysis of data and improvement strategies. Without faculty guidance, this project may not have been as successful.

    While departmental and medical centre backing enabled the financial incentive, our results suggest that the financial incentive was not the primary motivator for change in resident behaviour. Instead, the financial incentive likely increased awareness of the project and encouraged early adoption. We believe the success of the project stemmed from the selection of a goal that residents felt was important as well as the introduction of systems changes that facilitated completion of the goal. It is unclear how the results of this project would differ in an environment where a financial incentive is not available.

    Conclusions

    In conclusion, we describe an example of a resident-led QI project that was able to effect rapid and lasting behaviour change. Our experience demonstrates that residents, who are at the frontline of academic medical care, are well positioned to improve health systems in cooperation with hospital leadership. Our project may serve as a model for other training programmes as it highlights how traditional QI principles may be applied to engage residents in effective QI. Such projects prepare residents for today's careers in medicine by providing a practical experience to complement QI curricula.

    Acknowledgments

    The authors would like to thank Arlene Boro and the Department of Medicine's Revenue Management and Compliance Office staff for their provision of discharge reports. They would also like to extend thanks to Arpana Vidyarthi, MD and Adrienne Greene, MD for their support of this project through its entire process.

    References

    1. ↵
      1. Jenson HB,
      2. Dorner D,
      3. Hinchey K,
      4. et al
      . Integrating quality improvement and residency education: insights from the AIAMC National Initiative about the roles of the designated institutional official and program director. Acad Med 2009;84:1749–56.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Vidyarthi AR,
      2. Baron RB
      . Financial incentives for residents and fellows: a disruptive innovation to drive quality improvement. Acad Med 2011;86:1338.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Scott DR,
      2. Weimer M,
      3. English C,
      4. et al
      . A novel approach to increase residents’ involvement in reporting adverse events. Acad Med 2011;86:742–6.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Laiteerapong N,
      2. Keh CE,
      3. Naylor KB,
      4. et al
      . A resident-led quality improvement initiative to improve obesity screening. Am J Med Qual 2011;26:315–22.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Patow CA,
      2. Karpovich K,
      3. Riesenberg LA,
      4. et al
      . Residents’ engagement in quality improvement: a systematic review of the literature. Acad Med 2009;84:1757–64.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Kiefe CI,
      2. Allison JJ,
      3. Williams OD,
      4. et al
      . Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA 2001;285:2871–9.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Oxman AD,
      2. Thomson MA,
      3. Davis DA,
      4. et al
      . No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 1995;153:1423–31.
      OpenUrlAbstract
    8. ↵
      1. Grol R
      . Improving the quality of medical care: building bridges among professional pride, payer profit, and patient satisfaction. JAMA 2001;286:2578–85.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Grimshaw JM,
      2. Thomas RE,
      3. MacLennan G,
      4. et al
      . Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:iii–iv, 1–72.
      OpenUrlPubMed
    10. ↵
      1. Fair JF
      . Hospital discharge and death communications. Br J Hosp Med 1989;42:59–61.
      OpenUrlPubMedWeb of Science
    11. ↵
      1. Greene SE,
      2. Nash DB
      . Pay for performance: an overview of the literature. Am J Med Qual 2009;24:140–63.
      OpenUrlFREE Full Text
    12. ↵
      1. van Walraven C,
      2. Rokosh E
      . What is necessary for high-quality discharge summaries? Am J Med Qual 1999;14:160–9.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Tulloch AJ,
      2. Fowler GH,
      3. McMullan JJ,
      4. et al
      . Hospital discharge reports: content and design. Br Med J 1975;4:443–6.
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Shortell SM,
      2. Bennett CL,
      3. Byck GR
      . Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress. Milbank Q 1998;76:593–624, 510.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Penney TM
      . Delayed communication between hospitals and general practitioners: where does the problem lie? Bmj 1988;297:28–9.
      OpenUrlFREE Full Text
    16. ↵
      1. Mourad M,
      2. Vidyarthi AR,
      3. Hollander H,
      4. et al
      . Shifting indirect patient care duties to after hours in the era of work hours restrictions. Acad Med 2011;86:586–90.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Solberg LI
      . Guideline implementation: what the literature doesn't tell us. Jt Comm J Qual Improv 2000;26:525–37.
      OpenUrlPubMed
    18. ↵
      1. O'Leary KJ,
      2. Liebovitz DM,
      3. Feinglass J,
      4. et al
      . Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary. J Hosp Med 2009;4:219–25.
      OpenUrlCrossRefPubMedWeb of Science
    View Abstract

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors All authors were involved in conceiving of this study. KB, AG, SRR and MM were involved in the literature search that formed the background of this study. KB, AG and MM performed the data collection and analysis. KB and AG primarily drafted the manuscript with input from all authors. All authors have critically revised the manuscript and approved the final draft prior to publication.

    • Funding The UCSF Medical Center and the Office of Graduate Medical Education awarded financial incentive payments to residents an amount of $300 per person for achieving the department-specific goal. The University of California San Francisco Internal Medicine Residency Program's membership in the Accreditation Council of Graduate Medical Education Educational Innovations Project supported this work and the curricular innovations that followed.

    • Competing interests None.

    • Ethics approval The University of California San Francisco's institutional review board, The Committee on Human Research, granted expedited approval of this study.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Previous presentations Poster presentation at 2011 AAMC Integrating Quality Meeting.