Palliative care focuses on prevention and relief of suffering, improving communication, promoting concordance between treatment and individual preferences, and facilitating transitions across care settings for patients with life threatening illness and their families.^1–3 As such, it is increasingly accepted as an integral component of comprehensive ICU care for all critically ill patients, including those pursuing every reasonable treatment to prolong life.^4–8 At the same time, evidence has accumulated that the quality of ICU palliative care needs improvement:^9,10 patients experience high levels of pain and other distressing symptoms;^11–13 families fail to understand basic information about diagnosis, prognosis, or critical care treatments¹⁴ and experience high levels of depression and anxiety;¹⁵ care plans diverge from patients’ and families’ preferences;^16,17 and conflict among ICU clinicians, patients, and families is common.^18,19

The Institute of Medicine identified improvement of palliative care in the ICU and other care settings as a national health priority.⁸ For all healthcare providers and fields, it has also prioritized “closing the gap” between the current knowledge of optimal care and current clinical practice.²⁰ A consortium of five major US palliative care organizations undertook the National Consensus Project for Quality Palliative Care¹ which recently established clinical practice guidelines that are “applicable to specialist level palliative care delivered in a range of treatment settings, as well as to the work of providers in primary treatment settings where palliative approaches to care are integrated into daily clinical practice”. For the ICU, specifically, the Critical Care Peer Workgroup of the Robert Wood Johnson Foundation’s Promoting Excellence in End-of-Life Care Project²¹ used literature review and expert consensus to identify seven domains of palliative care quality and a series of clinician and organizational behaviors and interventions that could address indicators of quality in these domains.²² Over 85% of nearly 600 respondents to a recent national survey of physician and nurse directors of critical care units endorsed quality monitoring as a helpful strategy for improving ICU palliative care, while only 25% reported the availability of such a program in their ICU.²³ To evaluate palliative care and promote consistency as well as high quality in its delivery, ICUs would require operational measures that are feasible as well as scientifically sound.²⁴ Measures of this type have not been described to date for the critical care setting.

Quality of care can be evaluated on the basis of structure, process, or outcome.^25,26 Compared with other approaches, process measures are generally more sensitive and practical, and are also advantageous in that they focus on activities that are actionable by healthcare professionals.^25,27 The validity and credibility of such measures, however, depends ultimately on showing that variations in the processes of interest are associated with improved outcomes.²⁵ In this report we present a new set of process measures of the quality of ICU palliative care. We describe the development of this prototype, which is intended to be scientifically sound yet feasible for routine quality measurement and ongoing performance feedback to ICU caregivers. We also discuss steps and issues involved in selecting measures for inclusion and in drafting specifications and definitions; describe the results of pilot implementation; and suggest areas for further research to confirm validity.

METHODS

The “bundle” strategy for quality improvement

This project was undertaken as part of the national “Transformation of the ICU” (TICU) program, a performance improvement initiative through VHA Inc.²⁸ VHA is a cooperative network of more than 25% US not-for-profit community-based hospitals which (among other activities and services) educates ICU teams to measure performance on specific sets of quality indicators, provides “peer” benchmarks, and sponsors periodic telephone conferences, electronic discussions, and meetings to share data, experiences and ideas. To date, 75 ICUs in hospitals across the US have participated in the TICU program, represented by interdisciplinary teams including ICU physicians, nurses, social workers, respiratory therapists, and pharmacists.

TICU quality measures are organized into “bundles”—that is, a grouping of process measures based on best practices that individually improve care and are applied together for a fuller assessment of the quality of care in a clinical area.^29–31 For example, ICUs have implemented a “ventilator bundle” including semi-recumbent positioning, appropriate prophylaxis against deep venous thrombosis and peptic ulcer disease, and daily assessment for sedation and possible extubation.^29,32 Bundles of process indicators are also used both inside and outside the TICU program to measure the quality of sepsis management.³¹

Impetus for palliative care quality measurement

During 2003 and 2004 many TICU clinical teams requested programmatic assistance from VHA to improve the quality of palliative care in their units. They reported serious concerns about conflict over care goals between teams and patients/families as well as among different disciplines and specialists caring for the same patients, about perceived inadequacies in clinicians’ communication with patients and families, about unrelieved suffering of patients and families, and about other unmet family needs. The teams shared verbal and written complaints in these areas from patients and families, and described stresses and “burnout” among caregivers that the teams attributed to problems with palliative care in their ICUs. In two sessions dedicated to this topic at the November 2003 national TICU meeting (attended by approximately 200 representatives of approximately 40 ICUs), and in a series of three subsequent telephone conferences during this period in which the ICU teams participated with TICU project leaders (each involving participants from 10–15 separate ICUs teams for a total of approximately 40 teams), a broad consensus emerged in favor of developing performance measures for palliative care, similar to those already used by TICU ICUs in other practice areas. Accordingly, we undertook the development of a bundle of palliative care indicators.

Interdisciplinary collaboration

Our palliative care bundle development team (“project team”) represented the disciplines of medicine (two critical care physicians), nursing (one critical care nurse), and social work (a social worker assigned to ICUs and to palliative care consultation), with ongoing advice from a healthcare quality improvement consultant and two technical experts in quality measurement and data management from VHA. This team combined expertise and experience in critical care (medical and surgical), palliative care, and performance measurement to improve the quality of ICU care. Throughout the development process the project team collaborated in face-to-face meetings, telephone conferences, and electronic communications, making important decisions about the content and format of the measures by consensus after review and discussion among all members. In addition, dedicated sessions at biannual national meetings of VHA’s interdisciplinary TICU teams in 2004 and 2005 (attended by 186–191 members of 38–43 ICU teams, including physicians, nurses, ICU administrators, respiratory therapists, and pharmacists), and a series of four conference calls with these teams during the same time period (10–15 ICU teams per call) afforded access to their input from the time of the earliest planning discussions, through each major iteration of the measure set, to review of the pilot implementation of the measures at the end of 2005. We received this input both orally and in writing, recorded all comments and critiques, and incorporated them in each of four successive iterations.

Domains of quality of ICU palliative care

In a previous initiative sponsored by the Robert Wood Johnson Foundation (RWJF), the Critical Care Peer Workgroup of Promoting Excellence in End-of-Life Care²¹ synthesized research evidence and developed a consensus of expert opinion to identify seven domains of palliative care quality for the ICU.²² To evaluate the ongoing relevance and validity of these domains in a rapidly developing field, we performed an updated review of English language articles in PubMed using the search terms “intensive care”, “critical care”, and “critical illness” together with “quality improvement”, “quality assessment”, “quality care”, “quality indicators”, “palliative care”, “dying”, “end-of-life”, “good death”, “patient satisfaction”, “family satisfaction”, “preferences”, “communication”, “decision making”, “conflict”, “pain”, and “symptom”. We augmented our database search by reviewing references in retrieved articles, by accessing relevant websites (such as the Center to Advance Palliative Care² and Promoting Excellence in End-of-Life Care²¹ websites, which contain bibliographies), and by searching our personal files. We concentrated on randomized controlled research, rigorous observational research, and peer reviewed reports of interdisciplinary expert consensus; two clinical investigators on our project team (combining expertise in critical care, palliative care, and healthcare quality improvement research) evaluated validity and clarity of description of methods in the literature reviewed. We also reviewed the clinical practice guidelines recently published by the National Consensus Project for Quality Palliative Care^1,33 and current standards applied to ICUs by the Joint Commission for Accreditation of Health Care Organizations.³⁴ Our literature review supported the face and content validity of the domains previously described by the Critical Care Peer Workgroup, although we found little evidence defining attributes of quality palliative care from the perspective of ICU patients or their families (as opposed to taxonomies derived through professional consensus).³⁵

Measures of quality of palliative care

We then used the framework established by the National Healthcare Quality Report²⁴ which mandates that measures of healthcare quality not only address important practice issues and meet scientific standards, but that they are also feasible for the intended purpose of quality measurement. The project team decided that for our purpose—which was to develop measures for routine monitoring and performance feedback of the quality of palliative care across a broad range of ICUs—the advantages of a process approach outweighed its disadvantages compared with direct measurement of outcomes.^25,27,36 Based on previous experience in developing measures of quality for other areas of ICU practice within and outside the TICU program,³⁷ we concluded that feasibility would depend on the distillation of a short list of key measures rather than implementation of a more exhaustive but potentially burdensome inventory measuring all components of palliative care. Pragmatic concerns also prompted our decision to focus on documentation of care processes in the medical record, as was the data source for other measure sets successfully implemented in the TICU performance improvement program.

In accordance with research based recommendations for development of brief composite scales,³⁸ we used expert judgment for content sampling and narrowing of quality measures within the previously established domains. We reviewed the evidence synthesized by the Critical Care Workgroup,²² as updated by our own literature review, to identify ICU palliative care processes that were associated with desirable outcomes. Only a small number of studies used a rigorous empirical approach to establish such associations, reflecting both the nascent stage of development of this field of research and the unique difficulties of measuring quality outcomes in palliative care.³⁹ Even fewer focused on outcomes of particular importance to critically ill patients and families, as opposed to resource utilization outcomes (such as length of ICU stay). We also referred to processes identified as important by expert consensus^1,22,33 and sought to address deficiencies in ICU palliative care that had been demonstrated in rigorous descriptive studies. From these data, our project team culled a candidate list of indicators that we unanimously agreed to include, and we proceeded to draft corresponding quality measures, a measurement guide containing detailed specifications and definitions for the numerators and denominators of our measures, and a data collection tool.

Successive iterations of the bundle of palliative care measures

We formally solicited feedback on our first iteration of this bundle from 30 ICU teams representing 37 ICUs during a 4 hour face-to-face session conducted for this purpose at the June 2004 national TICU meeting. Before the meeting the teams received the bundle draft and were asked to bring to the meeting their feedback (oral and/or written) with respect to content and format. We also asked in an open ended manner whether additional content should be included in the measure set. No new measures were suggested at this meeting, nor was any measure in the initial draft felt to be unnecessary by the participating teams. Most discussion focused on specifications and definitions. We recorded all comments and used them to prepare a revised version of the bundle.

The revised version was distributed—together with a request for input about content and format—to an interdisciplinary panel of national experts outside the TICU program, including two members of the Steering Committee of the National Consensus Project for Quality Palliative Care;^1,33 the Director and Research Director of the Center to Advance Palliative Care;² the Chair and other members of the Critical Care Workgroup of the Robert Wood Johnson Foundation’s Promoting Excellence in End-of-Life Care Project and Palliative Care Excellence in Intensive Care Project;²¹ a senior quality measure developer from the Joint Commission on Accreditation of Health Care Organizations;³⁴ and two clinical researchers who authored a systematic review of palliative care quality measures prepared through the RAND Corporation for the Agency for Healthcare Research and Quality.⁴⁰ We received substantive written comments from each of these reviewers, which again focused primarily on specifications and definitions; none of the reviewers suggested additional measures for inclusion. This feedback was incorporated in a further iteration of the palliative care bundle.

Pilot testing

The measures and data collection instrument were first pilot tested in three ICUs participating in the TICU program—one academic surgical ICU at a university affiliated tertiary care hospital and two mixed medical-surgical ICUs at separate community hospitals in different parts of the country. In all of these ICUs, intensive care specialists managed or co-managed all patients. On two random days per week between March and May 2005 the ICU nurse manager applied the measures in our bundle through review of medical records of each patient in the ICU that day whose length of stay exceeded 5 days. We then conducted in-depth interviews of the pilot data collector and staff in each of these ICUs, focusing on clarity, feasibility, and acceptability of the measures. The pilot feedback was incorporated in revised versions of the measures and the data collection tool. These were further refined after additional input from ICU teams attending national TICU meetings and telephone conferences in 2005.

We then conducted a larger scale pilot test in November 2005, requesting participation by 18 ICUs comprising one of the five major groups of the TICU which are organized by year of entry in the TICU program. Each of these ICUs was asked to apply the bundle over a 4 week period to at least five patients with a length of stay of ⩾5 days. Sixteen of the 18 ICUs completed the requested data collection. In December 2005 we held a telephone conference involving all of these teams to discuss the pilot experience and solicit any further suggestions for the measures or the data collection tool. Finally, we convened a meeting of the interdisciplinary project team to review the bundle of indicators emerging from this process, analyze results of the recent pilot study, and plan next steps including further validation and wider implementation.

RESULTS

Table 1 shows the new bundle of palliative care quality measures. Our iterative process readily achieved broad consensus about the “numerators”—that is, what indicators the bundle should include. In early discussions within the project team we considered inclusion of several additional indicators, specifically addressing continuity of care, bereavement support, communication within the ICU team (as distinct from clinician-patient/family communication), education of ICU staff in palliative care knowledge and skills, and emotional support for staff. However, based on past experience in implementing bundles of measures for other practice areas within and outside the TICU program, we concluded that inclusion of more than 8–10 measures in the palliative care bundle would compromise feasibility, and that we should focus on interventions supported by the strongest evidence in the existing literature (table 1).

Determination of the appropriate denominator (the patient group to whom these measures should apply) presented especially challenging issues. Although we believed that some indicators (such as effective symptom relief) should apply universally, we wanted to ensure feasibility and acceptability by limiting application of other measures such as social work support and the formal interdisciplinary family meeting to patients and families in particular need because of high risk for prolonged stay and hospital mortality. We considered using severity of illness scores or diagnostic classifications as criteria, but concluded that these strategies might not be fully generalizable. We ultimately chose specified periods of stay as the basis for applying indicators in our prototype bundle (table 1).

The measures are stratified by length of stay triggers at day 1 (the day after ICU admission) (identification of appropriate decision maker, advance directive status, CPR status, information leaflet), at ICU day 3 (social work and spiritual support), and at ICU day 5 (interdisciplinary family meeting). Performance can be measured concurrently/prospectively—that is, on the “trigger day” or on the next day; alternatively, an ICU can review medical records retrospectively on a subsequent day—for example, at ICU day 5 (the last trigger day) or day 6 for compliance with the full set of measures (triggered at days 1, 3 and 5). Using length of stay >5 days to define the “denominator” patient group is a strategy to minimize the measurement burden and target patients who are most likely to benefit from these measures. At the same time, this definition captures a substantial subgroup of the ICU population—for example, from January 2004 to December 2005 between 10% and 15% of over 4200 discharges from the 16 ICUs participating in the pilot study were patients with a length of ICU stay exceeding 7 days. In addition, implementation for patients within the definition would probably entail changes in ICU practice and redesign of systems extending beyond the defined group.

Detailed specifications, including definitions for precision and reliability of measurement and to facilitate implementation, are shown in table 2. For example, indicator 2 relating to documentation of advance directive status is accompanied by a working definition of “advance directive”: “Written or oral instructions from the patient specifying the type of medical treatment that is desired if the patient becomes incapacitated. Advance directives include living will, durable power of attorney (note state-specific status) or any document that state law recognizes as an ‘advance directive’.” Specifications also explain that performance on the measure is achieved if there is “(1) documentation that the patient has an advance directive or (2) documentation that a full investigation failed to identify an advance directive”. The essential content of an acceptable information leaflet for indicator 4 is outlined, based on the description provided by Azoulay et al⁴¹ in their randomized controlled trial of this intervention. Required participants and content for the family meeting indicator (number 9) are also identified: the meeting is to include “at least the attending physician (either primary attending or ICU attending), a member of another discipline (nurse, social worker, or pastoral care representative), and the patient (and/or family)”, and to address “(1) the patient’s condition (diagnosis and prognosis), (2) goals of treatment; (3) the patient’s and family’s needs and preferences … [and] (4) the patient’s and family’s understanding of the patient’s condition and goals of treatment”.

If ICUs choose to review medical records retrospectively for performance on the specified trigger days, numerators (defined in the specifications) could be adjusted to include the trigger day—for example, number of patients for whom there is documentation of advance directive status on or before day 1 of the ICU admission—while the denominator would then be adjusted to reflect the ICU sample or time point for review—for example, total number of patients with an ICU length of stay ⩾5 days. An underlying assumption of this approach is that, over time, retrospective reviews would prompt attention to relevant practice and documentation by the trigger days.

Table 3 describes the baseline performance of ICUs in the November 2005 pilot study. ICUs participating in this study included five medical-surgical units, four medical units, three surgical units, and four mixed units from a total of 10 separate hospitals (N = 94 patients). We found documentation of a family meeting for 34/85 patients (40%) who were in these ICUs for at least 5 days. Medical records reflected that an information leaflet was distributed to 38/88 of families (43%), and 33 families (37.5%) received an offer of spiritual support by day 3 in the ICU. ICUs included in the pilot test fared best on pain assessment (pain scores documented for 521/600 (87%) of 4 hour nursing intervals), and records showed that pain was alleviated during 411/482 intervals examined (85%). Since TICU ICUs have been measuring the quality of pain assessment and management in another bundle of indicators, they have more experience with the pain indicators and the effects of routine monitoring and performance feedback using these indicators may already be evident.

DISCUSSION

In this report we describe as a prototype a new “bundle” of indicators for routine measurement, performance feedback, and improvement of the quality of palliative care in critical care settings. The bundle addresses basic components of palliative care (identification of patient preferences for treatment and of surrogate decision makers for patients lacking capacity, communication between clinicians and patients/families, social work and spiritual support, and assessment and treatment of pain) which have been prioritized for quality improvement by the Institute of Medicine⁸ and by all major societies representing critical care professionals. This bundle is an initial effort to operationalize important domains of palliative care quality in the form of process measures that are actionable by providers while attempting to minimize the measurement burden for ICUs. Detailed specifications and definitions were developed for these measures as well as a data collection tool. Pilot implementation in a total of 19 ICUs showed that the measures are feasible and useable and that opportunities exist for quality improvement. To our knowledge, this is the first operationalized set of quality indicators for ICU palliative care to be reported.

We sought specifically to develop a short list of important indicators for continuous monitoring and regular feedback to caregivers. Although this type of abbreviated tool might not suffice to establish exceptional quality in all aspects of palliative care, an exhaustive inventory would not be practical for routine use across a wide range of ICUs. We strived for an appropriate balance, using both empirical data and expert judgment to select indicators for inclusion. Additional research, which we hope will be stimulated by this report, is needed to strengthen the evidentiary foundation for such choices. In particular, it will be important to investigate the perspectives of ICU patients and families on attributes of palliative care quality and the relative importance of indicators and interventions to promote quality improvement.

The time triggers for our quality indicators also represent a balance of quality, feasibility, and acceptability. Although we achieved consensus on these triggers and have tested their feasibility with favorable results on a pilot basis, we do not yet have evidence that these or other time periods for the measured processes will achieve the most favorable outcomes. Both within and outside the TICU program, however, data suggest that palliative care is often deferred even longer than the periods in our bundle, or never given in the ICU.^17,42 We believe it is reasonable, therefore, to propose a “lower threshold” (or “floor”) for timely delivery of important palliative care components, as we have done with the triggers in our bundle. ICUs can choose to “raise the bar” higher—for example, move the interdisciplinary meeting up from day 5 to an earlier point in time, if this is feasible and acceptable in their practice. Such changes would not require modification of specifications in the numerators of the measures.

We primarily used “process measures” addressing what caregivers do, rather than “outcome measures” of the results they may achieve. For example, the ICU palliative care bundle assesses whether social work and pastoral care have been offered, but does not directly measure whether such support was accepted or effective. Similarly, we determine whether a family meeting has occurred, not whether the participants were fully informed or satisfied by the communication. We chose this approach because it is most pragmatic, particularly for routine monitoring and performance feedback; because care processes are within the control of ICU professionals, who can be encouraged by measurement toward desirable behaviors; and because this approach has been successful in other practice areas addressed by TICU quality improvement bundles. In addition, we believe it is independently important to assess the implementation of a quality improvement effort, which is the focus of process measurement. We included processes that have been associated with patient-centred and family-centred outcomes such as distribution of the family information leaflet, which was shown to improve family comprehension.⁴¹ We recognize, however, that evidence linking most processes and outcomes in ICU palliative care remains limited, and that definitive validation of our measure set will depend on examination of these associations in future studies. As direct measures of patient, family and staff satisfaction with palliative care continue to be refined, it may also become appropriate to incorporate such measures. In general, however, it is difficult and expensive and potentially burdensome—not only for the ICUs but for critically ill patients and their families—to use outcome measures for routine ongoing monitoring in ICUs.

Performance indicators in our bundle are measured by medical record review rather than direct observation of care. We cannot therefore determine whether poor performance observed in our pilot testing reflects deficiencies in actual care or strictly in documentation, which we evaluated on a retrospective basis. However, our pilot results are consistent with previous data documenting patient distress and inadequate communication in the ICU,^{11,12,14,15,17,23,43} and reinforce concerns that prompted this and other initiatives to improve the quality of ICU palliative care. We have focused on aspects of practice that are, or could be, routinely included in chart documentation, such as identification of advance directive and resuscitation status and assessment of pain.⁴⁴ In addition, documentation is an important form of communication among members of the clinical team, serves as a reminder and trigger for clinician action, provides a basis for reimbursement of services, and represents a workable strategy for quality improvement across a range of ICUs.⁴⁴ Further research will be needed to confirm the measurement validity of this approach—that is, to confirm that medical record documentation is a sufficiently accurate representation of practice for the processes of interest.

The TICU program has accomplished its goal of improving ICU care through routine quality monitoring of sepsis management, treatment of mechanically ventilated patients, and prevention of catheter related bloodstream infections. The intention of our new palliative care bundle (as for previous TICU bundles) is to close the “quality gap” between existing best evidence and current daily practice. For the palliative care measures, which address a relatively new area of practice and research, we also intend to provide a prototype for palliative care quality measurement in the ICU, and to stimulate additional studies that will inform the further development of this approach. Our palliative care bundle reflects the collective judgment of a spectrum of ICU, palliative care, and quality improvement experts and practising clinicians in the US, including representatives of academic institutions and community hospitals across the country, and social workers, pharmacists, and administrators as well as physician and nurses on ICU teams inside and outside the TICU program. However, before wider implementation, both qualitative and quantitative investigations are needed of patients’ and families’ views of high quality ICU palliative care, as are rigorous trials of a range of palliative care interventions, and additional studies of process-outcome relationships. In the meantime, as they work to fulfil the Institute of Medicine’s mandate to improve the quality of palliative care, ICUs may benefit from the experience and prototypical work product described here.