Article Text
PDF
Abstract
Introduction In 2015, the Centers for Medicare and Medicaid Services developed a national quality bundle for the management of patients with severe sepsis and septic shock (SEP-1). Despite performance improvement measures, compliance remains low. This needs assessment is the first stage of a quality improvement initiative to improve SEP-1 compliance. Using a conceptual outcomes framework, this needs assessment analyses SEP-1 compliance data, knowledge, and competence to identify gaps in care and educational opportunities.
Methods The needs assessment began with a review of national and statewide SEP-1 compliance data to identify a need for improvement. The needs assessment proceeded with a retrospective chart review to evaluate process measures and identify which providers would most likely benefit from educational interventions. A focus group provided perspective on the chart review findings.
Results During the period of 1 April 2017–31 March 2018, national SEP-1 compliance was 51% and compliance at the studied institution was 19%. The chart review included 51 patients (66.7% severe sepsis, 33.3% septic shock). Frequently missed SEP-1 measures included administration of intravenous fluids (0% severe sepsis, 58.8% septic shock), repeat lactate levels (52.6% severe sepsis, 60% septic shock), documentation of volume and tissue perfusion assessment (58.8%), vasopressor administration (73.3%) and administration of broad-spectrum antibiotics (76.5%, severe sepsis). Focus group perceptions identified themes related to gaps in declarative and dispositional knowledge.
Conclusions This educational needs assessment highlights gaps in SEP-1 clinician performance, competence and knowledge. A multifaceted education programme is the next step for this performance improvement project. Education should include a series of meetings, activities, and workshops that include declarative knowledge, procedural knowledge and dispositional knowledge. Simulation activities can provide an opportunity for providers to demonstrate competence. Point-of-care prompts and performance measurement and feedback of patient care data can support clinician performance. This needs assessment underscores the need for a multifaceted approach to clinician education and performance to improve SEP-1 compliance.
- Sepsis
- Education
- Healthcare quality improvement
- Health professions education
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Compliance with the 2015 Centers for Medicare and Medicaid Services quality bundle for management of patients with severe sepsis and septic shock (SEP-1) remains poor. Educational activities can affect change and improve awareness, knowledge, skills and attitudes related to the guideline components.
WHAT THIS STUDY ADDS
This study identified gaps in care and opportunities for improvement related to SEP-1 bundle compliance.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The authors propose educational intervention strategies to address the identified gaps in knowledge.
Introduction
Sepsis is a medical emergency and a public health issue. Described as life-threatening organ dysfunction caused by a dysregulated host response to infection, 1.7 million adults in America develop sepsis annually and nearly 270 000 die as a result.1 One-third of patients who die in the hospital have sepsis listed as the cause of death.1 In the USA, the mean cost for sepsis-related readmissions is estimated to be $16 852 per admission with an annual cost exceeding $3.5 billion.2
In 2015, the Centers for Medicare and Medicaid Services (CMS) developed a national quality bundle for the management of patients with severe sepsis and septic shock (SEP-1) based on the Surviving Sepsis Campaign (SSC) 2004 guidelines for the management of patients with severe sepsis and septic shock.3 Despite broad implementation of the guideline, compliance with the bundle remains low. In 2021, national compliance with the bundle was 60%.4
Performance improvement programmes are associated with improved bundle compliance and a decrease in mortality in those patients with sepsis, severe sepsis and septic shock.5 Projects that include both education and process change are associated with the greatest survival benefit.5 6 Using a conceptual outcomes framework,7 this needs assessment aims to analyse SEP-1 compliance data and focus group themes to identify gaps in care, knowledge and competence among medical attendings, residents, and advanced practice providers (APPs) regarding the care of patients with sepsis. This needs assessment is the first part of a quality improvement (QI) initiative to improve outcomes for patients with severe sepsis and septic shock. Proposed interventions address the gaps noted from the needs assessment and suggest next steps.
Review of literature
Sepsis, severe sepsis and septic shock definitions
Since the SSC guideline development in 2004,3 there have been updated guidelines in 2008,8 2012,9 201710 and most recently in 2021.11 The guideline updates include a change in the definition of sepsis; an elimination of the category of severe sepsis and systemic inflammatory response syndrome (SIRS) and the specific parameters of end-organ dysfunction; and the development of a 1-hour bundle that stresses the administration of antibiotics within 1 hour of sepsis diagnosis. Despite the guideline modifications, CMS has not changed the SEP-1 bundle. CMS continues to define SIRS, sepsis, severe sepsis and septic shock as published in 1992 (table 1) and continues to uses the 3-hour and 6-hour treatment bundles.12 Severe sepsis is considered an infection or suspected infection with two or more SIRS criteria plus one sign of organ dysfunction.13 Septic shock criteria include an initial lactate level greater than or equal to 4 mmol/L or persistent hypotension after volume resuscitation with a 30 mL/kg crystalloid fluid bolus.
Definitions of systemic inflammatory response syndrome (SIRS), infection, sepsis, severe sepsis and septic shock13
SEP-1 compliance
SEP-1 compliance is challenging due to its complexity.14 15 The measure includes a 3-hour bundle for patients with severe sepsis and a 6-hour bundle for patients with septic shock. Each bundle includes several measures that must all be performed completely, documented accurately and abstracted correctly in order to meet compliance.14 An analysis of the non-compliant components can help to target efforts to improve SEP-1 compliance.5 Patients may present to the emergency department with sepsis/severe sepsis/septic shock, or they may develop one of these conditions during a hospital stay for another problem. Patient location within the hospital is found to have an impact on SEP-1 compliance. Evaluating location data can help to direct educational efforts to the providers that are consistently failing to meet the bundle.5
Researchers consistently find that some of the more controversial components of the bundle, such as repeat lactate testing and fluid administration, tend to be areas of higher non-compliance.16 When Cabana et al17 explored physician compliance with guidelines, they found that ‘physician adherence is dependent on physician awareness, agreement, self-efficacy, outcome expectancy, motivation, and the absence of external barriers to perform guideline recommendations’ (p. 121). Educational activities can affect change in these areas and improve awareness, knowledge, skills and attitudes related to the guideline components.
Framework
A well-designed needs assessment is a systematic method of identifying the gap between current and desired conditions and can aid in prioritising needs.7 18 The conceptual outcomes framework used for this needs assessment has been updated from the 2009 expanded outcomes framework.19 Starting with the end in mind, educational planners move down the framework, identifying performance gaps until reaching a level at which no gap exists or level 3, the lowest level for gap analysis (table 2).7 19 Planners then devise an intervention strategy that has the prospect of addressing each gap starting with the lowest level and reaching as high up as resources permit.
Conceptual outcomes framework7 19 and examples for evaluation
Level 7, population health status, is assessed first and may be the goal of a performance improvement project. Population health status describes the health of a community in terms of one or more important outcomes, for example, the mortality rates for patients with sepsis at the national or community level. Level 6, patient health status, considers patient health outcomes at an individual provider or organisation level. Patient heath status can be assessed by noting the number of patients in a provider group or hospital network who receive the standard of care. This can be evaluated with SEP-1 compliance at the practice or institutional level. Level 5, clinician performance, is a measure of providers’ performance in completing processes of care. Clinician performance can be assessed with process measures associated with the SEP-1 bundle.
The conceptual outcomes framework stresses the differences between level 4, competence, and the different types of knowledge (ie, declarative, procedural and dispositional) included in level 3, knowledge.7 Competence is the integration or ‘combination of declarative, procedural and dispositional knowledge that enables an individual to perform a task effectively’ (p. 907).7 Upon identifying gaps in competence, further assessment of declarative knowledge, procedural knowledge and dispositional knowledge should occur to understand if any gaps exist at this level. Results of a gap analysis, especially at levels 3 and 4, inform the intervention strategy, which often includes an educational component.
The conceptual outcomes framework recognises the importance of clinician participation as a level 1 outcome and clinician satisfaction with the educational offering as a level 2 outcome. Level 1, participation, is often reported by educators as attendance. Level 2, satisfaction, refers to the ‘degree to which the expectations of the participants about the setting and delivery of the CME (continuing medical education) activity were met’ (p. 3).19 Satisfaction with a programme can impact the participant’s motivation to learn and interest in changing behaviour.20
Using this conceptual outcomes framework, planners can identify gaps in performance, competence and knowledge then design interventions to address these gaps, with the goals of improving performance and patient health.
Methodology
The needs assessment was performed at a single, academic, acute care hospital in the northeastern region of the USA. The needs assessment started at levels 7 and 6, evaluating population and organisation levels for improvement opportunities. Next, a retrospective chart review and analysis evaluated clinician performance, level 5 of the conceptual outcomes framework. These level 5 findings were evaluated further with a focus group discussion to identify perceptions regarding the level 5 findings and to identify gaps at levels 4 and 3.
Levels 7 and 6 assessment
Population health, level 7, and patient health, level 6, were assessed with data retrieved from Hospital Compare,4 a US government consumer-oriented website that shares hospital-reported data to evaluate national, statewide and hospital compliance with the SEP-1 measure.
Level 5 assessment
The chart review included adult (≥18 years of age) inpatients eligible for CMS reporting (table 3) who were discharged between 1 October 2018 and 30 September 2019. The stratified sample included 150 patients. Ten qualifying cases per month were analysed except for 1 month, during which only three patients met the criteria. The sample of 113 stratified patients, representing a 12-month period, provided a satisfactory sample to evaluate frequencies for this needs assessment.
SEP-1 inclusion and exclusion criteria21
Data were extracted from the electronic health record (EHR) by one author (TD). Gathered data included the SEP-1 criteria in compliance with the CMS manual V.5.5a.21 Each missed measure was evaluated to determine the cause of the failure, the medical specialty responsible for caring for the patient and the patient’s location within the hospital at the time the measure was missed.
Levels 4 and 3 assessment
A focus group was used to explore perceptions concerning the results obtained from level 6, patient health status, and level 5, clinician performance, specifically exploring gaps in level 4, competence, and level 3, knowledge.
The QI Committee at the studied institution provided a suitable opportunity for this focus group. One author (TD) developed and shared a Microsoft PowerPoint presentation that included a review of the conceptual outcomes framework in the context of the needs assessment: the data collected at levels 7, 6 and 5; and definitions and examples of competence, dispositional knowledge, declarative knowledge, and procedural knowledge.
Two days after the focus group, content was recalled and transcribed by the author (TD). These notes were reviewed, clarified and supplemented by another author (MD), who was present during the focus group. The focus group content was categorised into the themes relevant to the conceptual outcomes framework level 4, competence, and level 3, knowledge. The level 3 content was considered for further categorisation of declarative knowledge, procedural knowledge and dispositional knowledge.
Levels 2 and 1 assessments
Assessments for levels 2 and 1 are important to assess as part of the formative and summative evaluations with all education sessions. However, assessments of levels 2 and 1 were not indicated for this needs assessment.
Patient and public involvement
Patients and the public were not included in the study.
Results
Levels 7 and 6
Level 7, population health, was assessed for the period of 1 April 2017–31 March 2018. Nationally during this period, hospitals provided SEP-1 compliant care 51% of the time.22 The level 6, patient health, assessment included statewide hospital compliance ranging 15%–91%, with the study institution’s compliance at 19%.22
Level 5: retrospective chart review findings
From the stratified sample of 113 patients, 57 patients met the criteria for severe sepsis and septic shock. Six of these patients were excluded from the study based on exclusion criteria (table 3). Of the 51 patients who met criteria, 34 (66.7%) patients had severe sepsis and 17 (33.3%) patients had septic shock. The majority (48, 94%) of the patients were first identified as having severe sepsis/septic shock in the emergency department. Other areas where patients first developed severe sepsis/septic shock included the interventional radiology suite (1, 2%), the medical unit (1, 2%) and the post-anaesthesia care unit (1, 2%).
The 51 charts were analysed to determine the most missed SEP-1 bundle elements (table 4). The most frequently missed elements include: the initiation and administration of a targeted volume of intravenous fluids within a specified period (compliance of 0% in severe sepsis and 58.8% in septic shock), repeat lactate level collection (compliance of 52.6% in severe sepsis and 60% in septic shock), documentation of repeat volume and tissue perfusion assessment in septic shock (compliance of 58.8%), vasopressor administration (compliance of 73.3%) and the administration of broad-spectrum antibiotics in severe sepsis (compliance of 76.5%).
SEP-1 measure compliance
Levels 3 and 4: focus group findings
The focus group occurred during a regularly scheduled QI Committee meeting. Due to the pandemic, the format of the meeting was a blend of in-person and virtual. Fifteen participants were present in the auditorium and 25 participants attended via video conference. The focus group included registered nurses, APPs, medical residents, and members from the QI Committee that includes members of the Quality Department and physicians from various departments including medicine, cardiology, endocrine, nephrology, neurology, infectious disease, and emergency medicine.
The presentation and discussion for the focus group was facilitated by one author (TD). The project and goals of the focus group were described with the use of the prepared PowerPoint presentation. The participants were asked to recall their most recent observations regarding the care of a patient with severe sepsis or septic shock that was provided by a colleague, a medical resident or APP. The question, ‘In the context of competence and knowledge, what types of gaps in performance have you noted?’ prompted discussion that filled the allotted time frame of 30 min.
Declarative knowledge
The focus group discussion included comments and clarifying questions regarding the measures of the SEP-1 bundle and the ordering process within the EHR. The discussion began with the topic of the required lactate level and included the time frame requirement for the repeat level and the process for a reflex lactate order.
Discussion also elicited a deficiency in declarative knowledge among the participants related to the use of ideal body weight versus actual weight when determining the intravenous fluid (IVF) resuscitation for those patients whose body mass index (BMI) is >30. To use the ideal body weight, the ordering provider must document that the fluid calculation is based on the BMI. Some of the participants appeared to be unaware of this caveat for IVF administration and of the EHR smart phrase that was created several months prior to the focus group to assist providers in meeting this requirement.
Some of the participants were also unclear about the necessary documentation requirements for the repeat volume and tissue perfusion assessment. The clarifying questions about data to include in the physical examination and optional assessments such as point-of-care ultrasound and passive leg raise were interpreted to indicate insufficient declarative knowledge in this area. Additionally, some participants verbalised lack of knowledge concerning the smart text phrase within the EHR to meet compliance with this measure.
Dispositional knowledge
Participants discussed the sepsis order set and why it is not often used. A few of the participants reported that when they admit a patient from the emergency department, the order set has a large amount of repetition from orders already inputted in the emergency department. These participants report that due to the redundancy, they choose not to use the order set.
Discussion
This needs assessment, organised by a conceptual outcomes framework, has identified an opportunity for improvement related to the care of patients with severe sepsis and septic shock. Starting with the end in mind, the low SEP-1 bundle compliance at the institutional level supported the need for a gap analysis at other levels. Using the conceptual outcomes framework, the authors developed a strategy to assess performance gaps and once found, explore them further to understand their root causes. Guided by the framework, the authors suggest intervention strategies to address the gaps noted at the studied institution (table 5).
Needs assessment findings and suggested interventions by framework level*7
Level 7
The national SEP-1 compliance score of 51% at level 7, population health status, highlights the gap in providing evidence-based care to those patients with severe sepsis and septic shock. Initiatives across numerous institutions are needed to improve compliance and impact outcomes at the population level. Initiatives of payers, such as the CMS, strive to address these gaps. The gaps at this level are beyond the scope of this project.
Level 6
At level 6, patient health status, the institutional compliance score of 19% demonstrates a noteworthy gap in performance and evident need for improvement. The outcomes at level 6 can be improved by interventions at levels 3–5, so the level 6 gap was evaluated with a detailed needs assessment at these levels.
Level 5
Level 5, clinician performance, was assessed with a retrospective chart review examining the individual components of the SEP-1 bundle. The review identified the most frequently missed SEP-1 measures and the provider specialties associated with those missed measures. Potential reasons for the missed measure were explored and the patient’s location at the time of the missed measure gave insight regarding the medical specialty responsible for the patient at the time the measure was missed.
The chart review noted that 94% of patients with severe sepsis or septic shock are present in the emergency department at time zero and during the initial period of their sepsis care. The emergency department providers are responsible for meeting at least a portion of the SEP-1 bundle requirements for many of these patients. When the patients transfer out of the emergency department before the bundle components are completed, these patients with severe sepsis or septic shock are most often transferred to the care of the hospitalist and intensive care unit teams who then become responsible for ensuring that the rest of the SEP-1 bundle requirements are met. Educational interventions should include providers from each of these departments.
Levels 4 and 3
The focus group was used to explain the performance gaps noted in level 5 by exploring provider perceptions of level 4, competence, and level 3, knowledge. The focus group did not comment on competence; therefore, it is unclear if a gap in competence exists. Provider competence is integral to successful performance. Although no gap in competence was identified during the focus group, one may be present. Additional data collection evaluating competence is warranted.
Educational planners can gather supplementary data about provider competence during predisposing activities. Simulation activities that mirror the workplace environment provide an excellent opportunity to demonstrate the integration of declarative, procedural and dispositional knowledge. Simulation activities should include predisposing activities to assess the integration of knowledge, enabling activities to establish and extend competence, and reinforcing activities that occur afterwards, fortifying knowledge and competence.
The focus group identified gaps in declarative knowledge and dispositional knowledge. The sole gap in dispositional knowledge was noted when participants shared their disregard for using order sets due to order repetition. Educational meetings with predisposing, enabling and reinforcing activities can address dispositional knowledge. Predisposing activities may include sharing examples of the impact of sepsis on patients and how the sepsis bundles improve patient care. Aggregate data demonstrating performance gaps can emphasise the need to improve performance and create teachable moments. Presentations and worked examples can reinforce the SEP-1 bundle and follow-up newsletters sharing stories, outcomes and bundle compliance data can reinforce dispositional learning.
The focus group discussion shed light on gaps in declarative knowledge. These gaps include uncertainty regarding the time frame for certain requirements to be completed and the processes that have been put in place to assist providers in meeting the SEP-1 criteria. Predisposing activities such as a pretest on the SEP-1 components can get learners think about sepsis and prepare them to learn about sepsis. A pretest may also create a teachable moment through cognitive dissonance. Enabling activities such as SEP-1 bundle explanation and representative case studies can also enhance declarative knowledge. These activities should be followed up with reinforcing activities such as an email to participants that summarises the key points and provides additional educational resources.
The focus group did not identify gaps in procedural knowledge; therefore, it is unclear if a gap exists. Additional needs assessment data collection regarding procedural knowledge should be obtained prior to educational workshops. These workshops can include predisposing activities such as an email reviewing material from prior meetings and new material to be covered during the workshop and a pretest on procedural knowledge. Presentations can illustrate performance gaps during enabling activities and worked examples and deliberate practice can model and solidify skills developed during the educational workshops.19
Levels 2 and 1
Level 2, satisfaction, and level 1, participation, of the conceptual outcomes framework were not assessed with this needs assessment. The authors recognise that for learning to occur, the educational offerings must be convenient for the participants and held in comfortable environments. Additionally, the workshops should be well organised and tailored to the participants’ educational needs. Participant recruitment for SEP-1 education and training can occur through announcements at other educational offerings and via email notifications that highlight performance gaps. Assessments of satisfaction and participation should occur with each learning activity.
Limitations
The data included in this needs assessment were collected prior to the COVID-19 pandemic. Due to the delay in completing the needs assessment, this manuscript contains older data. Despite this limitation, many obstacles to SEP-1 compliance continue and the information remains relevant to the institution.
The focus group did not proceed as anticipated in several ways. First, technical issues affected the video and auditory recording; therefore, the content analysis of the session was based on the recall from two authors. In addition, there were a small number of novices who participated in the discussion regarding the use of order sets. The intent of the focus group was to gather perceptions about the gaps in compliance and knowledge that explained the level 5 findings. This was intended to be an evaluation by experienced and proficient attending physicians assessing the less experienced residents and APPs rather than first-hand reports of experiences and decision-making among the novice physicians. Despite this limitation, the data gathered from the novices’ first-hand experiences are pertinent in identifying areas for improvement. Lastly, the focus group also revealed that even those experienced providers working in an educational and supervisory capacity are unclear of the details of the SEP-1 bundle requirements and the institutional efforts to improve compliance. This gap in knowledge underscores the need for further education for all levels of providers.
This educational needs assessment uses a framework to assess learning needs and suggest interventions to improve provider knowledge and competence, levels 3 and 4 of the framework. Improved knowledge and competence are expected to lead to an improvement in clinician performance, level 5, and ultimately to impact SEP-1 compliance. Education is a starting point for this project and will need to reoccur as new providers are hired. The effectiveness of education on SEP-1 compliance can be assessed using a Plan–Do–Study–Act cycle.
Comprehensive QI processes to improve SEP-1 compliance are multifaceted and include structural and EHR interventions such as point-of-care reminders and prompts. A discussion of systems improvements is beyond the scope of this paper.
Conclusion
This needs assessment used a conceptual outcomes framework to evaluate gaps in SEP-1 compliance at a single institution in the northeastern USA. Using backwards planning and starting with the end in mind, gaps were identified at several levels of the framework. Next steps include educational offerings that should include a series of meetings, activities, and workshops that address declarative knowledge, procedural knowledge and dispositional knowledge related to the SEP-1 bundle requirements. Ultimately, providers can develop and demonstrate their SEP-1 competence, that is, the integration of declarative, dispositional and procedural knowledge, during simulation activities. Once competence is achieved, point-of-care prompts and performance measurement and feedback of patient care data can act as reminders to support clinician performance. This needs assessment underscores the need for a multifaceted educational approach to improve clinician performance and SEP-1 compliance.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
Institutional and university review boards determined that the chart review part of the needs assessment was exempt. The second phase of the needs assessment, the focus group, did not require institutional review board approval as it was part of a quality assessment and improvement activity.
References
Footnotes
Contributors All authors contributed to this article. TD was responsible for the overall content as the guarantor. TJVH was a primary advisor for the study design and primary reader. JE and MD were content advisors and readers.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Disclaimer The views expressed in the submitted article are the authors' own and not an official position of the associated institutions.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.