Article Text
PDF
Statistics from Altmetric.com
Narrative
Background
Engagement in quality improvement (QI) work is essential to ensuring emergency departments (EDs) are meeting the needs of all patient populations. In the USA, children represent approximately 27% (35 million) of all ED visits.1 Over 50% (18 million) of these children are seen in EDs that see fewer than 10 000 paediatric patients per year, with only 7% of children seen in extremely high-volume EDs (>50 000 paediatric patients per year).1 Among the 5000 EDs in the USA, 39% see fewer than 5 children per day, 30% see 5–14 children per day and 17% see 15–25 children per day.2 In the absence of repetitive performance and measurement, as seen in high-volume centres (greater than 10 000 paediatric visits annually), it may be difficult to assess the quality of care processes.3 In the USA, less than 50% of EDs report the inclusion of paediatric-specific elements in a department-wide QI plan.2 The aim of this manuscript is to identify a series of paediatric emergency care quality measures that would be valuable for low-volume EDs (<10 000 paediatric patients per year).
The National Paediatric Readiness Project applies a systems approach (care coordination, QI, policies and procedures, staff competencies, patient safety and equipment and supplies) to ensuring high-quality emergency care for children among diverse EDs.2 Paediatric readiness, as determined by the National Paediatric Readiness Assessment using a weighted 100-point scale, is associated with decreased paediatric mortality among critically ill and injured children.2 4–10 Facilities that incorporate paediatric-specific QI initiatives demonstrate a 26-point increase in their paediatric readiness score.11 Engagement in paediatric readiness efforts is high, yet integration of paediatric QI efforts in EDs is lagging.2
While over 400 paediatric emergency care performance measures have been proposed and prioritised, widespread uptake has been limited.2 12–14 Infrequent paediatric patient encounters make it difficult to assess the cause and effect of care processes. Furthermore, process and outcome measures within the paediatric emergency care landscape often require complex data extraction which may be especially difficult to implement in lower-resourced facilities.12–14 To date, no comprehensive assessment of the quality of paediatric emergency care delivery exists in low-volume EDs (<10 000 visits/year). The National Pediatric Readiness Quality Initiative (NPRQI) focuses on the undifferentiated patient experience that encompasses standard phases of care in the ED: assessment, interventions, diagnostics and disposition. We describe the development of a core set of NPRQI quality measures targeting common paediatric patient reports for adoption in low-volume EDs.
Methods
NPRQI conducted a five-phase modified Delphi process from November 2019 through January 2021.15 The study consisted of two content arms. Arm 1 included the cross-cutting clinical assessment and four clinical reports. Arm 2, which focused on behavioural health, was added 6 months into the study due to the evolving behavioural health crisis among children. The Delphi process included an initial meeting (Arm 1—in-person, synchronous; Arm 2—virtual, synchronous), a confidential online survey (Arm 1, Arm 2) and subsequent virtual meetings (Arm 1, Arm 2, synchronous). Both arms followed the same series of steps and were only separated by time.
Consensus panel
The panel consisted of 41 members who were either identified by their respective national professional society as a content expert or were selected based on the following criteria: expertise in paediatric emergency care applied research, emergency medical services for children, QI, QI data registries, specific areas of clinical practice, clinical practice setting, healthcare system networks, regulatory agencies and federal partners (table 1, online supplemental appendix A).
Supplemental material
Characteristics of consensus panel
Phase 1—determination of measures for consideration
The study team (KR, EE, KB, LG) and a subset of research panellists (ARM 1: AJ, CM, HH, LA, RS, BM; ARM 2: TC, SD) collated existing paediatric emergency care quality measures,10–12 specialty-specific professional guidelines, consensus statements and evidence-based reviews to identify measures for the ED setting. A total of five clinical reports (‘domains’) were identified based on the prevalence of clinical reports among ED visits, availability of evidence-based guidelines, alignment with national priorities and presence of validated screening tools.16 17 The five clinical domains selected for inclusion were: blunt head trauma, respiratory reports, seizures, behavioural health and vomiting. Cross-cutting measures for clinical assessment and interfacility transfer were included to capture foundational processes of care delivery, independent of the clinical report. Additional clinical domains were excluded due to scope. All proposed measures were characterised by clinical domain, the six domains of quality, phase of ED care (assessment, interventions, diagnostics, disposition) and measure type (process or outcome).18–20 Structural measures were excluded as they are the focus of the National Paediatric Readiness Assessment.5 Structural measures for behavioural health, proposed by the Emergency Medicine Quality Improvement Collaborative for Kids behavioural health consortium, are included in online supplemental appendix B for future consideration and consensus building; however, they were deemed outside of scope for NPRQI. This initial set of process and outcome measures were presented to the larger consensus panel for review and approval. Measures with a vote of greater than 50% were included in an online survey instrument (Research Electronic Data Capture; Vanderbilt University).
Supplemental material
Phase 2—evaluation of measures
The consensus panel was charged with rating each measure based on the National Quality Forum (NQF) Measure Evaluation Criteria: feasible for data collection in a low-volume, low-resourced ED setting, usable to an ED care team, important for patient-centred outcomes and scientifically acceptable.21 The goal was to identify fewer than six measures per clinical domain (assessment, interfacility transfer, clinical reports and behavioural health). Arm 1 added three additional stakeholders (AR, JL, KS) from low-volume facilities to assess feasibility, usability and importance of the measures in the ED setting. Each member completed a confidential, online survey rating each measure using the NQF Criteria on a 3-point Likert scale.21 Aggregate responses were compiled and the mean score was calculated for each measure (online supplemental appendix C). Measures with a mean score of greater than 2 or greater than 1.8 for behavioural health, were included in Phase 3.
Supplemental material
Phase 3—panel feedback: usability and importance
On completion of the survey, the panel was reconvened to share aggregate results: group mean and SDs for each measure (Arm 1 in-person; Arm 2 virtual). Panellists participated in open discussions regarding usability and importance. An importance statement was developed for each of the measures as further validation of usability and importance. Panellists had an opportunity to champion the inclusion of a measure or provide justification for exclusion. Those measures that did not achieve consensus (typically less than 80%) were excluded.
Phase 4—measure feasibility assessment
The feasibility of measures was determined based on viability of data extraction, accessibility of variables for each measure and complexity to implement. For both arms, the NPRQI leadership team (KR, EE, KB, LG) conducted a feasibility assessment based on whether discrete variables existed in emergency medical records for proposed measures and for Arm 2, whether the information would be accessible due to behavioural health privacy concerns and if discrete variables existed. Feedback was obtained from a subset of research panellists (AJ, BM, CM, HH, LA, RS, SD, TC) with respect to feasibility. Measures were presented to the consensus panel (virtual) and refined for clarity of interpretation and feasibility.
Phase 5—final review
The final list of measures was reviewed in totality with the panel (virtual) to review the final list of measures and importance statements, and ensure consensus. Each measure was further categorised by its Donabedian classification (process or outcome measure) and according to the phase of care in the ED (assessment, diagnostics, interventions and disposition).19
Results
The breadth of measures considered reflected NPRQI’s approach to identify cross-cutting care processes and common ED reports among undifferentiated paediatric patients. The seven clinical domains included two cross-cutting domains (recognition of the sick or injured child (8 measures) and effective transfer (4 measures)) and five clinical domains (blunt head trauma (10 measures), seizures (11 measures), respiratory reports (12 measures), vomiting (8 measures) and behavioural health (17 measures). Based on these clinical domains, 70 total measures were proposed for inclusion in NPRQI.
A total of 65 measures (53 for Arm 1 and 12 for Arm 2) were included in Phase 1 of the modified Delphi process (online supplemental appendix B). All measures were categorised according to the four phases of ED care: assessment, interventions, diagnostics and disposition. Behavioural health measures were categorised as follows: assessment (7 measures, 58.3%), interventions (1 measure, 8.3%), diagnostics (1 measure, 8.3%) and disposition (3 measures, 25%). The remaining clinical domain measures were categorised as follows: assessment (14 measures, 26.4%), interventions (19 measures, 35.8%), diagnostics (11 measures, 20.8%) and disposition (9 measures, 17%). In Arm 1, six measures scored 2.0 or less and were excluded from subsequent phases. In Arm 2, six measures scored 1.8 or less and were eliminated from subsequent phases. Phase 3 and 4 panel discussions resulted in the exclusion of 18 and 5 additional measures, respectively, from Arm 1 and the exclusion of 0 and 2 additional measures, respectively, from Arm 2. Measure exclusion during phases 3 and 4 was due to: lack of a validated screening tool (eg, human trafficking screening), lack of clarity in interpretation (eg, intubation following head trauma), outside locus of control for ED implementation (ED length of stay for mental health patients), redundancy with cross-cutting care process measures and/or complexity of data collection (table 2). Measures excluded based on complexity or feasibility to implement included: percentage of patients with abnormal vital signs included in provider notification process, percentage of families who received transfer packets, percentage of CTs that used appropriate weight-based dosing, percentage of patients with return precautions, medication dosing errors and behavioural health follow-up after discharge.
Modified Delphi process for measures selection
A final review by a subset of research panellists (AJ, BM, CM, HH, LA, RS, SD, TC) confirmed the 28 final quality measures for inclusion in NPRQI: cross-cutting care processes (8 measures, 28.6%), blunt head trauma (4 measures), seizures (3 measures), respiratory reports (6 measures), vomiting (3 measures) and behavioural health (4 measures) (table 3). All were classified as process measures. By phase of care in the ED the measures are characterised as follows: assessment (8 measures, 28.6%), interventions (13 measures, 46.4%), diagnostics (3 measures, 10.7%) and disposition (4 measures, 14.3%).
National Pediatric Readiness Quality Initiative quality measures
Discussion
Twenty-eight measures that capture five common clinical presentations and two cross-cutting processes of care were developed by the NPRQI with the participation of a diverse panel of experts and national organisations. These measures encompass the four phases of care in the ED: assessment, diagnostics, intervention and disposition. This approach allows for a prospective, feasible and patient-centred focus to paediatric-specific QI efforts. Low-volume EDs may never have sufficient patient encounters to focus on final diagnoses for timely QI efforts. By targeting critical cross-cutting processes and common clinical presentations, EDs can assess quality of care among a larger undifferentiated paediatric population.18 The patient experience model, as highlighted within the Institute for Healthcare Improvement’s Triple Aim, is based on the care provided in response to a clinical presentation, not a specific diagnosis.22 The measures within each clinical domain were derived from evidence-based research that links early and appropriate assessment, targeted diagnostics, timely interventions and disposition to improved outcomes.
The NPRQI represents a new framework for the inclusion of quality measures for all categories of paediatric care divided across four phases of care: assessment, diagnostics, interventions and disposition. Assessment serves as the foundation for paediatric patient safety. For example, early recognition of altered mental status is essential to timely management of head trauma.23–25 Early recognition of tachycardia and hypotension are core components of sepsis recognition.26 In turn, early recognition of the critically ill or injured child leads to timely administration of evidence-based interventions linked to improved outcomes. For example, early administration of steroids are associated with decreased hospitalisation among children with moderate and severe asthma and, among children with seizures, treatment delays can result in prolonged seizure activity.27 28 In addition to improved outcomes, timeliness of interventions improves patient experience.29 Diagnostic testing should be undertaken judiciously and in conjunction with family-centred care. Unnecessary exposure to radiation in the paediatric patient is both costly and associated with potential harm.30 31 Invasive procedures and unnecessary testing, too, can adversely impact patient experience and cost.32 Adherence to standardised, site-specific transfer criteria and processes promotes timely access to necessary resources and may minimise unnecessary cost to patients and families.33 A first step to adhering to evidence-based guidelines in specific populations is optimising care processes across all phases of ED care.
The past decade has fostered significant growth in the development of evidence-based guidelines as research in paediatric emergency care has shifted from single-centre data to multicentre studies enabling researchers to address low frequency, high-risk conditions in a more systematic manner. Multicentre research has also allowed for creation of evidence-based guidelines and validated decision rules for common paediatric reports. This approach is exemplified by the Emergency Medical Services for Children-funded Paediatric Emergency Care Applied Research Network (PECARN), which has published over 150 articles on a diverse range of paediatric emergency medicine topics.17 While much of the research from multicentre trials focuses on high risk, low-frequency events such as sepsis, some of the most prominent guidelines generated from PECARN relate to common clinical conditions encountered at virtually every ED in the USA: management of closed head injury in children, bronchiolitis management, pain management, suicidality and paediatric patient safety considerations.17 31 34–36
The NPRQI measures individually may overlap with the development of paediatric measures by others.12–14 What differs is the context in which they are implemented which focuses on clinical presentation rather than a specific diagnosis, and that they are imbedded in a set of measures that capture all ED phases of care for managing the patient, not the diagnosis. This approach can be applied across low-volume EDs where specific diagnoses are infrequent, but the phases of ED care are universal. The large proportion of NPRQI measures (28.6%) that fall within cross-cutting care processes further supports engagement by very-low volume EDs (fewer than five paediatric patients per day). These measures set the foundation for measuring the quality of ED care provided for most children in the USA. In the absence of pay-for-performance incentives, the adoption of paediatric-specific quality measures in low-volume EDs (where most children seek care) will depend on the ease of data collection, relevance to a large proportion of the paediatric population and linkage to patient-centred outcomes. These measures strive to be feasible at the local level and actionable for the ED care team.
Families rely on EDs of close proximity to meet the needs of critically ill and injured children, most of which see fewer than 15 children per day.1 37 The potential impact of quality measures on care delivery is dependent on relevance to the population served, and uptake by those who can implement change. Engagement of frontline practitioners in QI efforts is essential to ensuring the success of improvement strategies. The measures proposed herein are a first step to recognising variability in care within a single ED as the focus is optimisation of processes of care that are central to the delivery of high-quality care for all paediatric patients. The potential impact of the 28 NPRQI quality measures is that any ED can immediately improve processes of care that are linked to improved health outcomes in children.
The NPRQI measures provide a foundation for any ED care team to measure adoption of evidence-based guidelines for paediatric care using a patient-centred, provider-driven approach to QI. Unlike many quality measures that rely on administrative data or a diagnosis-based retrospective review, the NPRQI measures were designed for any ED to assess performance and improve delivery of care to the undifferentiated paediatric patient. The NPRQI paediatric quality measures may be biased from using a consensus model. It was critical to engage a diverse group to participate in the modified Delphi process as well as create an environment for sharing differing perspectives. The subset of research panellists engaged in measures proposal and final review were chosen for their clinical, research and implementation expertise to ensure feasibility. The consensus panel was composed of multidisciplinary organisational representatives that intersect with low-volume EDs. With significant input from consensus panellists and national organisations, the NPRQI measures are relevant, feasible and linked to improved outcomes for paediatric patients with common clinical presentations.
Conclusion
The majority of paediatric emergency care is sought in diverse, low-volume ED settings with variable capacity and capabilities and where the majority of patients are adults. We worked to identify relevant, feasible, patient-centred and process-focused paediatric quality measures for adoption in any ED, especially low-volume settings. The quality measures proposed herein were developed as part of the NPRQI, the operational arm of the National Paediatric Readiness Project.4
The NPRQI is an effort to optimise emergency care delivery for children regardless of site-specific resources or infrastructure. The development of relevant and applicable paediatric quality measures is the first step in implementation of paediatric QI processes. The presence of QI processes that includes paediatric-specific measures is associated with improved paediatric readiness and increased survival from critical illness and injury. The NPRQI framework differs from previous efforts in the following ways: (1) diversity and national representation of the selected consensus panel, (2) focus on the undifferentiated patient, (3) adherence to the perspective of healthcare providers in rural and community settings and (4) prioritisation of easily accessible data points. This is the first set of paediatric emergency care quality measures that supports implementation in low volume, low-resourced EDs; provider-driven improvement strategies; and encompasses all four phases of care in the ED setting. The NPRQI measures will be used to create a national open access, electronic paediatric emergency care QI data platform and registry to facilitate engagement in QI efforts, support ED providers to assess the current state of paediatric emergency care delivery and benchmark performance across similar ED settings. Future efforts will focus on establishing performance benchmarks across variably resourced EDs, geographical regions and patient demographic groups to begin to identify and address disparities in the ability of our nation’s emergency care system to meet the needs of all children.
Ethics statements
Patient consent for publication
Ethics approval
The NPRQI was reviewed by the University of Texas at Austin Institutional Review Board and deemed exempt as defined under 45 CFR 46.102(d).
Acknowledgments
The authors wish to acknowledge the NPRQI Quality Improvement and Analytics Advisory Board for their time and effort participating in the consensus process.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors KER, KAB and EAE conceived and designed the study. KER, KAB, LG, KSM and EAE provided study supervision. KER and EAE obtained funding. KER, KAB, LG, KSM and EAE drafted the manuscript and all authors contributed substantially to its revision. KER, KAB, LG, KSM and EAE take responsibility for the paper as a whole.
Funding The NPRQI is supported by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS) as part of an award (H34MC33244) totaling US$1.2 million with 0% financed with non-governmental sources. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by HRSA, HHS or the US Government.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.