Evaluating a novel, integrative dashboard for health professionals’ performance in managing deteriorating patients: a quality improvement project
===================================================================================================================================================

* Baneen Alhmoud
* Daniel Melley
* Nadeem Khan
* Timothy Bonicci
* Riyaz Patel
* Amitava Banerjee

## Abstract

**Background** The quality of recording and documentation of deteriorating patient management by health professionals has been challenged during the COVID-19 pandemic. Non-adherence to escalation and documentation guidelines increases risk of serious adverse events. Electronic health record (EHR)-integrated dashboards are auditing tools of patients’ status and clinicians’ performance, but neither the views nor the performance of health professionals have been assessed, relating to management of deteriorating patients.

**Objective** To develop and evaluate a real-time dashboard of deteriorating patients’ assessment, referral and therapy.

**Settings** Five academic hospitals in the largest National Health Service (NHS) trust in the UK (Barts Health NHS Trust).

**Intervention** The dashboard was developed from EHR data to investigate patients with National Early Warning Score (NEWS2)>5, assessment, and escalation of deteriorating patients. We adopted the Plan, Do, Study, Act model and Standards for Quality Improvement Reporting Excellence framework to evaluate the dashboard.

**Design** Mixed methods: (1) virtual, face-to-face, interviews and (2) retrospective descriptive EHR data analysis.

**Results** We interviewed three nurses (two quality and safety and one informatics specialists). Participants perceived the dashboard as a facilitator for auditing NEWS2 recording and escalation of care to improve practice; (2) there is a need for guiding clinicians and adjusting data sources and metrics to enhance the functionality and usability. Data analysis (2019–2022) showed: (1) NEWS2 recording has gradually improved (May 2021–April 2022) from 64% to 83%;(2) referral and assessment completion increased (n: 170–6800 and 23–540, respectively).

**Conclusion** The dashboard is an effective real-time data-driven method for improving the quality of managing deteriorating patients. Integrating health systems, a wider analysis NEWS2 and escalation of care metrics, and clinicians’ learning digital solutions will enhance functionality and experience to boost its value. There is a need to examine the generalisability of the dashboard through further validation and quality improvement studies.

*   Clinical Audit
*   COVID-19
*   Standards of care
*   Quality improvement
*   Critical care

### WHAT IS ALREADY KNOWN ON THIS TOPIC

*   The COVID-19 pandemic significantly impacted clinicians’ adherence to routine practice in managing deteriorating patients.

*   Real-time dashboards are advanced auditing tools used for monitoring and improving healthcare workflow, yet not been used or studied in improving the escalation of care practice.

#### WHAT THIS STUDY ADDS

*   Real-time auditing can be an effective and efficient method for enhancing the quality of managing deteriorating patients in healthcare settings.

#### HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

*   The study informs decision-makers of the need for using advanced data-driven digital methods to improve the quality of managing critically ill patients.

*   The study recommends incorporating clinicians’ feedback in the development and evaluation of auditing dashboards and expanding the implementation in other hospital settings for further evaluation and improving deteriorating patient care.

## Introduction

The COVID-19 pandemic has taken its toll on healthcare services globally. The escalating pressure has significantly raised the surge in deteriorating patients and the need to escalate their care.1 There was an increase in daily tasks for nurses, physicians and rapid response teams to cope with the COVID-19 strain.1 Clinicians’ practice can be adversely affected by the increased patient to staff ratios,2 the complexity of patients’ clinical care,3 and the ongoing pandemic impact on the healthcare service and individual staff.4 The quality of clinicians’ assessment, documentation and timely referral for escalation can suffer.

Early warning scores (EWS) are widely implemented predictive tools to detect deterioration in an early stage of critical illness. Their performance has been variable,5 and their effectiveness is subject to multielement in the clinical settings.6 7 EWS performance in detecting critical events is not only related to the score’s sensitivity. Nurses’ adherence to recommended monitoring and escalation guidelines and physicians’ compliance with critical events and sepsis assessment may correlate with the outcomes studied in EWS validation.8 Common problems found in clinicians’ behaviour towards EWS protocol include non-compliance with recommended monitoring frequency, notification of doctors when indicated by EWS, or timely response of doctors and CCRT.8 9 Therefore, serious adverse events occur due to misclassification of patients and poor allocation to critical care despite the implementation of EWS and escalation guidelines. Along with established implementation and validated performance of EWS, human factors are vital for the success of EWS application.

Real-time auditing can be an effective method to detect the roots of clinicians’ adherence defects. With the availability of electronic health records (EHR) systems, a representative, generalisable dataset can be captured and analysed at scale via integrated EHR dashboards in a constructed, organised form.10 Healthcare dashboard is an electronic analytics tool to monitor healthcare key performance indicators by displaying outcomes, auditing progress, identifying deficiencies, and manage professional and clinical activities in healthcare organisations.11–13 Digital dashboards systems capture EHR data to generate information on the healthcare system, individual professionals’ performance and the patient’s prognostic status. Prompt, concise and context-specific display of the performance provides analysis of hospital and patients status, facilitating clinical decision-making and quality improvement.14 15 For example, the National Health Service (NHS) Pathways of Coronavirus Triages and Activity dashboards in NHS hospitals are examples of live data visualisation for the public information at an organisational level.16 Although dashboards have proven efficiency in providing real-time information for hospital management and stakeholders,14 17 18 functionality is limited when addressing performance issues from patient chart data. Logging into each patient’s chart several times during the day for specific information is time consuming and problematic, that is, completion of EWS recording in a day shift. Healthcare dashboards are concise, time-saving and intuitive tools for up-to-date assessment and escalation auditing of deteriorating patients.

### Problem

Despite the widespread implementation and expenditure of EHRs in healthcare settings, secondary use of data for improving quality and safety is limited. The full potential benefits from EHRs data are far from realisation currently despite massive efforts of investment in health technology.19 Challenges, such as the conflicts between public interest, individual patient safety, and optimising application of health information systems, have restricted the potentials of data use.20 21 With the growing integration of EWS into EHR, a significant amount of data for critically ill patients are available,22 but not yet used.

Furthermore, COVID-19 has affected the quality of clinicians’ routine practice and hindered tasks that could be regularly and efficiently carried out prior to the pandemic.23 As a result, appropriate, timely management of acutely ill patients declined.24 25 Escalating care of an acutely ill patient must follow timely and careful assessment, then communicating the evaluation to the designated critical care professional for further intervention. Errors in detecting worsening of the condition and failure to communicate or intervene can hamper escalation of care and negatively impact clinical outcomes and National Early Warning Score (NEWS2) performance. Non-compliance with escalation protocols or recommended documentation guidelines may result in serious healthcare errors.26 The Record Management Code Practice 2021 provides a framework to guide organisational and individual responsibilities when managing patient records.26 Auditing is an integral part of healthcare records policy and guidelines to assess the standard achieved in records and find areas needing improvement for health data and staff.27

In Barts Health NHS Trust, the National Early Warning Score (NEWS2) has been nationally endorsed and implemented.28 A digital transformation took place in Barts hospitals by shifting NEWS2 recording into digital format and automating routine monitoring, hoping to increase accuracy of information if the escalation protocol is optimally followed. However, implementing digital NEWS2 requires complete and updated her, which has been more difficult in the COVID-19 context at Barts and other hospitals. Local audits in the trust showed non-adherence to routine monitoring and recording, for example, in 2019–2020 NEWS2 status was incomplete and not in line with guidelines in approximately a quarter of the patients’ population (complete vitals: 61%–79%, 75%–83%, 84%–87%, 83%–91% and 79%–86% from September 2019 to January 2020 in Newham, Royal London, St Bartholomew’s, and Whipps Cross hospitals, respectively) (online supplemental appendix 1).

### Supplementary data

[[bmjoq-2022-002033supp001.pdf]](pending:yes)

A dashboard integrated in EHR would allow performance of health professionals, data quality patient care to be monitored and improved by facilitating timely health resources management and support informed clinical decision-making. We therefore conducted a quality improvement study to evaluate deteriorating patient dashboard and provide evidence for an exemplary quality of care for other healthcare settings.

### Aims

To create and evaluate an EHR data-driven dashboard of real-time assessment of deteriorating patients and escalation of care. We aim to evaluate the dashboard through PDSA cycles, including:

1.  Examine the views on the dashboard and areas that need improvement from key users’ perception; and implement necessary actions for development.

2.  Evaluate the performance of nurses and physicians in the stages of managing deteriorating patients on all trust levels through historical tracking of data.

## Methods

### Context

The dashboard was implemented in the largest NHS trust in the UK (Barts Health NHS Trust) in five academic hospitals: Mile End Hospital, Newham University Hospital, Royal London Hospital, Whipps Cross University Hospital and St Bartholomew’s Hospital.

The dashboard was developed by creating Vitals data table, then transformed into a thorough and more robust data visualisation of NEWS2, assessment and escalation of deteriorating patients via Qlik Sins. Development was led by DM and executed by NK. Data of around 1.2 million recordings of 110 000 admissions from August to October 2020 were extracted from EHR (Cerner Millennium) (figure 1 and online supplemental appendix 2).

![Figure 1](http://bmjopenquality.bmj.com/https://bmjopenquality.bmj.com/content/bmjqir/11/4/e002033/F1.medium.gif)

[Figure 1](http://bmjopenquality.bmj.com/content/11/4/e002033/F1)

Figure 1 
Illustration of deteriorating patient’s dashboard development from patients electronic health record (EHR) data.

The user interface includes live and accumulative data of patients with high NEWS2, and performance tracking of stages in deteriorating patients’ management by nurses and physicians on all trust levels. Health professionals’ performance is measured by the completion of the assessment, escalation of care and sepsis treatment (figure 2).

![Figure 2](http://bmjopenquality.bmj.com/https://bmjopenquality.bmj.com/content/bmjqir/11/4/e002033/F2.medium.gif)

[Figure 2](http://bmjopenquality.bmj.com/content/11/4/e002033/F2)

Figure 2 
Escalation of care assessment flow chart. CCOT, Critical Care Response Team; SBAR, Situation-Background-Assessment-Recommendation tool for communication between healthcare team.

### Intervention

We adopted the Plan-Do-Study-Act (PDSA)29 model to examine the objectives of creating the dashboard and evaluating clinicians’ performance in the deteriorating patient management cycle. The PDSA model involves:

*   Plan: plan the test, intervention, or observation, including a method for collecting data.

*   Do: conduct the intervention on a small to a bigger scale.

*   Study: analyse the data and study the results.

*   Act: refine the change based on what was learnt from the best.

#### PDSA cycle 1

We planned to implement the dashboard and evaluate the performance from the perspective of key people in management who used it for auditing. The dashboard was initially launched in May 2021, and several improvements to the metrics and filters to serve the function until the version which we investigated in September 2021. The dashboard was introduced to ward managers, quality improvement and patient safety teams. The improvement plan was mapped out for the involved teams by informing them of the release of the dashboard and its objectives and functions. The guidelines for the utilisation of the dashboard were not developed fully yet; therefore, they were not disseminated to ward managers. By the end of this phase (October 2021), we interviewed three staff who took part in the initial roll-out: senior nurse for quality and safety, nursing informatics specialist, and patients safety practitioner. The information collected allowed us to create the next PDSA cycle to improve the effectiveness of the dashboard for a wider cohort of users.

#### PDSA cycle 2

In this phase, the Electronic Prescribing for Medicine Administration (EPMA) implementation has already taken place. A how to guide was developed to educate users on effectively making the most of the dashboard. Due to the EPMA’s role in handling drug prescribing data, data for this metric were missing from EHRs and have to be extracted from EPMA. After refining the dashboards and releasing the guidance, managers were encouraged to view and report information from the dashboard, and nurses and doctors were informed of areas in practice that needed adjustments. Data from the dashboard user interface were assessed to evaluate changes through time in deteriorating patients’ management. There was a plan to integrate EPMA data into the dashboard in the coming stage and provide further valuable data, that is, time of treatment prescribing.

### Measurements

We conducted individual interviews to evaluate the perception on the dashboard. Interview questions were created to gather qualitative data and adapted from a previous evaluation of dashboards of ward specific performance12 and aligned with the Technology Acceptance Model framework13 (figure 3). The key questions were on the perceived benefit, usability, the intention to use and the actual functionality, and desired adjustments needed to improve the dashboard (online supplemental appendix 2). A purposive sample of key users, as guided by DM, was interviewed by BA.

![Figure 3](http://bmjopenquality.bmj.com/https://bmjopenquality.bmj.com/content/bmjqir/11/4/e002033/F3.medium.gif)

[Figure 3](http://bmjopenquality.bmj.com/content/11/4/e002033/F3)

Figure 3 
Illustration of the Technology Acceptance Model (TAM) to examine the perception of the deterioration dashboard.

A before-and-after data evaluation was run to assess the change in performance in the measured metrics. The period was divided into five phases to interpret the improvement in recordings and forms completion. Phases were as follows: pre-EHR integration (August–November 2019), post-EHR integration (December 2019–September 2020), automation period (October 2020–April 2021), implementation period (May–September 2021), and post-feedback (October 2021–April 2022). EHR integration is the process of digitalising the calculation and scoring of NEWS2 from routine monitoring in EHRs without the need for nurses’ calculation or the use of paper format of NEWS2. Automation period is the phase of introducing automated monitoring, and implementation period when the dashboard was rolled out in the Barts trust. Due to the potential effect of skewing doctors’ assessment and treatment data caused by shifting entry from EHRs to EPMA, we compared the two metrics between the Dashboard period (May–August 2021) with post-EPMA rollout (September 2021–April 2022). Definitions of terms are in table 1.

View this table:
[Table 1](http://bmjopenquality.bmj.com/content/11/4/e002033/T1)

Table 1 
Definitions of terminologies

### Analysis

Interviews were audio taped and transcribed, and qualitative data were analysed using NVivo software following a content analysis approach by BA. The method is suitable for analysing interviews through systemic coding of transcriptions to indicate the presence of meaningful content related to the evaluated domains. Data analysis was iterative, where content was coded, grouped to form subcategories, and then into themes that represent the topics creating the focus of the evaluation. The transcripts and analysis were checked independently as developed by two researchers to ensure rigour.

Descriptive analysis was done on the data collected from the dashboard using the R programme. Pearson’s χ2 test was used to compare periods identified for NEWS2 recording and forms completion. *A* p value of <0.05 is considered statistically significant.

### Patient and public involvement

Patients were not involved in setting the research question, outcome measures nor in the design of the study. Patients were not involved in interpretation, writing up of the results. There is no plan for the results to be disseminated to the patient population affected.

## Results

Three participants were interviewed. Participants expressed their perceived advantages and usability of the dashboard for escalation of care, auditing NEWS2 recording and forms completion, and areas in need of improvement. The interviews data content formed two elements: (1) dashboard function, and (2) obstacles and improvement. From the descriptive data, we found a gradual improvement in NEWS2 and forms compliance by nurses and doctors

### Dashboard function

The dashboard is perceived was a tool for quality improvement. There was an agreed perception of its analytics function on individual nurses’ and doctors’ performance in the escalation of the care process. Primary auditors are ward managers, senior nurses and quality improvement officers. They could view periods and specific ward improvements and where the trend is declining; in patients’ status and clinician’s practice; to analyse the reasons for changes to planning for enhancement.

The dashboard helped nurses and doctors chase the escalation of deteriorating patients from the auditing function by monitoring NEWS2, SBAR referral and assessment completion to push for a better result.

Another benefit found was the attention to one’s performance due to being tracked in real time. It was reported that staff were becoming more interested in completing the forms and monitoring within the time frame and how properly the documentation in return to the observed numbers of deteriorating patients scoring high NEWS2.

### Obstacles and improvements

Participants reported issues in lack of engagement by managers in relation to difficulty in usability, such as locating and navigating it properly. It was suggested to improve in the dashboard function and utility. They recommended comprehensive showcasing of the dashboard for clinicians to understand the benefits and purpose. Participants expressed the need for a clear guide for utilisation to encourage clinicians make the most of the dashboard confidently. In addition, it was suggested for databases to be stored in a unified standard system to facilitate data extraction and query writing and explore the possibility of creating more functions from the health systems. Several additions were recommended by participants to enhance the role of the deterioration dashboard, including additional assessment metrics, sepsis diagnosis and treatment time, and monitoring wrist bands scanning for ID confirmation.

### Compliance measure

The audit showed poor compliance with vital signs and NEWS2 recording in the baseline period (64%), then improved gradually after the EHR integration of NEWS2 (81.5%), followed by an increase after automating vitals monitoring and dashboard implementation (85%–83%, respectively). Patients with high NEWS2 reached a peak between April and May 2020 and in January 2021 (~25%) when the first and second waves of COVID-19 occurred. Complete referral and nurse assessment forms were boosted after dashboard implementation (n: 170–6800 and 23–540, respectively). The screening and prescribing by doctors improved in the first dashboard phase (n: 22–36 and 15–26, respectively), then had a sudden drop (n: 8 and 6, respectively) after EPMA became the data entry point (table 2 and figure 4).

View this table:
[Table 2](http://bmjopenquality.bmj.com/content/11/4/e002033/T2)

Table 2 
Dashboard metrics trend measured in different phases

![Figure 4](http://bmjopenquality.bmj.com/https://bmjopenquality.bmj.com/content/bmjqir/11/4/e002033/F4.medium.gif)

[Figure 4](http://bmjopenquality.bmj.com/content/11/4/e002033/F4)

Figure 4 
Snapshot of front page of deteriorating patients’ dashboard from August 2019 to March 2022. SBAR: esclation of care handover tool (situation, background, assessment and recommendation); Sepsis screening forms: assessment of sepsis forms by nurses; Dr Assessment complete: sepsis screening by doctors; Sepsis 6: prescribing sepsis 6 bundle therapy. Colour codes: orange: upper and lower control limits and outliners; light blue: complete; dark blue: trends; green: central lines; graphs with red frame represent metrics that will be adjusted in the next stage.

## Discussion

We report the experience of development and implementation of an EHR-integrated dashboard for NEWS2 and escalation of care auditing in cardiac specialist teaching hospitals. This novel and unique initiative focused on monitoring and improving early recognition and expediting shared care by nurses and physicians. Our data visualisation is a practical and effective way for performance tracking; and, therefore, promoting timely NEWS2 recording and forms completion for improved escalation of care. We had three main findings. First, participants perceived the dashboard as an excellent auditing tool that improves deteriorating patients’ care. Second, some improvements are needed to enhance its functionality and user experience. Third, historical data have shown an increase in clinicians’ compliance with documentation and escalation protocol that may have responded to monitoring individual and unit performance and, therefore, driving quality improvement.

We found our results consistent with previous findings confirming the effectiveness of dashboard analytics in evaluating clinical performance.30 31 It has been shown that dashboards are visual tools to examine the programme or protocol’s effectiveness in meeting the objectives.16 The perceived individual staff monitoring and improving own actions came supporting to previously highlighted impact of dashboards on nurses’ awareness. Nurses’ attention to the ward’s accomplishments affected patients’ outcomes and gave them a sense of control and satisfaction with their achievements.13 17 31 Investigating the deteriorating patient management provides on-time tracking of escalation of care steps and NEWS2 recording to have a transparent understanding of its predictive performance in the care settings. Display and use of performance data are keys to identifying areas of strengths and weakness for quality improvement.

For a dashboard system to be widely used, it must be an easy, user-friendly and intuitive system. Our results indicate the need for clinicians to learn about digital tools’ usability, as the previous implementation of digital health systems showed the demand for embedding technology training and education.32 33 In addition, expanding the structure and functionality through integrating different data sources and refining the design is a substantial gain. Previous health dashboards where EHRs are combined with other systems, such as PACS (picture archiving and communications systems), have given professionals a broader view and knowledge of patients' health status.14 By integrating multiple digital systems, there is a potential for great use of information to understand, research and explain unclear data of one system by the other. The reduction in doctors’ assessment and Sepsis 6 was due to the data entry shift from EHRs to EPMA. In the trust, EPMA is believed to add the benefit of providing more true, timely treatment information for auditing. EPMA integration can provide data on prescribing antibiotics as per the recommended protocol, therefore, will be conducted in the next phase. Other data resources could also enhance and maximise the functionality, such as data of patients with COVID-19, resources tracking such as intensive care unit (ICU) beds and Critical Care Outreach (CCOT) staff, and timeliness of escalation of care steps data. Data sources addition could help doctors, nurses and managers organise the treatment plan promptly for clinicians’ and patients’ benefit.

From time-series analysis, we interpreted a positive change in NEWS2 recording and formed completion post dashboards intervention and potential for a further improvement as quality is monitored. In the current integration of EWS into EHRS, displaying the real-time score and generating alerts of EWS, like Modified Early Warning Score, Paediatric EWS, and NEWS, has shown several advantages in different care settings.34–36 It allowed for a real-time prediction of critical events associated with reduced hospitalisation costs and, more importantly, is believed to be a keystone for safe practice. In addition, dashboards have been increasingly implemented in healthcare and supported healthcare services, such as communicating patient-reported and clinical data in cystic fibrosis.37 Ward patient status and clinician performance auditing represent a modern method of quality improvement in the digital clinical environments to be promoted and examined widely.

Several additional characteristics could enhance the function of dashboard auditing deterioration management. Producing a live NEWS2 score for patients would add an advantage to estimating validation if analysed on wards and specialities level. In addition, alerting the first-line responders to escalation, including the CCOT team and bedside nurses, would benefit managers in tracking the escalation and whether it occurred due to the score, clinicians’ observation or the two combined. Expanding the functionality guided by the escalation of care and EWSs protocols would be needed to enhance the performance impact. However, further studies need to evaluate the effect of advancing the dashboard in the coming stages from a user perspective and the extent to which it can positively impact clinical care outcomes and work performance.

### Limitations

The dashboard was developed and tested at the time of the COVID-19 pandemic. This factor could have affected the engagement of key users and, therefore, the response to using the dashboard for auditing. Another disadvantage is the small team of clinical informatics that developed the dashboard during the pandemic pressure, which might have resulted in delays for further adjustments and applying a third PDSA cycle. We interviewed three health professionals during the examined phases; feedback from ward managers, nurses and doctors will show a better view of its validity for monitoring escalation of care. Furthermore, we did not verify the generalisability of the dashboard system in other trusted hospitals. Studies need to examine its feasibility and usefulness in different hospitals with differing structures and patients’ population.

## Conclusion

On-time data visualisation of deteriorating patient care is an effective and efficient method for establishing quality improvement. The deteriorating patient dashboard facilitated timely investigation and improvement of the practice of assessment and treatment from key users’ perspectives and performance analysis over time. Evaluating adherence to NEWS2 recording and escalation of care protocol can help clarify EWS validation where it is implemented. Advancing health dashboards by facilitating multiple health systems integration and clinicians learning digital health solutions will enhance dashboard functionality and improve user experience. In addition, functionality could be upgraded by analysing further NEWS2 and escalation of care protocol metrics and times; promoting live and historical data value. There is a need for further validation and quality improvement studies to verify the generalisability of the dashboard system in different settings.

## Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

## Ethics statements

### Patient consent for publication

Not applicable.

### Ethics approval

The study is registered and approved by the Health Research Authority (HRA) and Health and Care Research Wales (HCRW) and sponsored by University College London (UCL). The Stanmore Research Ethics Committee (REC) in London approved conducting the study; reference number: 20/PR/0286. Participants gave informed consent to participate in the study before taking part.

## Footnotes

*   Twitter @BaneenAlhmoud

*   Contributors DM has developed the project plan, and NK executed the development with guidance from DM. BA evaluated the dashboard, including interviews and analysis, with the guidance of AB. BA wrote the manuscript and is the guarantor. All authors contributed to the finding’s interpretation and revision of the manuscript.

*   Funding Baneen Alhmoud has received PhD funding from the Saudi Arabian Cultural Bureau to conduct this research. Grant number: ELP003964.

*   Competing interests None declared.

*   Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  Rocha HAL, Alcântara ACdeC, Netto FCdeB, et al. Dealing with the impact of the COVID-19 pandemic on a rapid response team operation in Brazil: quality in practice. Int J Qual Health Care 2021;33. doi:[doi:10.1093/intqhc/mzaa114](http://dx.doi.org/10.1093/intqhc/mzaa114). [Epub ahead of print: 20 Feb 2021].pmid:http://www.ncbi.nlm.nih.gov/pubmed/32991710
    
    

2.  Redfern OC, Griffiths P, Maruotti A, et al. The association between nurse staffing levels and the timeliness of vital signs monitoring: a retrospective observational study in the UK. BMJ Open 2019;9:e032157.[doi:10.1136/bmjopen-2019-032157](http://dx.doi.org/10.1136/bmjopen-2019-032157)pmid:http://www.ncbi.nlm.nih.gov/pubmed/31562161
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOS85L2UwMzIxNTciO3M6NDoiYXRvbSI7czoyNToiL2JtanFpci8xMS80L2UwMDIwMzMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  Salvatore FP, Fanelli S. Patient-Related complexity of care in healthcare organizations: a management and evaluation model. Int J Environ Res Public Health 2020;17:3463.[doi:10.3390/ijerph17103463](http://dx.doi.org/10.3390/ijerph17103463)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32429208
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

4.  Kostakis I, Smith GB, Prytherch D, et al. Impact of the coronavirus pandemic on the patterns of vital signs recording and staff compliance with expected monitoring schedules on general wards. Resuscitation 2021;158:30–8.[doi:10.1016/j.resuscitation.2020.11.014](http://dx.doi.org/10.1016/j.resuscitation.2020.11.014)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33221355
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

5.  Alhmoud B, Bonnici T, Patel R, et al. Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review. BMJ Open 2021;11:e045849.[doi:10.1136/bmjopen-2020-045849](http://dx.doi.org/10.1136/bmjopen-2020-045849)pmid:http://www.ncbi.nlm.nih.gov/pubmed/36044371
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvNC9lMDQ1ODQ5IjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpxaXIvMTEvNC9lMDAyMDMzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

6.  Burke C, Conway Y. Factors that influence Hospital nurses' escalation of patient care in response to their early warning score: a qualitative evidence synthesis. J Clin Nurs 2022. doi:[doi:10.1111/jocn.16233](http://dx.doi.org/10.1111/jocn.16233). [Epub ahead of print: 25 Mar 2022].pmid:http://www.ncbi.nlm.nih.gov/pubmed/35338540
    
    

7.  Petersen JA, Rasmussen LS, Rydahl-Hansen S. Barriers and facilitating factors related to use of early warning score among acute care nurses: a qualitative study. BMC Emerg Med 2017;17:36.[doi:10.1186/s12873-017-0147-0](http://dx.doi.org/10.1186/s12873-017-0147-0)pmid:http://www.ncbi.nlm.nih.gov/pubmed/29191159
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

8.  Petersen JA. Early warning score challenges and opportunities in the care of deteriorating patients. Dan Med J 2018;65:B5439.[doi:https://ugeskriftet.dk/files/scientific\_article\_files/2018-08/b5439.pdf](http://dx.doi.org/https://ugeskriftet.dk/files/scientific_article_files/2018-08/b5439.pdf)
    
    

9.  Petersen JA. Early warning Score-Challenges and opportunities in the care of deteriorating in-hospital patients: PHD thesis. University of Copenhagen: Faculty of Health and Medical Sciences, 2016.
    
    

10. Pivovarov R, Elhadad N. Automated methods for the summarization of electronic health records. J Am Med Inform Assoc 2015;22:938–47.[doi:10.1093/jamia/ocv032](http://dx.doi.org/10.1093/jamia/ocv032)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25882031
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1093/jamia/ocv032&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=25882031&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

11. Donaldson N, Brown DS, Aydin CE, et al. Leveraging nurse-related dashboard benchmarks to expedite performance improvement and document excellence. J Nurs Adm 2005;35:163???172–72.[doi:10.1097/00005110-200504000-00005](http://dx.doi.org/10.1097/00005110-200504000-00005)pmid:http://www.ncbi.nlm.nih.gov/pubmed/15834255
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

12. Albanese MP, Evans DA, Schantz CA, et al. Engaging clinical nurses in quality and performance improvement activities. Nurs Adm Q 2010;34:226–45.[doi:10.1097/NAQ.0b013e3181e702ca](http://dx.doi.org/10.1097/NAQ.0b013e3181e702ca)pmid:http://www.ncbi.nlm.nih.gov/pubmed/20562572
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=20562572&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

13. Mick J. Data-Driven decision making: a nursing research and evidence-based practice dashboard. J Nurs Adm 2011;41:391–3.[doi:10.1097/NNA.0b013e31822edb8c](http://dx.doi.org/10.1097/NNA.0b013e31822edb8c)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21934423
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1097/NNA.0b013e31822edb8c&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=21934423&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

14. Lee K, Jung SY, Hwang H, et al. A novel concept for integrating and delivering health information using a comprehensive digital dashboard: an analysis of healthcare professionals' intention to adopt a new system and the trend of its real usage. Int J Med Inform 2017;97:98–108.[doi:10.1016/j.ijmedinf.2016.10.001](http://dx.doi.org/10.1016/j.ijmedinf.2016.10.001)pmid:http://www.ncbi.nlm.nih.gov/pubmed/27919400
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

15. Beasley JW, Wetterneck TB, Temte J, et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011;24:745–51.[doi:10.3122/jabfm.2011.06.100255](http://dx.doi.org/10.3122/jabfm.2011.06.100255)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22086819
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjQvNi83NDUiO3M6NDoiYXRvbSI7czoyNToiL2JtanFpci8xMS80L2UwMDIwMzMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

16. Data dashboards - NHS Digital. Available: [https://digital.nhs.uk/dashboards](https://digital.nhs.uk/dashboards) [Accessed 14 Apr 2022].
    
    

17. Jeffs L, Beswick S, Lo J, et al. Insights from staff nurses and managers on unit-specific nursing performance dashboards: a qualitative study. BMJ Qual Saf 2014;23:1001–6.[doi:10.1136/bmjqs-2013-002595](http://dx.doi.org/10.1136/bmjqs-2013-002595)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25193341
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjEwOiIyMy8xMi8xMDAxIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpxaXIvMTEvNC9lMDAyMDMzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18. Lahey T, Nelson W. A Dashboard to improve the alignment of healthcare organization Decisionmaking to core values and mission statement. Camb Q Healthc Ethics 2020;29:156–62.[doi:10.1017/S0963180119000884](http://dx.doi.org/10.1017/S0963180119000884)pmid:http://www.ncbi.nlm.nih.gov/pubmed/31858947
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

19. Reisman M. EHRs: the challenge of making electronic data Usable and Interoperable. P T 2017;42:572–5.pmid:http://www.ncbi.nlm.nih.gov/pubmed/28890644
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

20. Meszaros J, Ho C-hsing, Ho C. Building trust and transparency? challenges of the opt-out system and the secondary use of health data in England. Med Law Int 2019;19:159–81.[doi:10.1177/0968533219879975](http://dx.doi.org/10.1177/0968533219879975)
    
    

21. KAPLAN B. How should health data be used?: privacy, secondary use, and big data sales. Camb Q Healthc Ethics[doi:10.1017/S0963180115000614](http://dx.doi.org/10.1017/S0963180115000614)
    
    

22. National Institute For Health And Care Excellence (NICE). National early warning score systems that alert to deteriorating adult patients in hospital. Medtech Innov Brief 2020;4731:1–18.[doi:https://www.nice.org.uk/advice/mib205/resources/national-early-warning-score-systems-that-alert-to-deteriorating-adult-patients-in-hospital-pdf-2285965392761797](http://dx.doi.org/https://www.nice.org.uk/advice/mib205/resources/national-early-warning-score-systems-that-alert-to-deteriorating-adult-patients-in-hospital-pdf-2285965392761797)
    
    

23. Rocha HAL, Alcântara ACdeC, Netto FCdeB, et al. Dealing with the impact of the COVID-19 pandemic on a rapid response team operation in Brazil: quality in practice. Int J Qual Health Care 2021;33:mzaa114.[doi:10.1093/intqhc/mzaa114](http://dx.doi.org/10.1093/intqhc/mzaa114)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32991710
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

24. Mitchell OJL, Neefe S, Ginestra JC, et al. Impact of COVID-19 on inpatient clinical emergencies: a single-center experience. Resusc Plus 2021;6:100135.[doi:10.1016/j.resplu.2021.100135](http://dx.doi.org/10.1016/j.resplu.2021.100135)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33969324
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

25. Bardi T, Gómez-Rojo M, Candela-Toha AM, et al. Rapid response to COVID-19, escalation and de-escalation strategies to match surge capacity of intensive care beds to a large scale epidemic. Rev Esp Anestesiol Reanim 2021;68:21–7.[doi:10.1016/j.redar.2020.09.003](http://dx.doi.org/10.1016/j.redar.2020.09.003)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33293100
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

26. NHS. Records management code of practice 2021. NHS, 2021. Available: [https://www.nhsx.nhs.uk/information-governance/guidance/records-management-code/](https://www.nhsx.nhs.uk/information-governance/guidance/records-management-code/)
    
    

27. Georgiou A, Pearson M. The role of health informatics in clinical audit: part of the problem or key to the solution? J Eval Clin Pract 2002;8:183–8.[doi:10.1046/j.1365-2753.2002.00323.x](http://dx.doi.org/10.1046/j.1365-2753.2002.00323.x)
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=12180366&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

28. Brand JW, Morrice DJ, Patteril MV, et al. National Early Warning Score 2 (NEWS2) to identify inpatient COVID-19 deterioration: The importance of pO2:FiO2 ratio. Clin Med 2021;21:e315–6.[doi:10.7861/clinmed.Let.21.3.2](http://dx.doi.org/10.7861/clinmed.Let.21.3.2)pmid:http://www.ncbi.nlm.nih.gov/pubmed/34001590
    
    [FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImNsaW5tZWRpY2luZSI7czo1OiJyZXNpZCI7czo5OiIyMS8zL2UzMTUiO3M6NDoiYXRvbSI7czoyNToiL2JtanFpci8xMS80L2UwMDIwMzMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29. Christoff P. Running PDSA cycles. Curr Probl Pediatr Adolesc Health Care 2018;48:198–201.[doi:10.1016/j.cppeds.2018.08.006](http://dx.doi.org/10.1016/j.cppeds.2018.08.006)pmid:http://www.ncbi.nlm.nih.gov/pubmed/30270135
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

30. Duffy JR, Brewer BB. Feasibility of a multi-institution collaborative to improve patient-nurse relationship quality. J Nurs Adm 2011;41:78–83.[doi:10.1097/NNA.0b013e3182059463](http://dx.doi.org/10.1097/NNA.0b013e3182059463)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21266886
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=21266886&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

31. St Pierre J. Staff nurses' use of report card data for quality improvement: first steps. J Nurs Care Qual 2006;21:8–14.[doi:10.1097/00001786-200601000-00003](http://dx.doi.org/10.1097/00001786-200601000-00003)pmid:http://www.ncbi.nlm.nih.gov/pubmed/16340681
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1097/00001786-200601000-00003&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=16340681&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

32. Davidson E, Heslinga D. Bridging the it adoption gap for small physician practices: an action research study on electronic health records. Information Systems Management 2006;24:15–28.[doi:10.1080/10580530601036786](http://dx.doi.org/10.1080/10580530601036786)
    
    

33. Borycki E, Kushniruk A, Armstrong B. Integrating electronic health records into health professional and health informatics education: a continuum of approaches. Acta Informatica Medica 2010;18:20–4.[doi:https://www.bibliomed.org/?mno=3299](http://dx.doi.org/https://www.bibliomed.org/?mno=3299)
    
    

34. Chapman SM, Maconochie IK. Early warning scores in paediatrics: an overview. Arch Dis Child 2019;104:395–9.[doi:10.1136/archdischild-2018-314807](http://dx.doi.org/10.1136/archdischild-2018-314807)pmid:http://www.ncbi.nlm.nih.gov/pubmed/30413488
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo5OiIxMDQvNC8zOTUiO3M6NDoiYXRvbSI7czoyNToiL2JtanFpci8xMS80L2UwMDIwMzMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

35. Horton DJ, Graves KK, Kukhareva PV, et al. Modified early warning score-based clinical decision support: cost impact and clinical outcomes in sepsis. JAMIA Open 2020;3:261–8.[doi:10.1093/jamiaopen/ooaa014](http://dx.doi.org/10.1093/jamiaopen/ooaa014)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32734167
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

36. Bedoya AD, Clement ME, Phelan M, et al. Minimal impact of implemented early warning score and best practice alert for patient deterioration. Crit Care Med 2019;47:49–55.[doi:10.1097/CCM.0000000000003439](http://dx.doi.org/10.1097/CCM.0000000000003439)pmid:http://www.ncbi.nlm.nih.gov/pubmed/30247239
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1097/CCM.0000000000003439&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

37. Van Citters AD, Gifford AH, Brady C, et al. Formative evaluation of a dashboard to support coproduction of healthcare services in cystic fibrosis. J Cyst Fibros 2020;19:768–76.[doi:10.1016/j.jcf.2020.03.009](http://dx.doi.org/10.1016/j.jcf.2020.03.009)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32354650
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

38. Lockwood JM, Thomas J, Martin S, et al. AutoPEWS: automating pediatric early warning score calculation improves accuracy without sacrificing predictive ability. Pediatr Qual Saf 2020;5:e274.[doi:10.1097/pq9.0000000000000274](http://dx.doi.org/10.1097/pq9.0000000000000274)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32426639
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

39. Khanna AK, Hoppe P, Saugel B. Automated continuous noninvasive ward monitoring: future directions and challenges. Crit Care 2019;23:194.[doi:10.1186/s13054-019-2485-7](http://dx.doi.org/10.1186/s13054-019-2485-7)pmid:http://www.ncbi.nlm.nih.gov/pubmed/31146792
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 

40. DIgital NHS. Electronic prescribing and medicines administration (EPMA) data, 2012
    
    

41. Ludikhuize J, de Jonge E, Goossens A. Measuring adherence among nurses one year after training in applying the modified early warning score and Situation-Background-Assessment-Recommendation instruments. Resuscitation 2011;82:1428–33.[doi:10.1016/j.resuscitation.2011.05.026](http://dx.doi.org/10.1016/j.resuscitation.2011.05.026)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21719184
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1016/j.resuscitation.2011.05.026&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=21719184&link_type=MED&atom=%2Fbmjqir%2F11%2F4%2Fe002033.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000297104800017&link_type=ISI) 

42. McGregor C. Improving time to antibiotics and implementing the "Sepsis 6". BMJ Qual Improv Rep 2013;2:u202548.[doi:10.1136/bmjquality.u202548.w1443](http://dx.doi.org/10.1136/bmjquality.u202548.w1443)