FormalPara Take-home message

In NHS hospitals, deteriorating ward patients referred to ICU are vulnerable: one in eight will die within a week of the referral, and half of those deaths will occur without ICU admission. While ICU admission delays are common, this study shows that prompt admission reduces mortality after allowing for both observed and unobserved differences in prognosis.

Introduction

Recent policy stresses the importance of identifying and responding to the deteriorating ward patient [1]. Current guidelines recommend that critical care admission should be delivered within 4 h [2]. However, supporting evidence is limited because randomised evaluation of prompt admission to critical care is deemed unethical. Yet without quantification of the benefits, it is difficult to assess the magnitude and importance of this problem.

Non-randomised evaluations are primarily confounded by treatment allocation bias [3]. Patients are prioritised on the basis of clinical severity so prompt admissions tend to have poorer prognoses. Risk adjustment will help remove this bias, but depends heavily on adequate measurement of all factors driving the decision about how to treat. In general, measured severity is an incomplete description, and often there are other end of the bed factors prompting clinicians to recommend prompt admission to critical care.

An alternative to experimental randomisation is to seek an instrument that naturally randomises patients to prompt admission or not. This natural randomisation is known as instrumental variable (IV) analysis, and has been similarly used to remove unmeasured confounding in the assessment of influenza vaccine efficacy and cardiac catheterisation [4, 5].

Here, we used critical care unit strain, measuring bed occupancy rates at the specific time of the patient’s assessment, for this purpose. This approach circumvents selection bias found in previous observational studies comparing prompt to delayed admission [6,7,8,9,10,11,12,13]. Delay can only be defined with hindsight. The bedside choice is to ‘admit now’ or to ‘watch and wait’, not to admit now or deliberately delay. A clinician neither admits the watched patient who improved nor can admit the watched patient who unexpectedly died. The delayed admissions are the subgroup of watchful-waiting controls who survive but continue to deteriorate.

A fair comparison requires prospective follow-up of all watchful-waiting controls. Our prospective study does this and also exploits the learning opportunity created by a constrained supply of critical care beds. Critical care bed provision in the UK is lower than in the majority of European (6.6 adult critical care beds versus a European average of 11.5 beds per 100,000 population) and North American health care systems [14, 15]. In settings where critical care capacity is less constrained, using strain to evaluate the effect of delay would be difficult.

We first describe the effect of critical care strain on decision-making, and the delivery of critical care for all deteriorating ward patients referred to critical care. We explore whether delays to admission engendered by high strain allow us to estimate the effect of effect of delay on patient outcome. Finally, we focus on the subgroup recommended for critical care by the bedside clinician.

Methods

Study design, participants and procedures

The full study protocol is available on the Intensive National Audit and Research Centre’s (ICNARC) website. In brief, the (SPOT)light study was a prospective cohort study of the deteriorating ward patient referred for assessment by critical care. The assessment had to be conducted on an inpatient ward by either a member of the critical care medical staff or the critical care outreach team (CCOT). Repeat visits and re-admissions were excluded as were patients where intensive care units (ICU) admission was either a priori refused (treatment limitation orders) or inevitable (cardiac arrests, admissions temporarily housed in theatre recovery etc.). Admissions following surgery, where delay was due to the process of care, were also excluded.

The study was registered on the National Institute of Health Research (NIHR) research portfolio (No. 9139). Hospitals were eligible for inclusion if they participated in the national clinical audit for critical care—the ICNARC case mix programme (CMP). Research teams at each hospital attended a data set familiarisation course and were given a manual of data definitions. The ICNARC clinical trials unit provided support for the study.

Reporting was via a secure online web portal that performed real-time field and record level validation. Hospitals were asked to report all consecutive ward referrals to critical care. Contemporaneous data collection was recommended, but missed referrals were sought and accepted retrospectively. We used the proportion of unplanned ward admissions to critical care in the CMP that were successfully linked to the (SPOT)light database to monitor data capture each month. Data from specific months in which data linkage rates fell below 80% were excluded from the primary analysis (but explored in sensitivity analyses). Further online validation reports were completed by all hospitals before the database was locked in September 2012. Fact and date of death were then requested from the NHS Information Service. CCOT provision was reported by participating hospitals. CMP and hospital episode statistics (HES) data were used to define critical care provision and hospital characteristics.

Definitions

Physiology measurements at the time of the ward assessment were abstracted. From these, the ICNARC physiology score, the NHS National Early Warning Score (NEWS) and the Sequential Organ Failure Assessment (SOFA) score were calculated with missing values given zero weights as recommended [16,17,18]. The patient’s existing dependency at assessment was defined using the UK critical care minimum data set (CCMDS) levels of care: levels 0 and 1 are most commonly provided on normal wards while levels 2 and 3 are within high dependency (HDU) and ICU respectively [19]. The assessor was asked to recommend a future level of care, and recommendations for levels 2 or 3 were considered as recommendations for critical care admission. Prompt admission was defined as one within 4 h of ward assessment, in line with recently published UK guidelines [19].

The indicator of critical care unit strain was the difference between the maximum number of beds reported to ICNARC and the number of actively treated patients (not medically fit for discharge) occupying those beds at the time the ward patient was assessed. Units were defined as being under low, medium or high strain corresponding to having two or more, one, or zero or fewer empty beds (the last of these where strain exceeded reported capacity) respectively.

Statistical analysis

The aim of the primary analysis was to estimate the effect of prompt critical care admission versus watchful waiting on 90-day mortality. We repeated the analysis in the subgroup recommended for critical care at the bedside assessment.

We first built orthodox proportional hazards models with risk adjustment to handle the anticipated treatment allocation bias according to those risk factors that were observed. We then built IV models in two stages. In the first stage, a selection model was constructed to predict prompt admission including the effect of strain. In the second stage, an outcome model was built replacing strain with the fitted prediction of prompt admission from the selection model. All models were adjusted for patient-level confounders including age, sex, reported referral delay, sepsis diagnosis, peri-arrest status, existing CCMDS level of care, and severity of illness using the ICNARC, SOFA and NEWS scores.

We tested for weak instruments using the Kleibergen–Paap F test, and used Huber–White (robust) standard errors to allow for hospital-level clustering and potential heteroscedasticity [20]. Bivariate probit IV models were used to ensure that model predictions were correctly constrained [21]. To aid interpretation, we also calculated the marginalised average treatment effect (ATE) for the population, and converted coefficients to approximate odds ratios (OR) by scaling by 1.6 [22]. Finally, we examined the sensitivity of the results to changes in the data linkage quality threshold (between 70% and 90%).

Implementation

The study was registered with ClinicalTrials.gov (NCT01099813). The sample size was calculated to evaluate mortality increases from delay to admission using estimates from 2007 ICNARC CMP data. The target sample size was 12,075–20,125 patients referred to critical care, allowing for delays to occur in 10–40% of admissions and mortality effect sizes of 5–10%.

Categorical data were reported as counts and percentages, and continuous data as mean (SD) or median (IQR) values. Effect measures are reported with their 95% confidence intervals. Analyses were performed in R (version 3.03) except for the IV analysis which used the ivregress and biprobit commands provided in Stata (version 12.1).

Results

Between September 2010 and December 2011, 49 hospitals (10 university-affiliated) submitted records for 435 study months. After cross-checking against national audit records, reporting was potentially incomplete for 66 (15%) months which were excluded. The primary analysis therefore included 369 study months, equivalent to a median eight study months per site (IQR 5–9) with a mean data linkage rate of 95%.

The 369 study months captured 18,122 consecutive ward assessments. We excluded a further 2555 (14%) patients with treatment limitation orders and 1632 (9%) post-critical care follow-up visits. Timing data were unavailable for a further 129 patients. This left 12,380 patients in 48 hospitals available for analysis [a median of 222 patients (IQR 142–304) per hospital] (Fig. 1).

Fig. 1
figure 1

Flow diagram of patients screened: ward referrals assessed for eligibility at participating hospitals, reasons for exclusion, and admission timing following bedside assessment for all patients assessed, and for the subgroup recommended for critical care at assessment

Full size image

Participating hospitals

There was a median of 12 critical care beds per hospital (IQR 9–18, mixed level 2 and level 3 beds) most often in a single physical location (45 hospitals). Each unit admitted a median 20 unplanned admissions (IQR 14–26) from the ward per month, representing 36% of all ICU admissions (IQR 31–43%). Critical care outreach was available 24/7 in 14 hospitals, daily but not overnight in 19 hospitals, weekday daytime only in 13 hospitals and was not offered in two hospitals.

Patient characteristics

Table 1 shows the baseline data for all ward patients assessed. Sepsis was reported in 7586 (61%) patients; of these, the respiratory system was considered to be the source in 3851 (51%). Organ failure, defined as a SOFA score greater than one, was present in 4227 (34%) of patients. A total of 1173 patients (9%) were in respiratory failure, 2403 (19%) were in renal failure and 3629 (29%) were shocked. There was a clear correlation between physiological severity and short-term (7-day) outcome (Supplemental Fig. 1), but organ support at the time of assessment was uncommon (694 patients, 5%).

Table 1 Study patients and those admitted promptly
Full size table

Recommendation for critical care at bedside assessment

A total of 4560 (37%) patients were recommended for critical care at the bedside assessment. These patients were younger (by 1.3 years, 95% CI 0.7–1.9) and more acutely unwell (by 4.1 ICNARC physiology points, 95% CI 3.8–4.4) than those not recommended. Patients older than 80 years were less likely to be recommended for critical care (OR 0.61, 95% CI 0.53–0.71) even after risk adjustment (Supplemental Table 1).

Prompt admission to critical care

Overall, 36% (4401 patients) were admitted to critical care in the following week rising to 72% (3296) of those recommended on assessment. The median time from assessment to admission was 3 h (IQR 1–9) overall and 2 h (IQR 1–5) for those recommended. Admissions were prompt for 19% (2411 patients) rising to 49% (2250 patients) for those with a bedside recommendation.

A total of 1636 (13%) patients died in the week following the assessment with 856 deaths (52%) following critical care admission and 780 deaths (48%) on the ward without admission. For those recommended for critical care at assessment, there were 862 deaths (19%) of which 663 (77%) received critical care before death and 199 (23%) occurred on the ward without ICU admission.

Prompt admission and risk-adjusted mortality

For those admitted promptly, 90-day mortality was 36.7% (885 deaths) compared to 28.6% (2851) for the watchful-waiting control group. Patients who were admitted promptly had higher physiological severity scores; for example, the ICNARC physiology score was 4.4 ICNARC physiology points [95% CI 4.0–4.7] higher on average. Without risk adjustment, the proportion of patients who died prior to 90 days was higher for prompt critical care admissions with a hazard ratio (HR) of 1.42 (95% CI 1.32–1.54). With risk adjustment (Supplemental Table 3), survival was equivalent [HR 0.98 (95% CI 0.88–1.09), p = 0.702].

For the 4560 patients recommended for critical care, 90-day mortality was 37.2% (837 deaths) for prompt admissions versus 34.4% (794 deaths) for controls. Prompt admissions again had higher physiological severity [2.0 ICNARC physiology points (95% CI 1.6–2.5)]. Before risk adjustment, survival was worse (HR 1.12, 95% CI 1.02–1.24), but again was again equivalent after risk adjustment [HR 0.99 (95% CI 0.85–1.10), p = 0.852, Supplemental Table 4].

Critical care strain

There were 10,039 (81%) bedside assessments when there were two or more empty beds on the critical care unit, 1353 (11%) when there was just one empty bed and 988 (8%) when the unit was already fully occupied (Table 2). As strain increased, the proportion of prompt admissions fell (21%, 15% and 9%, p < 0.0001), corresponding upward trend in the median time to admission: 3 h (IQR 1–8), 4 h (IQR 1–12) and 5 h (IQR 2–16) (p = 0.0009, Supplementary Fig. 2).

Table 2 Effects of strain on the admission pathway: recommendation for, and prompt admission to critical care, severity of illness, and outcomes stratified by critical care unit occupancy at the time of the bedside assessment
Full size table

Strain varied by time of day, day of the week, and season; however, the relationship between prompt admission and occupancy remained even after adjustment (Supplemental Table 2), and there was a strong negative correlation between strain and prompt critical care admission (Kleibergen–Paap F statistic 83, p < 0.0001).

There was evidence that the delay in admission translated into further ongoing physiological deterioration (ICNARC physiology scores increased by 3.1, 3.3 and 4.5 points respectively, p = 0.03). Regardless of timing, the overall probability of receiving critical care also fell (38%, 32% and 25% for assessments during times of low, medium and high strain, p < 0.0001).

Within the subgroup recommended, the proportion of prompt admissions also fell as strain increased (53%, 38% and 23%, p < 0.0001) and median time to admission increased: 2 h (IQR 1–4), 3 h (IQR 1–6) and 4 h (IQR 2–9) (p < 0.0001). The proportion of patients managed without critical care increased from 25%, to 33%, to 50% during low, medium and high strain periods (Supplemental Table 5).

Critical care strain and mortality

The 90-day mortality was 30.0% (3007 deaths), 30.8% (417 deaths) and 31.6% (312 deaths) for deteriorating ward patients assessed at times of low, medium and high critical care strain respectively (Fig. 2). Using the change in mortality driven by critical care strain, the instrumental variable model estimated a reduction in 90-day mortality of 7.4% (95% CI 1.7–18.5%, p = 0.117) which was equivalent to an odds ratio of 0.68 (95% CI 0.42–1.10, Table 3 and Supplemental Table 6).

Fig. 2
figure 2

Patient disposition over time following bedside assessment: proportion of patients who are alive in critical care, who died within or following a critical care admission, or who died without admission to critical care by critical care unit strain at the time of the bedside assessment

Full size image
Table 3 Instrumental variable model for the effect of prompt admission on 90-day mortality: for all patients, and for the subgroup with recommended to critical care at the bedside assessment
Full size table

Among those recommended for critical care, unadjusted 90-day mortality was 35.0% (1307 deaths), 41.8% (197 deaths) and 35.9% (127 deaths) for ward assessments at times of low, medium and high strain. For this subgroup, the average reduction in 90-day mortality for prompt admission was 16.2% (95% CI 1.1–31.3%, p = 0.036) equivalent to an odds ratio of 0.46 (95% CI 0.22–0.96, p = 0.036).

Discussion

This prospective cohort study describes the outcomes of more than 12,000 ward patients assessed by critical care teams in 48 acute NHS hospitals. Delay to admission is common. Those patients admitted promptly to critical care are manifestly more unwell, with a higher unadjusted mortality (37% vs 29%). Risk adjustment for observed severity is unable to show a benefit for a prompt admission strategy. However, we argue that unobserved differences in baseline risk are part of a clinician’s bedside assessment, and our risk adjustment is likely to be incomplete.

We exploit the random variation in critical care strain to additionally allow for unobserved differences in baseline severity. We show that prompt critical care admission is less likely for patients referred during times of strain. Using this natural experimental set-up, we found that prompt admission reduces 90-day mortality without reaching statistical significance. For the subgroup recommended for critical care, the estimated reduction in 90-day mortality is larger and statistically significant. There is a large amount of uncertainty in the precise estimate of these effects, but the weight of evidence favours a prompt admission strategy, especially where the bedside assessor recommends critical care.

The prompt admission strategy might confer a survival advantage in two distinct ways: explicitly be delivering critical care to a particular patient more promptly; or implicitly, by increasing the opportunity for critical care in the population. Of note, half of the deaths in the first week occur without admission to critical care, and death without critical care is more common when strain is greater.

It is not easy to separate out these two effects. Although previous studies compare the prompt to the ‘delayed’ admission, the delay is defined with hindsight which creates exclusion and survivor bias [6,7,8,9,10,11,12,13]. The real bedside decision is to ‘admit now’ or to ‘watch and wait’. Direct patient-level randomisation to evaluate a ‘prompt’ versus a ‘watchful waiting’ strategy is ethically challenging. Recent attempts have resorted to cluster randomisation of a strategy to increase referrals to critical care (specifically in the elderly and focussing on access not timing) [23]. This increased access but also increased severity of illness leading to the same issues with adjusting for unobserved differences in baseline severity [24].

Our attempt to minimise treatment selection bias (confounding by indication) relies on the assumption that critical strain acts as a natural randomisation event. We defend this assumption in three ways. Firstly, critical care occupancy at a specific time is unpredictable. It is the net result of all the decisions that lead to admission, discharge and death in a complex, maybe chaotic, system. Secondly, we have included controls where occupancy might be associated with baseline severity (i.e. increased strain during the winter). Thirdly, although strain might translate as a crowding effect thereby impairing the quality of care within the ICU, the evidence for this is conflicting [25, 26]. Moreover, crowding could not possibly affect the outcomes of the two thirds of patients never admitted to critical care. We additionally observed that patients assessed when there were no available beds deteriorated more prior to admission. This suggests a causal pathway that is independent of any effect of crowding after admission.

An important limitation of IV analyses is that they are notoriously weak, and, without very large sample sizes, there is a risk of not detecting a true difference when one exists [27, 28]. This may be understood if we consider IV analysis as a randomised controlled trial but with poor compliance. Critical care strain (at the time of the bedside assessment) is the random coin toss and prompt admission is the treatment randomised. However, clinical teams may find ways to deliver prompt admission even at times of high strain (perhaps by accelerating discharge or flexing staffing). We saw 92 patients admitted promptly even though there were no beds at the instant of bedside assessment. Imperfect compliance requires either large samples or large effect sizes to achieve significance. Of note, our effect size was larger for the most unwell patients, and only then achieved significance at the 5% threshold.

Our finding of harm from reduced or delayed access to critical care does not stand in isolation. At a patient level, reducing exposure to critical care by premature discharge rather than delayed admission has also increased mortality [29]. At a population level, expanding critical care capacity through an increase in funding during health service reforms is similarly associated with an improvement in outcomes [30]. The one previous study (five hospitals, 749 patients) of prompt admission to critical care that also used the watchful-waiting cohort as controls also found benefit for prompt admission [31].

We also need to acknowledge that in an observational study of this size, there are limitations in the quality of the data recorded. We did not perfectly capture all admissions to critical care in the study database, and we must assume that a proportion of referrals were also missed. However, we tested our findings by raising threshold for judging data capture to 90% so that the median proportion of eligible admissions was 97%. We found no consistent difference in any result other than a fall in precision as the quality threshold increased, and the sample size inevitably fell.

Aspects of the study stand independent of these limitations. Regardless of the effect of prompt admission to critical care, we have identified a cohort of hospital patients at very high risk. This risk is heavily front-loaded, and the window for intervention is short. The bedside assessment is an effective but imperfect triage tool, as the mortality in those initially refused admission is high. Given that we already excluded patients with treatment limitations, it is of concern that nearly half of these early deaths occur without a trial of critical care.

A substantial proportion of patients recommended for critical care are not offered a bed, and this proportion increases when capacity is limited. Although expanding critical care bed numbers may help, supply quickly saturates this expensive resource [32]. Given that the benefit of prompt critical care is unlikely to be equal for all referrals, the challenge now is to better prioritise. Interestingly, this same approach has also been highlighted as crucial to both patients and the public [33].