Prospective cohort study of routine use of risk assessment scales for prediction of pressure ulcers
BMJ 2002; 325 doi: https://doi.org/10.1136/bmj.325.7368.797 (Published 12 October 2002) Cite this as: BMJ 2002;325:797
- Lisette Schoonhoven (L.Schoonhoven{at}jc.azu.nl), PhD studenta,
- Jeen R E Haalboom, associate professor of medicineb,
- Mente T Bousema, dermatologistc,
- Ale Algra, associate professor of clinical epidemiologya,
- Diederick E Grobbee, professor of clinical epidemiologya,
- Maria H Grypdonck, professor of nursing sciencesa,
- Erik Buskens, assistant professor of medical technology assessment,for the prePURSE study groupa
- aJulius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, PO Box 85500, 3508 GA, Utrecht, Netherlands
- bDepartment of Internal Medicine, University Medical Centre Utrecht
- cDepartment of Dermatology, Eemland Hospital, PO Box 1502, 3800 BM, Amersfoort, Netherlands
- Correspondence to: L Schoonhoven
- Accepted 15 April 2002
Abstract
Objective: To evaluate whether risk assessment scales can be used to identify patients who are likely to get pressure ulcers.
Design: Prospective cohort study.
Setting: Two large hospitals in the Netherlands.
Participants: 1229 patients admitted to the surgical, internal, neurological, or geriatric wards between January 1999 and June 2000.
Main outcome measure: Occurrence of a pressure ulcer of grade 2 or worse while in hospital.
Results: 135 patients developed pressure ulcers during four weeks after admission. The weekly incidence of patients with pressure ulcers was 6.2% (95% confidence interval 5.2% to 7.2%). The area under the receiver operating characteristic curve was 0.56 (0.51 to 0.61) for the Norton scale, 0.55 (0.49 to 0.60) for the Braden scale, and 0.61 (0.56 to 0.66) for the Waterlow scale; the areas for the subpopulation, excluding patients who received preventive measures without developing pressure ulcers and excluding surgical patients, were 0.71 (0.65 to 0.77), 0.71(0.64 to 0.78), and 0.68 (0.61 to 0.74), respectively. In this subpopulation, using the recommended cut-off points, the positive predictive value was 7.0% for the Norton, 7.8% for the Braden, and 5.3% for the Waterlow scale.
Conclusion: Although risk assessment scales predict the occurrence of pressure ulcers to some extent, routine use of these scales leads to inefficient use of preventive measures. An accurate risk assessment scale based on prospectively gathered data should be developed.
What is already known on this topic
The incidence of pressure ulcers in hospitalised patients varies between 2.7% and 29.5%
Guidelines for prevention of pressure ulcers base the allocation of labour and resource intensive measures on the outcome of risk assessment scales
Most risk assessment scales are based on expert opinion or literature review and have not been evaluated
The sensitivity and specificity of risk assessment scales vary
What this study adds
The effectiveness of available risk assessment scales is limited
Use of the outcome of risk assessment scales leads to inefficient allocation of preventive measures
Introduction
Patients admitted to hospital or otherwise confined to a bed, chair, or wheelchair are at risk from developing pressure ulcers. This presents a huge burden for health care in Western countries. In the Netherlands, more than 1% of the total healthcare budget is spent on preventing and treating pressure ulcers or the prolonged hospital stay once a pressure ulcer has developed. Pressure ulcers are the third costliest disorder, after cancer and cardiovascular diseases.1 In 1999, 8-10% of hospital patients in the Netherlands had pressure ulcers of grade 2 or worse.2 The proportion of patients newly admitted to hospital that developed pressure ulcers varied from 2.7% to 29.5%.3–8 Preventive measures are expensive and labour intensive: patients at risk of developing pressure ulcers should be identified. 9 10
At least 40 risk assessment scales exist.11 Most reflect expert opinion, literature review, or adaptation of an existing scale. Neither risk factors nor weights attributed to them have been identified, using adequate statistical techniques.12 Only six risk assessment scales have been tested for their predictive validity.12 Of these six, the Norton and Waterlow scales have been tested twice and the Braden scale nine times. The results varied, and little evidence of predictive value or accuracy of the scales was available. 3 9 12 13–15Moreover, most of the studies had methodological limitations12: they were small and conducted in varying populations. Also, in some studies the nurse was not blinded when doing the scoring nor were the results adjusted to take account of preventive measures.
Despite these shortcomings, the Braden and Norton scales are recommended tools in North American guidelines for the prevention of pressure ulcers.4 In the United Kingdom, the Waterlow and Norton scales are the two scales most commonly used,14 and expensive preventive measures are taken based on their outcome. Because the Norton,16 Braden,17 and Waterlow scales (www.awma.com.au/pages/Guidelines.pdf,10 can be viewed as a standard of reference and are recommended in several practice guidelines, we chose to evaluate their predictive value in a prospective cohort of 1229 hospitalised patients.
Methods
The prevention and pressure ulcer risk score evaluation study (prePURSE) is a prospective cohort study that includes patients from the Utrecht University Medical Centre and Eemland Hospital, Amersfoort, the Netherlands. Between January 1999 and June 2000,patients admitted to the surgical, internal, neurological, and geriatric wards were asked to participate in the study. Patients without pressure ulcers, older than 18 years, and with an expected admission of at least five days were eligible: a total of 6000 during the study period. Research nurses visited each ward twice a week and asked eligible patients admitted in the past 48 hours to participate. A quarter (1536) of patients were visited, of whom 93.2% (1431) agreed to participate. Eventually, 80% of patients (1229) had at least one follow up visit before discharge (table 1).Although not strictly random, we enrolled an unselected representative sample of patients. The study was approved by the ethics committees of both hospitals.
Characteristics of participants who had at least one follow up before discharge (n=1229). Values are numbers (percentages) unless stated otherwise
Data collection
A research nurse visited patients within 48hours of admission and once a week subsequently until they developed a pressure ulcer, were discharged, or had stayed in hospital for more than 12 weeks. A nurse checked for the presence of pressure ulcers and collected information on all risk factors included in the risk assessment scales (table 2). The scales sum the points for individual items into one overall score. A threshold given by the original author of the scale divides the patients into at risk or not at risk for developing pressure ulcers. At each visit, we collected information on preventive measures. Attending nurses were blinded for the observations by the research nurse.
Items included in the risk assessment scales
Pressure ulcers were classified according to the four grades of the European pressure ulcer advisory panel.10 Pressure ulcers of grade 2 or worse were included.10 Preventive measures were included if, at the time the skin was inspected, the patient had a pressure reducing mattress or bed or was repositioned regularly.
Statistical analysis
Patient's scores on the Norton, Braden, and Waterlow scales were calculated for each visit. The ability of the scales to predict whether pressure ulcers will develop was determined from the area under the receiver operating characteristic curve, using SPSS. 18 19 The area under the curve can range from 0.5 (no discrimination) to 1.0 (perfect discrimination).
We carried out analyses each week using receiver operating characteristic curves, but as most pressure ulcers occurred in the first four weeks, we limited our “per week analyses” to these. Similar analyses were carried out excluding 57 patients who received preventive measures without developing pressure ulcers. These patients were excluded because it was impossible to distinguish effective prevention from false positive cases. In addition, similar analyses were performed excluding 747 surgical patients, because these patients may be at specifically high risk of developing pressure ulcers.20 We also combined data for 2190 patient weeks and repeated the analyses.
To enable comparison with other studies, we calculated positive predictive value and negative predictive value, as well as sensitivity and specificity, at recommended cut-off points. 10 21 22
Results
A total of 135 (11%) patients developed pressure ulcers while in hospital. Most pressure ulcers (129) developed in the first four weeks. Overall, the weekly incidence of patients with pressure ulcers was 6.2% (95% confidence interval 5.2% to 7.2%). The maximum number of follow up visits was 12, the median two.
A total of 57 patients received preventive measures for 101 patient weeks in total. Patients receiving preventive measures were about five years older than those not receiving such measures, and a higher proportion developed pressure ulcers (17.8% v 5.5%) (table 3). Most patients at risk, according to the assessment scales, did not receive preventive measures; some patients considered not at risk did receive preventive measures.
Use of preventive measures in patients at risk of pressure ulcers
For all patients, the area under the curve for the first week of follow up was 0.51 (0.44 to 0.58) for the Norton scale, 0.52(0.45 to 0.59) for the Braden scale, and 0.60 (0.53 to 0.66) for the Waterlow scale (figure; table 4). Results were similar when the 57 patients who received preventive measures without developing pressure ulcers were excluded. With both the 57 patients who received preventive measures without developing pressure ulcers and the 747 surgical patients excluded, the areas under the curve were 0.69 (0.63 to 0.76), 0.70 (0.63 to 0.77), and 0.67 (0.61 to 0.73), respectively. Excluding only the 747 surgical patients gave similar results. In subsequent weeks, the areas under the curves for the risk assessment scales did not differ substantially between the subpopulation and the entire group (table 4).
Areas under receiver operating characteristic curve (95% confidence intervals) for the Norton, Braden, and Waterlow risk assessment scales for pressure ulcers at each week of follow up of hospitalised patients
Receiver operating characteristic curves for Norton, Braden, and Waterlow risk assessment scales in the first week, including all patients (top) and for all weeks combined, in subpopulation excluding surgical patients and patients with preventive measures who did not develop pressure ulcers (bottom)
For all patients over all weeks (2190 patient weeks), the areas under the curves were 0.56 (0.51 to 0.61), 0.55 (0.49 to 0.60), and 0.61 (0.56 to 0.66). In the subpopulation excluding patients who received preventive measures without developing pressure ulcers (83 patient weeks) and surgical patients (752 patient weeks), the areas under the curve were 0.71 (0.65 to 0.77), 0.71 (0.64to 0.78), and 0.68 (0.61 to 0.74) (table 5; figure).
Areas under operating receiver characteristic curves and test characteristics (95% confidence intervals) for entire dataset for three risk assessment scales*
Because the areas under the curves indicated relatively good performance of the risk assessment scales in the subpopulation, we also calculated the positive predictive values, negative predictive values, sensitivity, and specificity of the scales at their respective cut-off points (table 5).
Discussion
The three scales most commonly used to assess the risk of developing pressure ulcers— the Norton, Braden, and Waterlow scales—do not satisfactorily predict pressure ulcer development in patients admitted to hospital. This may be because the risk assessment scales are based on clinical observation and pathophysiological insights, and not on adequate prospective or prognostic research.
We observed patients once a week. Another study found that the Braden scale was more useful between 48 and 72 hours after admission.22 Guidelines about preventing pressure ulcers advise assessing the risk on admission and once a week afterwards,23 at periodic intervals,4 or depending on changes in the condition of the patient.10 In hospitals, the interval between assessments is usually a week.
Preventive measures
Preventive measures may modify the association between scores on risk assessment scales and the development of pressure ulcers. Only 10 patients received preventive measures both at the visit during which the skin was assessed and the previous visit. Because patients who had not received preventive measures at the previous visit could have had preventive measures applied later that week, we defined preventive measures as measures present at the time the skin was assessed. By doing so, however, we also took into account an unknown number of patients who received the measures as treatment for their pressure ulcers. To assess the effect of these measures on the association between risk assessment scales and development of pressure ulcers we excluded only those patients who received measures and did not develop pressure ulcers. The results did not differ from those for the total population; preventive measures did not affect the association between score on risk assessment scales and the development of pressure ulcers. We considered pressure reducing mattresses or beds and regular repositioning to be preventive measures. As there are no conclusive comparative studies on effectiveness of these measures, we did not distinguish between them.
Week of admission
The first and later weeks of follow up differed in discriminative ability. This may be explained by differences in patients' characteristics over the period of admission. In the first week of admission more than half of the patients (747) had undergone surgery; surgery is considered a risk factor for pressure ulcer development.20 Incidence of pressure ulcers in surgical patients varies from 19% to 66%,20 and almost a quarter (23%) of the pressure ulcers which develop in the hospital may be acquired intraoperatively,24 but only the Waterlow scale takes surgery into account. Although the area under the curve for the Waterlow scale was statistically significantly greater than 0.5, this scale discriminated only marginally better than the Norton and Braden scales. This may be due to an imprecise definition of surgery or the weight attributed to surgery, in the Waterlow scale. After we excluded surgical patients, the area was greater and was significant for all scales. These results may partly be explained by the fact that we observed patients once a week. Most patients undergoing surgery were in a reasonable physical condition at admission (first observation) and therefore not at risk, according to the risk assessment scales. It may well be that the scales would have discriminated better immediately after surgery rather than a week after admission. Intraoperatively acquired pressure ulcers, however, still could not have been predicted. Including imminent surgery as a factor in the risk assessment might improve prediction.
The discriminative ability of the scales in all weeks of follow up did not change greatly when surgical patients were excluded. We combined the data of the different weeks of follow up. The scales are able to predict whether or not a patient develops a pressure ulcer in 70% of the cases. The positive predictive value of the risk assessment scales, however, varied between 5.3% and 7.8%: only 5% to 8% of the patients for whom the risk assessment scales recommend receiving preventive measures actually develop pressure ulcers. Although the scales predict development of pressure ulcers, to some extent, strict application of the scales leads to inefficient use of preventive measures.
Previous studies
Although some earlier studies reported higher sensitivity and specificity for the Norton and Braden scales, 9 16 17 we have confidence in our results. We defined pressure ulcers as grade 2 or worse. Older lesions of the skin would still have been visible as a scab at a subsequent visit. Consequently, no pressure ulcers could have been missed. Also, the results of the earlier studies may well have been flawed because preventive measures were not taken into account.14 Preventive measures may have stopped pressure ulcers developing. Most preventive measures were taken in patients who were not at risk, according to the risk assessment scales. In fact, in only 67 (7.8%) of the patient weeks which the Norton scale considered high risk, preventive measures were given. Despite prevention, eight patients developed pressure ulcers. Accordingly, the performance of the Norton scale may have been modified only in 6.9% (59) of the patient weeks at risk. A similar outcome was found for the other two scales: the effect of preventive measures was small.
As the incidence of pressure ulcers (6.2%) is low, it may not be possible to improve much on the prediction of pressure ulcers. Eventually, it may be more effective to treat a grade 1 pressure ulcer immediately than to try to “predict and prevent.”
Conclusion
The broadly advocated advice to use risk assessment scales for pressure ulcers and to use the outcomes to decide on preventive measures leads to ineffective and inefficient treatment for most patients. Future research should identify factors actually associated with the development of pressure ulcers and evidence based risk assessment scales should be constructed.
Acknowledgments
Contributors: JH had the idea for the study, obtained the grant, and designed and supervised the study. EB contributed to the design and supervision, helped to analyse and interpret the data, and revised the article. LS managed the project, collected some of the data, analysed and interpreted the data, and wrote the article. MB contributed to the design of the study and supervised the data collection. MG, DG, and AA helped interpret the data and revised the article. All authors reviewed the paper and contributed to the final version. EB is guarantor. Chantal Cornelis was responsible for data collection and Bep Verkerk created the database and was responsible for its management.
Footnotes
-
Funding Zorg Onderzoek Nederland (ZON) (Praeventiefonds 28-2821).
-
Conflict of interest None declared.