Article Text
Abstract
Background: The National Institute for Health and Clinical Excellence (NICE) issued guidance on surgical techniques for tonsillectomy during a national audit of surgical practice and postoperative complications.
Objective: To assess the impact of the guidance on tonsillectomy practice and outcomes.
Design: An interrupted time-series analysis of routinely collected Hospital Episodes Statistics data, and an analysis of longitudinal trends in surgical technique using data from the National Prospective Tonsillectomy Audit.
Participants: Patients undergoing tonsillectomy in English NHS hospitals between January 2002 and December 2004.
Main outcome measure: Postoperative haemorrhage within 28 days.
Results: The rate of haemorrhage increased by 0.5% per year from 2002, reaching 6.4% when the guidance was published. After publication, the rate of haemorrhage fell immediately to 5.7% (difference 0.7%: 95% CI −1.3% to 0.0%) and the rate of increase appeared to have stopped. Data from the National Prospective Tonsillectomy Audit showed that the fall coincided with a shift in surgical techniques, which was consistent with the guidance.
Conclusion: NICE guidance influenced surgical tonsillectomy technique and in turn produced an immediate fall in postoperative haemorrhage. The ongoing national audit and strong support from the surgical specialist association may have aided its implementation.
Statistics from Altmetric.com
The National Institute for Health and Clinical Excellence (NICE) is responsible for providing national guidance on clinical practice for the National Health Service (NHS) in England and Wales.1 On 24 March 2004, NICE issued interim guidance on tonsillectomy,2 an operation that is performed on around 50 000 patients each year in English NHS hospitals.3 The guidance concerned the use of diathermy, a “hot” electrosurgery technique that could be used for dissecting the tonsil and/or stopping subsequent bleeding (haemostasis). It recommended that surgeons use as little diathermy as possible, especially where it was used for both dissection and haemostasis. The guidance also recommended that surgeons consider stopping the use of monopolar diathermy.
The guidance was developed from the interim results of the National Prospective Tonsillectomy Audit (NPTA) which had begun in 2003 to monitor the occurrence of postoperative haemorrhage and other complications. These interim results showed that the rate of haemorrhage was three times higher with the use of diathermy throughout an operation when compared with the traditional approach of “cold” dissection, in which the tonsils are removed with cold steel instruments, and subsequent bleeding is controlled with ties or gauze packs.4 The British Association of Otorhinolaryngologists—Head and Neck Surgeons (BAO-HNS) were actively involved in the NPTA and supported the development and dissemination of the guidance. It wrote to its members cautioning against the excessive use of diathermy and encouraged surgeons to continue to support the Audit.5 However, the guidance was potentially controversial because the popularity of diathermy had increased over the past 20 years and had become the most widely used technique.6
To date, there is little evidence that NICE guidance has been effective in changing surgical practice.7–9 In this paper, we describe the impact of the interim NICE/BAO-HNS guidance on the use of diathermy and the rate of postoperative haemorrhage after tonsillectomy in English NHS hospitals.
METHOD
The study used data extracted from the Hospital Episodes Statistics (HES) database and data collected by the NPTA. While information on the number of tonsillectomies and complications was available in both HES and NPTA datasets, the HES dataset was used to examine postoperative haemorrhage rates because it provided a longer time frame for analysis than the NPTA dataset and because it provided more complete information about the occurrence of these complications. Information on which surgical technique had been used for a tonsillectomy was only available in the NPTA dataset.
Records of tonsillectomies performed in NHS trusts were extracted from the HES database for the period between January 2002 and December 2004. Tonsillectomies were defined as records with a bilateral tonsillectomy code (OPCS-4 F34.1-4) in any of the first four procedure fields. Patients were excluded if other diagnosis or procedure codes suggested the operation would have been excluded from the Audit (see below). Rules for defining a haemorrhage were developed in the following way. First, any readmission within 28 days was linked to its index tonsillectomy using the HES identification number. Diagnosis and operation fields in both the initial admission and any readmission were then examined for codes whose definition and frequency of use probably indicated a postoperative haemorrhage. On this basis, postoperative haemorrhage was defined by ICD-10 codes T81.0 “Haemorrhage and haematoma complicating a procedure” and T79.2 “Traumatic secondary and recurrent haemorrhage” or OPCS-4 F36.5 “Surgical arrest of postoperative bleeding from tonsillar bed.”
Data for consenting patients were submitted to the Audit by participating hospitals from 7 July 2003 until 30 September 2004 (full details have been published elsewhere10). Any patient undergoing a tonsillectomy during this period was eligible for inclusion unless the procedure was a tonsillectomy revision or a tonsillar biopsy, was for confirmed cancer, or was undertaken with palatal surgery. Data were collected on patient characteristics, the surgical operation and subsequent complications. A postoperative haemorrhage was defined as any bleeding that led to delayed hospital discharge, blood transfusion or return to theatre during the initial stay or that led to readmission to hospital within 28 days of surgery.
Tonsillectomies were grouped according to the techniques used for dissection and haemostasis. Cold steel dissection with the use of ties/packs was the only category not involving a “hot” technique. Two separate categories were defined by the use of monopolar or bipolar diathermy for haemostasis after cold steel dissection. Further categories were defined by the use of a “hot” technique for both dissection and haemostasis, namely monopolar diathermy, bipolar diathermy (forceps or scissors) and coblation. All other combinations of techniques were categorised as “other.” The relative risks of tonsillar haemorrhage associated with these techniques4 are shown in table 1.
The HES-derived haemorrhage rates were analysed using interrupted time-series analysis. This involved fitting a linear regression model to detect a change in the level or trend after the guidance was issued and an autoregressive moving average (ARMA) model to account for any autocorrelation in the regression residuals.11 Haemorrhage rates were expressed as percentages and are presented with 95% confidence intervals (CI). The analysis of tonsillectomy techniques using NPTA data covered the interval between 1 August 2003 and 30 September 2004 to exclude the start-up period of the Audit. Finally, the proportion of eligible tonsillectomies reported to the Audit was estimated by comparing the number of operations submitted to the NPTA by each trust with the numbers recorded in HES during the period the trust was registered with the Audit.
RESULTS
There were 112 092 tonsillectomies recorded in HES between January 2002 and the publication of the guidance in March 2004, and 37,986 tonsillectomies in the following period. There were typically between 1700 and 2200 operations per fortnight, except for a drop around Christmas each year. The rate of postoperative haemorrhage increased from 2002 by 0.5% per year (95% CI 0.3% to 0.8%), reaching 6.4% when the guidance was published (fig 1). After the publication, the rate of haemorrhage dropped by 0.7% (95% CI −1.3% to −0.0%), and the increase over time was no longer apparent (estimated slope −0.4% per year, 95% CI −1.7% to 0.8%).
The NPTA received information from 122 of 135 eligible NHS trusts (90%). Trusts submitted details on 14 193 consenting patients between 1 August 2003 and 23 March 2004. Details of 11 914 patients were received in the period after the NICE guidance was published and 30 September 2004. The pattern of tonsillectomy techniques in the two periods is described in table 1. There was a 10% decrease in the use of bipolar diathermy as the dissection tool, an increase in the use of cold steel for dissection, using either bipolar diathermy (9%) or ties/packs for haemostasis (4%), and a decrease in the use of monopolar diathermy. The direction of change was consistent with the interim guidance in each case, and the change occurred directly after publication (fig 2). The differences between the two distributions of techniques were statistically significant (chi-squared test, p<0.001).
Table 1 also gives the total number of procedures recorded in both data sources between August 2003 and September 2004 in participating NHS trusts. Despite the encouragement of the BAO-HNS for continued participation, the proportion of tonsillectomies submitted by trusts fell from 60% to 51% after the guidance was published. The proportion of complications submitted by trusts also fell after publication. The observed haemorrhage rates in the pre- and postguidance periods in the NPTA data were 4.2% and 2.9%, respectively. Between August 2003 and September 2004, the HES-data gave pre- and postguidance haemorrhage rates of 6.2% and 5.9%, respectively.
DISCUSSION
The combined analysis of HES and NPTA data suggest that the NICE/BAO-HNS interim guidance influenced clinical practice and produced a modest improvement in patient outcomes. The shift in the use of surgical techniques was observed immediately after publication and was consistent with the guidance recommendations. A fall in the HES-derived haemorrhage rate was also observed directly after publication, and the underlying rate of postoperative haemorrhage was also estimated to stop increasing. Since the study was based on a “before and after” design, we cannot rule out the possibility that factors other than the guidance caused surgeons to modify their practice. However, the speed and the timing of the observed change identify the guidance as the most likely influence. An independent survey of BAO-HNS members also reported that some surgeons had changed their practice in light of the guidance.12
The study suffers from various limitations. First, the general standard of clinical coding entered in HES is known to differ between hospitals, in both completeness and the appropriateness of codes selected, and will also change over time.13 In this study, completeness is unlikely to be an issue because tonsillectomy is a common procedure with a specific OPCS-4 code. We aimed to minimise the effect of inconsistencies in coding practice by identifying patterns among diagnosis and operation codes within the records of admissions associated with the tonsillectomy or that occurred within 28 days. Changes in coding practice over time in HES data might potentially affect the observed trend in complications but are unlikely to have affected the change observed when the guidance was published.
Second, the incompleteness of the NPTA dataset, and the slight fall in case-ascertainment after the guidance was published, raises the possibility that the observed shift in surgical technique was due to a change in the type of tonsillectomies submitted to the Audit. Selective notification of tonsillectomies to the NPTA would be expected to affect the distributions of other variables, but there were no substantive changes between the two periods in the distribution of patient characteristics or treatment factors such as the proportion of operations performed by junior surgeons. The average age and sex of patients in the two data sources were also similar. Therefore, the magnitude of this confounding is likely to be small compared with the observed shift.
Finally, the number of complications was under-reported by hospitals, and the level of under-reporting increased after the guidance was published. Selective reporting of outcomes might have led to the haemorrhage rate of each technique being inaccurately estimated, thus weakening the proposed link between the surgical technique and postoperative haemorrhage. However, the haemorrhage rate in the cold steel & ties/packs group would need to be three times higher to remove the observed difference between it and the diathermy techniques, a difference of sufficient magnitude to suggest selective under-reporting is not responsible. Moreover, risk models were used to adjust for potential confounders from both the full dataset and a subset of 55 trusts that had similar numbers of initial operations and readmissions observed in the NPTA and HES data.10 This sensitivity analysis found that the pattern of relative-risks associated with the different techniques remained stable.
Previous research has cast doubt on the effectiveness of NICE guidance as an instrument to improve surgical practice. In a study using various methods, Sheldon et al examined the impact of the NICE guidance on hip prostheses and laparoscopic surgery for colorectal cancer.7 NICE had advised surgeons to use hip prostheses that met benchmark replacement rates, and to use an open rather than laparoscopic technique for resection of the colon. Neither was found to affect surgeon behaviour.7 Sheldon et al and an early study also found the guidance advising against laparoscopic surgery for hernia repair had no discernible impact on the proportion of operations undertaken using this technique.78 An instance where the national distribution of evidence-based guidance appeared to affect practice was in the treatment of persistent glue ear.1415 An already declining rate of surgery accelerated after the publication of the 1993 Effective Health Care Bulletin. Its influence was judged to depend on favourable conditions, such as a growing skepticism about the value of surgery for glue ear among GPs, surgeons, purchasers and parents.15
It is recognised that the uptake of clinical guidelines is influenced by many variables.71617 Effective implementation depends upon clear recommendations, an accurate reflection of the complexity of the clinical area, a strong, undisputed evidence base, the availability of adequate funding, and the support of professional bodies. The uptake of the tonsillectomy guidance may have been helped because a number of these were met requirements. First, the recommendations on the use of diathermy were specific and clear, although it was left to each surgeon to determine what level of diathermy was “excessive.” Second, surgeons could change from using diathermy throughout the operation to just using it for haemostasis with minimal disruption or cost. Third, the guidance was actively supported by the surgical specialist association, and the recommendations about practice were reiterated in the letter to members of the BAO-HNS.5 This support was underpinned by the involvement of its Comparative Audit Group in the NPTA. Fourth, the NPTA provided NICE with timely evidence, and the results were derived from the largest prospective study on tonsillar haemorrhage at that time. This may have given the evidence base of the guidance legitimacy among the ENT community. Finally, the issue of postoperative haemorrhage had been of concern among ENT surgeons for a number of years prior to the guidance18, and the NPTA, with support for it from the BAO-HNS, served to ensure the issue received regular coverage. However, in opposition to these probable enablers, several barriers to implementation are notable. In contrast to the guideline on surgery for glue ear, whose effectiveness was being increasingly doubted, the popularity of diathermy has been rising. In addition, surgeons have a large degree of autonomy and may believe that changing from their preferred technique, given their experience and training, will result in worse performance.
From the various enablers discussed above, it is clear that the NPTA had a prominent role. As such, it raises the question of whether integrating the publication of NICE guidance on surgical practice with national clinical audits would be a generally effective method of increasing compliance. Further research would be needed to assess this. In particular, the likely gain in the level of implementation must be weighed against the cost and burden on hospital staff of a national audit. Moreover, the publication of the guidance was a mixed blessing for the NPTA, given the decrease in case ascertainment that was observed after the guidance was published.
Linking NICE publications and national audits would also allow for more complete and reliable evaluations of the impact of surgical guidelines when compared with using routine data alone. Routine data sources may only allow a limited number of process or outcome indicators to be defined, and they may contain insufficient variables to adjust for known confounders. In this study, routine data enabled us to derive changes in outcomes (haemorrhage rate), while the NPTA data enabled us to derive changes in process (surgical technique). Greater analytical power would be gained if the data could have been linked at a patient level, but due to issues associated with protecting the confidentiality of patient data, this could not be achieved.
Acknowledgments
HES data were supplied by The Information Centre for Health and Social Cave. We thank A Luton and M Cuthbertson for the development of the web-based data entry system. We thank the staff of all the hospitals in England and Northern Ireland who contributed data to the National Prospective Tonsillectomy Audit. The project team and steering group is especially grateful for the many valuable insights and contributions from Rowena Ryan, both to the NPTA and to the ENT community. Rowena sadly died before this paper was finished. NICE published its final guidance on tonsillectomy in December 2005, and this is available at http://www.nice.org.uk.
Footnotes
This Audit was carried out by the Comparative Audit Group of the British Association of Otorhinolaryngologists—Head and Neck Surgeons and the Clinical Effectiveness Unit of The Royal College of Surgeons of England—London School of Hygiene and Tropical Medicine.
Project Team: P Brown, Milton Keynes General NHS Trust, Milton Keynes; R Ryan, Northwick Park Hospital, London; M Yung, Ipswich Hospital NHS Trust, Ipswich; J Brown, L Copley, D Cromwell, J Horrocks, J Lewsey, D Lowe, A Luton, J van der Meulen (chair), Clinical Effectiveness Unit, The Royal College of Surgeons of England and London School of Hygiene and Tropical Medicine, London, UK.
Steering Group: R T. Ramsden (chair), B Bingham, N Black, S Dixon/L Shurlock, L Flood, M Haggard, S Ludgate/J Hopper, J Oates, J Shotton, A Tomkinson, J Toner, J van der Meulen, G Weiner and P Woods.
P Brown, R Ryan and M Yung initiated the Audit; J van der Meulen, J Browne, P Brown, R Ryan and M Yung developed the protocol for the Audit; D Lowe coordinated the data collection supported by L Copley and J Horrocks; J Lewsey, D Cromwell and L Copley analysed the data, supported by D Lowe and J van der Meulen; D Cromwell and J van der Meulen wrote the manuscript with contributions from all Project Team members; the Chair of the Steering Group and N Black commented on the penultimate draft.
Funding: The Audit was funded by the Department of Health (England) and the Department of Health, Social Services and Public Safety (No rthern Ireland). Both departments had no role in study design, data collection, data analysis and interpretation, the writing of the report, or the decision to submit the paper other than by being represented in the steering group. J van der Meulen is supported by a National Public Health Career Scientist Award, Department of Health—NHS R&D Programme UK.
Competing interests: None.
Ethics approval: The Audit was approved by the Northern and Yorkshire Multi-Centre Research Ethics Committee.
Linked Articles
- Quality lines