Problem

Within the first few weeks of a placement as a GP trainee at Weston General Hospital (WGH) I was struck by how those junior to myself, F1s, and F2s were reluctant to report safety related incidents or hazards. On one occasion a consultant specifically asked us, as a group on a ward round, to report the fact that we had not followed hospital protocol regarding a patient with spontaneous bacterial peritonitis. When I began to fill in the "Datix" report (the existing computer based system used at WGH and many other hospitals for reporting significant events) the F1 asked “Why are you doing that, you will get x (the registrar) into trouble.”

On another occasion a different F1 came up to me while I was filling in a report and asked, worriedly, “That’s not about me, is it?” Junior doctors appear to perceive the Datix system with suspicion, more as a disciplinary tool rather than one promoting an open culture toward patient safety. This view is echoed in a recent article [1] by James Titcombe (safety advisor to the Care Quality Commision) who wrote that a phrase that has emerged within NHS staff is "I'll Datix you", used as a thinly veiled threat.

A team of doctors and quality improvement specialists from WGH got together to explore the problem. The team included Dr Andrew Bell (consultant gastroenterologist), Dr Tricia Woodhead (consultant radiologist), Dr Julian Abel (consultant palliative care), Dr Harriet Treacey (F1), Dr Sarah Johnson (GP specialist trainee) and the WGH QI Hub team (an office of two staff dedicated to promoting and supporting QI work).

Weston General Hospital is the smallest acute Trust in the UK. It serves a resident population of 212 000 plus 3.3 million day trippers and 375 000 staying visitors a year. The hospital has struggled to recruit medical staff and is currently undergoing an NHS only acquisition process from a neighbouring Trust.

Background

Hazard and incident reporting have a long history in the aviation industry as well as engineering and industrial production. Initially it was a way of troubleshooting production line processes but aviation took it to a new level with an emphasis on passenger and crew safety. What is clear is that although the importance of error recognition and incident reporting in healthcare surfaced as long ago as the 1990s with the work of Leape[2] and Reason,[3] the NHS is still playing catch up. Thus far take up is largely limited to retrospective significant event reporting using computer systems such as Datix. Alongside a lack of provision of quick and easy hazard reporting systems, there is also a cultural reluctance of doctors to report.[4]

Inspiration for this quality improvement project came, in part, from the work of Terry Fairbanks, director for the National Centre for Human Factors Engineering in Healthcare, USA. In particular, he trialled an anonymous touch screen hazard reporting system at two hospitals in the USA which increased hazard reports from doctors in one hospital from two a year to 75 a month[5] and in the other from 41 to 210 a month.[6] A revised computer based system similar to Fairbank's was considered at WGH but was deemed to be too complex and expensive an undertaking for a trial. In addition the project team thought that a phone system would be quicker and easier to use and also help with anonymity.

Terry Fairbanks succinctly sums up the problem at WGH (Weston General Hospital) and indeed at most hospitals anywhere in the world: “Problem number one, we’re not getting enough reporting, problem number two, we’re not doing anything with the near miss reports.”[7] Fairbanks believes hospitals are too reactive and that a good hazard reporting system will help hospital leaders be more proactive.

Baseline measurement

The first step was to find data to support or refute the proposal that doctors were under reporting at WGH. It quickly became apparent that doctors were making very few Datix submissions, an average of three per month throughout the entire hospital. Administrative staff were making nearly twice as many incident reports as doctors, and nurses were making nine times more reports than doctors (table 1). Proportionately these reporting rates cannot be explained simply by the fact that there were more nurses and admin staff (In August 2014 WGH employed 210 doctors, 458 nurses and 410 administration/clerical staff). Nor was there any evidence that doctors were delegating reporting to nurses or administration staff.

Two busy general medical wards were selected to trial the telephone hotline. They were selected because two project members worked on or were familiar with the wards and their staff and it was felt this may make it easier to educate staff there about the hotline. A snapshot of reporting on these two wards showed that in April and May 2014 four Datix submissions had originated from doctors on ward 1, and none from ward 2. One of the reports was from an F1 doctor, three from F2 doctors.

In July 2014 a small (n=23) written anonymous qualitative survey of doctors (all grades) was carried out to explore current attitudes to reporting (Figures 1 to 7). From these results it was proposed that making a system that was easier, quicker and anonymous, compared to current "Datix" system, was likely to result in an increase in doctor reporting rates. The aim was to increase reporting rates by doctors by at least 100%.

The majority of questions were bipolar semantic differential scales but respondents were also asked specific questions regarding their reporting behaviour over the past three months and also given the opportunity to add comments in a free text box.

Three doctors (13%) said they had submitted at least one Datix report over the preceding three months. Eighteen (78%) doctors said they had noticed hazards or incidents but had not reported them. Of these doctors on average each doctor had noticed, but not reported, three incidents or hazards over a period of three months (figure 2) (an extreme outlier of 90 incidents noticed but not reported was removed from analysis as skewing results).

Survey respondents were given the opportunity to add comments, a selection is given below:

- F1 “Datix forms are long and complex and don’t come to anything. I have started them and not finished”

- F2 “Most ‘Datix’ ignored”

- CT1 “Often have to stay late to complete as they take a very long time, at least 30 minutes”

- F1 “I have reported incidents in the past with no feedback, so feel there is no point”

- F1 “I got several consultants very annoyed by Datixing incidents when I started. Have not rushed to repeat the experience”

- F2 “I have little faith in system being changed”.

See supplementary file: ds5361.docx - “Table 1, figures 1 to 7, figure 8 and 9”

Design

The idea of trialling a phone hotline for reporting was disseminated to senior staff (consultants, nursing managers, ward sisters, management) through a presentation at the monthly WGH governance meeting and through smaller informal discussions with ward nurses and doctors. Posters (figure 8) were put up around each of the wards.

At WGH it was not practicable to introduce reporting specific computers or apps as had been trialled elsewhere, so the decision was taken to buy a single answerphone and to locate it in the QI Hub office. Doctors on two general medical wards were asked to phone the answerphone over a period of 12 days (July 28th to August 8th, 2014) to report hazards, incidents or raise an issue of concern. Reports could be submitted 24 hours a day, seven days a week.

The study team agreed that it was essential to feedback to reportees regarding what the response to their reports had been and this was supported by the qualitative survey (figure 5). If reportees gave their names on the answerphone message feedback could be given in person by the consultant of the day or Hub team, if not then a summary sheet of feedback, regarding all reports from the past seven days, was placed on the two wards each Friday. This was located near the pod system so staff would get used to knowing where to find it.

In the context of this study “hazards” include incidents and near misses but not serious incidents, which should continue to be reported by the Datix system. A driver diagram (figure 9) was created to plan the study.

Strategy

Because this study involved such a safety critical aspect of the hospital's processes it was decided to effectively run contiguous PDSA cycles. This means that problems were rapidly studied as they arose and changes made quickly if necessary.

PDSA cycle 1: Each morning the QI Hub team transcribed the reports from the answerphone messages and categorized them into Red (high risk), Amber (medium risk) and Green (low risk) according to perceived risk level. Every report was then reviewed by a consultant of the day (from volunteer consultants each taking a day or two each). The consultant then actioned the reports. It soon became apparent that the problems being reported were often complex, involving multiple operational teams. Sometimes it wasn't even clear which team or teams were responsible. Because of this a "response protocol" was quickly developed (figure 10) so that all the consultants responded in a methodical and reproducible way. The protocol acknowledged that it would not be possible to fix all problems immediately. Some were complex and would need more time and possibly a further QI project to fix.

Other reports exposed systemic weaknesses. In these cases any immediate safety issues that could be addressed were mitigated, but otherwise the problem would be referred up to the appropriate existing management structure such as the medical director, clinical advisory group, or local governance team. This change obviously did not affect reporting rates but rather how efficiently the reports were dealt with downstream. Over the short term it was not possible to measure whether this resulted in an increased number of reports although in the long term one could predict that effective resolution of problems would encourage reporting.

PDSA cycle 2: The complete report list was reviewed by the consultant of the day usually mid afternoon of the following day. On two occasions the QI Hub team discovered reports of issues which they thought could not wait until the afternoon consultant review and therefore rang the consultant for advice immediately. It was decided therefore that the reviewing consultant was effectively needed to be "on call" for the hotline and available on the end of a phone for the entire day. Again, this change obviously did not affect reporting rates but rather how efficiently the reports were dealt with.

PDSA cycle 3: Four days into the study and disquiet rose as some managers felt unsettled by the presence of a new reporting system. Clearly having two reporting systems co-existing, albeit briefly, did raise important safety issues and the study team were therefore careful to ensure that where appropriate reports were forwarded to the existing Datix system. However it became clear that senior nursing staff wanted to make the decision themselves about which cases warranted a "Datix", so it was agreed that a summary of all phone reports would be sent to senior nursing staff on a daily basis, rather than weekly as previously agreed. This change did not affect reporting rates directly but clearly a ward culture which is against any particular reporting system would mitigate against its use.

PDSA cycle 4: Between three to seven reports were received each day during the first five days of the new system being in place. However, on days six and seven no reports were received and only five over the following five days. Days six and seven, when no reports were made, were a weekend. This suggests that either there were fewer problems at weekends or doctors were too busy to report. The general falling off of reporting rate we believe was due to the annual changeover of junior doctors that happened on August 4th (day 8 of the study). Although we noted the falling off of reporting rates from day six and the previously unforeseen problem of doctor changeover, unfortunately we did not have the manpower or time in our study group to address the problem by educating the new cohort of doctors about the phone line. Overall this decline in reporting rates at doctor's changeover suggests that education and awareness play an important role in reporting.

Results

The reporting hotline was theoretically operational for 12 days, however the annual junior doctor changeover happened toward the end of the trial period and very few reports were received after the changeover because the new junior doctors were unaware of the trial. The effective trial period was therefore July 28th to August 4th 2014 (eight days). During this eight day period 25 reports were received from the two wards, compared to four reports over the previous two months. This is an increase of 525%, or a greater than five fold increase. The reports can be further broken down as follows (tables 2 to 7).

While this trial was aimed at just two wards at WGH, doctors from other wards visiting these wards, or seeing the posters, used the system to report problems elsewhere in the hospital. This suggests that there is a demand for, and enthusiasm for, using the new service.

The qualitative survey suggested that anonymity would be an important factor in increasing reporting rates for some doctors, but not for others. The response was bipartite (figure 5). However, evidence from the trial itself (table 3) shows that only three doctors (making four reports) were willing to identify themselves (and two of these doctors were involved in the study team). It can be concluded that anonymity is an important incentive to increase reporting rates. The disadvantage of anonymity is that if further information was needed it would have been difficult if not impossible to obtain.

Most reportees gave their grade simply as “doctor” so it is not possible to be sure how many of each grade made reports. It is unlikely that many, if any, were above senior house officer (SHO) level. This is at one level perhaps to be expected because most day to day ward work is undertaken by juniors. On the other hand the higher the grade the potential for harm arguably rises as grade rises. Just as importantly junior doctors need the support and example of seniors to report and embrace an open reporting culture.

The types of problems reported were very diverse (table 5), ranging from low risk, “There are no butterfly needles (on the ward)” to five Red high risk issues. The process of investigating some of the high risk reports revealed some extraordinary information. For example, there were four reports about the pod system (a vacuum device which transports blood samples, drug charts and test requests through tubes throughout the hospital). There were either not enough pods or pod system not working. There were also three separate reports about missing or delayed return of drug charts from pharmacy (via pod system). Once the consultant of the day had investigated, facilities revealed that they believed there might be a hole, or multiple holes, in the pod system and that drug charts may be being sucked into ceiling voids and lost. The reporting system had unearthed a problem that, thus far, the parties involved (facilities, pharmacy, doctors, nurses), had not been fully aware of and which posed significant risks.

One report was about a drug chart which had been in pharmacy (or a hole in the pod system?) for two days, a new chart was then written, but the old one turned up and with two charts being used there was a risk of the patient being given drugs twice. The investigation process also revealed that it was not clear which department was responsible for the maintenance and performance of the pod system.

The four other Red high risk reports concerned patients not having drug reconciliation on admission and being discharged with incomplete prescriptions; an IT problem which meant discharge summaries were not automatically sent to GPs and ward teams were not clearly made aware of the problem; a cardiac arrest trolley with defibrillator pads which did not work during an arrest and all ABG machines in the hospital being out of order at the same time. None of these significant risks were picked up by the current Datix system.

See supplementary file: ds5363.docx - “Fig10, Tables 2-7”

Lessons and limitations

The qualitative survey was limited by relatively small numbers (n=23) and the trial period, just eight days, was short. The phone line was only open to doctors because their reporting rates were significantly lower than other health care professionals. Future studies need to extend to use to other groups of staff.

One of the key features of this intervention was how it enabled staff to raise a concern about a potential hazard rather than report an actual incident (which is the main thrust of the current Datix system). This is relevant in light of Bird’s [8] old but famous study which shows that there are 600 missed opportunities before one fatality.

During the study one manager commented, "I am very keen that we do not remove the ward sisters' responsibility and accountability for managing safety on their wards." This view raises many questions regarding where responsibility for safety lies in hospitals.

Others expressed concern that the hotline was exclusively for doctors, feeling that nurses were being excluded. The rationale for the study concentrating on doctors initially was that the evidence showed that doctors were nine times less likely to report problems compared to nursing staff. In addition the idea was always that the system would work best if, eventually, all staff including nurses, doctors, porters, lab technicians, etc were included. Another manager expressed concern that they had not been made aware of the study. Although attempts had been made to explain the project and it's aims widely prior to it starting, in retrospect this important communication task could have been done more thoroughly.

The outcomes of this study are potentially sustainable but it would be important to unpick in further studies which factors are the most important in increasing reporting rates. While the qualitative survey suggested anonymity, ease of use and speed of system were all individually important the fact that there was a significant fall in reporting at the time of doctor changeover suggests that education and awareness of reporting, especially at the time of handover, is another important factor. However, the Datix reporting rate during the trial was exactly the same as the year before (six Datixes were put in by doctors from 28.7.14 to 8.8.14, the same amount as the exact time period in 2013) suggesting that the anonymity, speed and ease of the hotline were more significant than education and awareness. Sir Robert Francis has recently publically reiterated a need for the NHS to embrace open and transparent incident reporting.[9] While this author believes that should be the ultimate objective, it may be that anonymity is a pragmatic and necessary stepping stone toward a full change of culture.

Conclusion

The trial of the reporting answerphone demonstrated that if reporting was made easier, quicker and anonymous then reporting rates increased exponentially. The trial was on two wards for doctors only but there was no reason to believe it would not work equally well for all staff. To explore this theory, it has been decided to trial the system on more wards and for all staff not just doctors.

Making change is difficult and not always welcome. As the study progressed it became clear that the prospect of receiving many more safety reports, especially from doctors, was not universally welcomed at WGH. This may have been because of the perceived increased workload, financial implications or increased involvement of doctors in what some saw as the nurses' domain. These wider cultural and procedural issues would need to be explored further in future studies.

Ideally though the next fundamental step is to ensure there is a system in place, and a will to act on the information reported in by staff. This depends on all managers (from IT to cleaning, porters to pharmacists) buying into the quality improvement ethos and accepting that the reports are to be welcomed as a means of improvement for all (patients and staff), rather than a form of disciplinary action or method of “feeding” national safety databases. The hospital must be also be willing to make change based on the reports or they have no value.

This author believes that the five red (high risk) reports alone make the case for a new reporting system for doctors at WGH urgent and irrefutable. The new system should be quicker and easier than the current system, but also offer anonymity. This study has proved that we can obtain the information, which was previously invisible, necessary to improve WGH. Now it is for the managers and leaders to establish a systematic and sustainable way of using that information to make things better.

References

1. Titcombe J. Transform the culture of fear into a culture of learning. Health Serv J. July 1st 2015.

2. Leape L. Error in medicine in Rosenthal, M., Mulcahy, L. and Lloyd-Bostock, S. (eds.) Medical Mishaps, Buckingham: Open University Press. 1999.

3. Reason J. Managing the Risks of Organizational Accidents, Aldershot: Ashgate. 1999.

4. Waring J J, Beyond Blame: cultural barriers to medical incident reporting. Soc Sci Med 2005;60(9):1927-35.

5. Fairbanks RJ. Acad Emerg Med 2009 (16:4) S245.

6. Ryan A, Fairbanks RJ, Sample G, Lee B, North-Saale J, Barton N. Use of a hazard reporting with user centered design and weekly feedback increases staff participation. 2012 Annual Congress: Society of Critical Care Medicine (Abstract), Houston, Texas.

7. Fairbanks T. Speaker. Was it Really a Miracle on the Hudson? Aviation meets healthcare safety (NPSF, May 2012).

8. Bird F Jnr. The Accident Triangle, 1969.

9. Francis R. Beyond blame: cultural barriers to medical incident reporting. Feb 2015. https://www.gov.uk/government/publications/sir-robert-francis-freedom-to-speak-up-review.

Declaration of interests

Dr Andrew Bell and Dr Sarah Johnson are both members of this study team and doctors on Berrow ward, one of the trial wards. Dr Johnson made two reports to the hotline in her capacity as an SHO on Berrow ward. Dr Harriet Treacey was a part of this study team and also made a report to the hotline.

Acknowledgements

Dr Andrew Bell, Dr Tricia Woodhead, Dr Julian Abel, Dr Harriet Treacey, the QI Hub team at WGH.

Ethical approval

This project was deemed an improvement study and not a study on human subjects and therefore ethical approval was not required.