Article Text
PDF
Abstract
Reports of adverse events and near-misses provide the opportunity to learn about latent (systems) errors. However, voluntary incident reporting systems are underused by physicians. While reports submitted by nursing staff relate to common hazards such as medication administration or falls, physicians have broader exposure to patients’ entire hospital journey. Reports by physicians have the potential to uncover more serious errors that could span multiple departments and layers of personnel. Organisational safety culture thrives when all staff are represented and feel empowered to share safety concerns.
At the South Health Campus (SHC) Hospital in Calgary, Alberta, Canada, the baseline proportion of physician-submitted reports within our site’s Reporting and Learning System (RLS) from July 2013 to December 2016 was 1.12%. We implemented an intervention to double the proportion of physician-submitted RLS reports, using quality improvement methods.
Focus groups identified lack of experience with the RLS system, lack of feedback or closure after an RLS submission, and apprehensions about disclosing the incident to the affected patient as barriers to physician submission. Accordingly, the intervention involved direct responses from physician leadership to each physician-submitted RLS report, multimedia demonstrations of efficient RLS submission to physician groups and medical learners, and linkage to materials on safe disclosures. Effectiveness was assessed using a controlled before-and-after design, comparing SHC with the rest of Calgary and with the rest of Alberta.
Following the intervention, the proportion of RLS reports that were physician submitted increased to 2.65% (OR 2.42 [95% CI 1.96 to 3.02], p<0.001), sustained over the following 4 years. While an increase was observed for the rest of Calgary, it was smaller (OR 1.27 [1.15 to 1.40], p<0.001). A decrease in the odds of physician submission was observed for the rest of Alberta. Differences between sites were significant (p<0.001).
Overall, we found that physician-submitted incident reports can be increased and sustained over time if submitters receive personalised feedback by a physician safety leader. At our site, reports submitted by physicians have been valuable in uncovering complex systems issues that may not have been readily apparent.
- incident reporting
- adverse events, epidemiology and detection
- patient safety
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Voluntary incident reporting systems are underused by physicians.
WHAT THIS STUDY ADDS
Physician-submitted incident reports can be increased and sustained over time if submitters receive personalised feedback by a physician safety leader.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE, OR POLICY
Reviewing all physician-submitted incident reports can be a valuable activity for an organisation’s patient safety leadership team. Increased awareness of the organisational risks and trends that are reported can lead to a more proactive, anticipatory approach to safety. It has also been our experience that the involvement of front-line physicians in finding solutions to complex, serious safety problems can contribute considerably to an organisation’s safety culture.
Problem
Alberta Health Services (AHS) is Canada’s largest province-wide, fully integrated health system, responsible for delivering health services to nearly 4.4 million people living in the province of Alberta. AHS is divided into five geographical zones, including North Zone, Edmonton Zone, Central Zone, Calgary Zone and South Zone.
The AHS organisation leverages several methods for monitoring and detecting patient safety problems. One tool is the Reporting and Learning System for Patient Safety (RLS), a centralised electronic incident reporting (IR) system which encourages any staff member to voluntarily report patient safety adverse events, close calls and safety hazards.1 This system is used to identify patterns, generate reports, and communicate information to both leaders and front-line staff, so that action can be taken to reduce harm. The North Zone was the first to pilot the electronic RLS system in March 2010, and it was sequentially rolled out to the other zones, with full provincial usage by March 2011. Prior to the RLS, paper-based systems had been used to capture patient safety events.
The South Health Campus (SHC) is the newest hospital in the Calgary Zone, opening in September 2012 for ambulatory care visits, with operational emergency department and inpatient units in January 2013. The vision, values and philosophy of this hospital are based on four foundational pillars, which include collaborative practice, innovation, patient-centred and family-centred care, and wellness. In addition, SHC was established with a strong emphasis on quality improvement and Patient Safety. The hospital’s Patient Safety leadership team has had a vision for strong physician engagement in patient safety among the approximate 200 attending physicians who are contracted to provide medical care at this facility. One mechanism to better engage physicians in safety culture was thought to be ensuring adequate physician representation in submitting RLS reports.
The number of RLS reports submitted by physicians (composed of attending physicians, resident physicians and medical students) was extracted and analysed on a monthly basis from the RLDatix (DatixWeb 14.0.35.1; Datix Ltd 2018) incident reporting software system since 2013. We selected July 2013 to begin tracking baseline data as the SHC hospital’s inpatient units were fully operational by that time. The baseline proportion of physician-submitted reports from July 2013 to December 2016 at SHC was 1.12%, while the baseline proportion provincially was 0.59%; the baseline proportions for the rest of Calgary and for the rest of Alberta were 0.62% and 0.59%, respectively (see online supplemental table for more detail).
Supplemental material
We aimed to increase the proportion of RLS submissions by physicians at SHC from the baseline rate of 1.12% to 2% or greater within a 1-year time frame. We selected the target of doubling the proportion of physician-submitted reports based on discussion among the Patient Safety team and hospital leadership and administrative stakeholders, as well as reviewing the literature on other organisations’ physician reporting rates.
Background
Receiving healthcare is not as safe as it could be. The Canadian Adverse Events Study found that 7.5% of adult patients admitted to a Canadian hospital were affected by an adverse event2 while the Canadian Pediatric Adverse Events study found an even higher 9.2% rate of adverse events in hospitalised paediatric patients.3 Identifying hazards, followed by reporting of these hazards to a database network, was one of the nine key ‘Principles for Action’ recommended by the National Steering Committee on Patient Safety, in their strategy for improving patient safety in Canada.4 WHO has also prioritised work to develop, manage and promote IR systems.5
Reports of adverse events, near-misses and other safety hazards provide the opportunity to learn about latent (systems) errors, so that effective solutions that address the identified deficiencies can be instituted. However, most voluntary IR systems detect only a small subset of events that healthcare personnel choose to report, thus lacking the ability to quantify the magnitude of each type of safety problem. Major sources of harm that are unreported may remain unrecognised. Another limitation of IR systems is that they are underused by physicians in many jurisdictions globally.6–8 Physician-initiated reports present a valuable perspective that can uncover different types of hazards not captured by reports from non-physician colleagues. For example, many reports submitted by nursing staff relate to common hazards such as medication administration or falls, occurring on a specific hospital ward where that nurse works; each isolated incident would be unlikely to warrant independent investigation.9 However, physicians have broader exposure to patients’ entire hospital journey, beyond one specific hospital ward or department. Therefore, reports by physicians have the potential to uncover more serious systems errors that could span multiple departments and layers of personnel. One group found that the types of safety incidents reported by physicians at their hospital were of greater severity with more serious consequences, compared with reports submitted by colleagues from a different health discipline.10
When a large, representative sample of hospitals in the USA was surveyed, 86% of these hospitals indicated that their attending physicians submitted few or no reports,6 and other organisations have similarly been affected by low rates of physician reporting.7 8 In a survey exploring barriers to using reporting systems, some reasons identified were that respondents had not encountered any significant problems or errors perceived as necessary to report, and respondents did not think their reports would effect change. When questioned on how to overcome barriers to reporting, 85% of respondents agreed that feedback should be provided on the action that is taken to rectify the problem(s) reported.11 Unfortunately, failure to respond to, and to act on, incident report submissions can lead to a domino effect, in which providers become even less likely to invest time in reporting incidents in the future.9
Measurement
The primary effectiveness outcome was the proportion of RLS reports that were physician submitted. RLS counts were generated directly from the RLDatix incident reporting software system, and separated into those submitted by physicians (including resident physicians and medical students) and those submitted by other healthcare professionals. Data were aggregated on a quarterly basis to provide near-real-time tracking of RLS submissions, plotted in statistical process control (SPC) P-charts using the QIMacros software program for visualisation.
The process outcome was the proportion of physician-submitted reports reviewed by a physician member of the SHC Patient Safety Team.
We used a controlled before-and-after design to determine the effectiveness of the intervention. The time period July 2013–December 2016 constituted the baseline period. Quality improvement activities started in November 2016 and have been sustained through the present day. We considered the entire period January 2017–December 2021 to be post-intervention. The proportion of RLS reports that were physician submitted was compared before and after the intervention using an OR as the summary metric of the intervention effect. The observed effect at SHC was compared with the rest of Calgary Zone and with the rest of the province of Alberta as concurrent non-intervention control sites. CIs and p values for each OR was estimated using Z-tests with a two-sided p value threshold of <0.05 for statistical significance. The presence of differences between sites was identified statistically using a Mantel-Haenszel test for heterogeneity. Analyses were performed using Stata 16 (StataCorp, College Station, TX).
The ARECCI (Alberta Research Ethics Community Consensus Initiative) screening tool determines the category of risk for project participants. Because the ARECCI score for this non-research QI project was within the ‘minimal’ risk category, we forewent formal scrutiny by a Research Ethics Board.
Design
To better understand the reasons for low physician reporting rates at SHC, the site Patient Safety Team conducted a focus group at a Site Leadership Council Meeting, with attendance by over a dozen Site Physician Leads for various specialties in November 2016. There was representation by the Departments of Emergency Medicine, Critical Care Medicine, Cardiac Sciences, Medicine, Family Medicine – Hospitalist Section, Neurosciences, Anesthesia, Obstetrics and Gynecology, Psychiatry and Surgery. Due to the nature of this project and the intricacies of the RLS system, it was not appropriate to involve patients or the public in the design, or conduct, or reporting, or dissemination plans of this quality improvement initiative.
The objective of the focus group was to discuss the RLS system and seek feedback on why physician report submission rates were low, as well as to brainstorm ways to increase physician submissions. Separate focus groups sessions were set up for specialties whose Site Leads expressed interest in further discussing this reporting initiative at their respective divisional meetings, attended by frontline physicians. These specialties included General Internal Medicine, Family Medicine Hospitalist physicians, Gastroenterology, Anesthesiology and Psychiatry groups, composed of approximately 50 physicians. Finally, a focus group was conducted with approximately a dozen Internal Medicine resident physicians, who had work rotations at SHC.
During these focus groups, examples of physician-submitted RLS reports were shared with attendees, to demonstrate the unique impact and value that these reports had had on collective safety learning. An example of an RLS ‘trending report’ was shared with the group, which highlighted some of the rich learning that could come from the system and which also revealed that a very small proportion of reports were physician submitted. Next, the focus group worked through standard survey questions including (1) awareness of the RLS system and how to access the online form, (2) personal experiences with submission of an RLS report, (3) barriers faced by physicians in submitting reports and (4) how to increase rates of physician report submissions. Open-ended discussion was encouraged. Following the discussion, we conducted the first Plan-Do-Study-Act (PDSA), in which there was an educational demonstration to the group on how to access the RLS Datix system to fill out a report, what types of cases should be reported within the RLS system (vs other mechanisms to address concerns unrelated to systemic issues, such as intrapersonal conflict) and guidance on what constitutes ‘good’ report content.
Through the focus group, two main themes emerged as barriers to physician-submitted reports. The first was that physicians either had limited experience with using the RLS system, or they perceived the process to be time consuming. During the demonstration, the Safety Team showed attendees that a ‘good’ report could be submitted in 5–10 min, depending on how much detail and content is included in the description field of the report form. The second, major barrier conveyed was that physicians who had submitted RLS reports in the past felt discouraged from continued use of the system because of lack of feedback following their prior report submissions. The physicians perceived lack of personal closure or resolution of their concerns and, in most cases, were unsure if their submission effected any positive systemic change. Finally, within the RLS report form, there is a mandatory field to be filled out on whether disclosure of the incident being reported had been provided to the patient. A small subset of physicians conveyed limited experience on providing disclosure to patients, and expressed reluctance to do so, due to fear of patient-initiated complaints or litigation.
The RLS system itself is managed provincially; therefore, it was not feasible for SHC address directly the concern raised that the reporting process was time consuming. However, providing feedback to physicians who non-anonymously submit reports was within the scope of what the SHC safety team could address. Providing demonstrations on how and when to use the RLS system to physicians was another straightforward intervention. Finally, from the sentiments expressed by a small group of physicians on inadequate training with respect to disclosure of adverse events to patients, further education in this area was also within the realm of what the SHC safety team could accomplish.
Strategy
Change interventions
PDSA #1 (November 2016–March 2017): This PDSA cycle addressed the barrier of limited knowledge of when and how to efficiently use the RLS system. The intervention consisted of a demonstration to attendees at a site leadership focus group on how and when to access the RLS Datix system to fill out a report. Subsequently, similar demonstrations occurred during divisional meetings for General Internal Medicine, Family Medicine Hospitalist physicians, Emergency Medicine, Gastroenterology, Anesthesiology and Psychiatry groups.
PDSA #2 (January 2017–current): This PDSA cycle addressed the barrier of physicians feeling discouraged from the lack of feedback after submitting reports. A Patient Safety team physician leader reviewed each RLS report submitted by a physician, and responded directly to the report submitter by electronic mail within 1 week of report submission, outlining the steps that would be taken to address the safety concern. After this change intervention, we received very positive feedback from front-line physicians, and we believe this intervention to have had the most impact on improving the proportion of physician-submitted RLS reports. This intervention has been sustained.
PDSA #3 (January 2018–current): This PDSA cycle addressed the barrier of limited knowledge of when and how to efficiently use the RLS system for medical learners. Educational sessions were held at the beginning of and at the end of the Medical Teaching Unit 4-week rotation for resident physician and medical student learners, facilitated by Attending Physicians with Quality Improvement training. There was discussion of the RLS system and a demonstration of when and how to submit an RLS report during these educational sessions. At the beginning of the 4-week rotation, data collected showed that 30–50% of medical learners knew how and when to use the RLS system; by the end of the rotation, over 90% of learners present indicated that they knew how and when to use the RLS system. This educational intervention has been sustained.
PDSA #4 (2018–2019): This PDSA cycle addressed some physicians’ limited experience with providing disclosure to patients. A ‘Disclosure Done Well’ video series was filmed by SHC Safety Team along with AHS Provincial Safety Team members in 2018 (available at https://www.albertahealthservices.ca/assets/info/hp/ps/if-hp-ps-qhi-qpse-disclosure-done-well-video-fac-guide.pdf). These videos were shared province-wide for general education on how to provide disclosure to patients.
Results
Since 2017, on a monthly basis, the absolute volume of physician-submitted reports has ranged from 8 to 36. In total, there were 383 physician-submitted reports at SHC during the 4-year post-implementation period from January 2017 to December 2021, and the SHC physician leadership team provided personalised follow-up responses for 100% of reports submitted non-anonymously. In cases where reports were submitted anonymously, but with the reporter’s job function role listed as Physician, a response was directed to the most applicable site Division Lead based on case details gathered within each report.
The proportions of RLS reports submitted by SHC physicians, Calgary Zone (CZ) physicians and all physicians provincially at baseline, from July 2013 to December 2016, were 1.12%, 0.62%, and 0.59%, respectively. Following the implementation of the change interventions at SHC from January 2017, the proportion of physician-submitted RLS reports improved at SHC to 2.72%, while CZ (excluding SHC reports) saw improvement to 0.79%. Provincially (excluding CZ reports), the proportion of physician-submitted reports decreased to 0.20%.
The odds of an RLS report being physician-submitted (as opposed to non-physician-submitted) increased 2.42-fold (95% CI 1.96 to 3.02, p<0.001) at SHC after the intervention. An increase in the odds of physician submission was seen in the rest of Calgary Zone, although to a lower degree (table 1). In contrast, the rest of Alberta experienced low proportions of physician-submitted RLS reports, and a decline in the odds of physician submission during the same before-and-after period. The Mantel-Haenszel test of heterogeneity was significant (p<0.001) indicating the presence of differences in ORs between sites.
Before-and-after data for physician RLS submission rates
A statistical process control (SPC) P-chart was generated from the SHC data pre-intervention and post-intervention. Following the improvement interventions in 2017, there was a significant change in the centre line and the upper and lower control limits seen in the SPC chart (figure 1).
SHC proportion of RLS reports submitted by physicians. RLS, Reporting and Learning System for Patient Safety; SHC, South Health Campus.
The increase in physician-submitted RLS reports at other Calgary Zone sites represents an unintended, but positive, effect from our intervention may. Approximately one-third of physicians who work at SHC also work at other hospitals in the CZ. Some of these individuals may have attended the initial RLS system demonstration sessions at the SHC site leadership focus group and/or during site divisional meetings. Once they received this training, their RLS reporting behaviours may have changed not only while working at SHC but also at other sites. In addition, medical learners who were provided with education on the RLS system move on to clinical rotations at other hospital sites within CZ after their 4-week clinical rotation at SHC. Therefore, it is plausible that these resident physicians continued to use the RLS system to report incidents encountered at other sites.
Lessons and limitations
We achieved a significant increase in the proportion of physician-submitted reports at SHC since 2017, when we introduced a multifaceted intervention involving direct responses from physician leadership to each physician-submitted RLS report, multimedia demonstrations of efficient RLS submission to physician groups and medical learners, and linkage to materials on safe disclosures. The increased proportion of physician-submitted reports at our hospital has been sustained for several years and integrated into our hospital’s Safety Culture.
One challenge for the Patient Safety team has been ensuring resources to manage the higher volume of cases reported. Fortunately, we have a very dedicated Patient Safety team that has been very committed to prioritising this RLS initiative. While there was straight-forward resolution in the majority of RLS cases, some complex and serious cases have required considerable time to thoroughly address. A number of cases required escalation to a formal organisational Quality Assurance Review or Patient Safety Review using AHS Systems Analysis Methodology,12 to evaluate the root causes of some of the problems uncovered. Those cases often required a collaborative problem-solving approach, and our site was able to engage some front-line physician RLS report submitters to become part of the review team, helping to first identify root causes and, subsequently, solutions for the same safety problems they had reported via the RLS system. Although the initial aim of this project was to improve our hospital’s safety culture by increasing front-line physicians’ use of the RLS system, inadvertently, some of these physicians have gained in-depth exposure to patient safety concepts and incident analysis skills, beyond what we had anticipated. The benefits of our quality improvement initiative in enhancing Safety Culture at our hospital have transcended the increase in the number of RLS submissions themselves.
Within the Calgary Zone, the proportion of physician-submitted RLS reports also increased, although not as much. One explanation for this improvement is that medical learners and some physicians based at SHC also work at other hospital sites in Calgary. We were unable to control for the effects of the educational component of the intervention beyond SHC.
For the rest of the province, as a whole, the proportion of RLS submissions by physicians have decreased. It is outside the scope of our current project to understand reasons for reduced physician use of the RLS system provincially. Perhaps there was more physician engagement and education on this system when it was first rolled out to all AHS sites in March 2011. Over time, there may have been fewer ongoing physician engagement initiatives provincially to encourage ongoing utilisation. A similar challenge at our site has been lack of dedicated RLS system training for new attending physicians who are recruited to work at SHC, which comprise a very small number annually (less than 5% physician turnover annually at our site). Work is underway to identify how we can best incorporate this training as part of the on-boarding of new physicians.
Another limitation of our study is that we did not perform a post-survey to follow-up on the initial survey and education provided during the physician focus group and divisional meeting sessions. Therefore, the possibility that other secular or temporal effects may be responsible for the improvement in physician submission of incident reports seen at SHC exists. However, during the education sessions conducted at the beginning and end of each 4-week rotation for medical learners, we did collect data on awareness of how and when to use the RLS system; by the end of the 4-week rotation, the majority of medical learners knew how to submit an RLS report. However, having knowledge of how and when to use the RLS system does not necessarily translate into encountering safety cases that should be reported or having the initiative to submit reports.
In addition, through the focus groups that were held with Physician Site Leads and Frontline Physicians, which were intended to help improve physician utilisation of the RLS system, we uncovered a more general gap in healthcare provider proficiency in disclosure of harm to patients, which led to the development of the Disclosure Done Well video series, which has been rolled out provincially for a broader purpose.
Conclusion
The proportion of physician-submitted incident reports can be increased and sustained if (a) physicians and medical learners receive education on how and when to use the incident reporting system, and (b) report submitters receive personalised case review updates and feedback by a Patient Safety team physician leader. At our site, we have shown sustainability over 4 years. Fortunately, we have strong Patient Safety team infrastructure and leadership, to ensure that these reports will continue to receive personalised review. At more resource-limited centres, it may be harmful for an organisation to undertake this project with insufficient resources to respond to the increased volumes incident report submissions, as report submitters could become discouraged and less likely to invest time in reporting incidents in the future.
We have shared our learnings with other hospital sites in the Calgary Zone, and work is underway at these sites to identify whether their Physician Safety leaders can successfully incorporate RLS report reviews into their existing Patient Safety strategy.
Reviewing all physician-submitted RLS reports has been a valuable activity for the SHC Safety team leaders. Becoming more aware of the organisational risks and trends has led to a more proactive, anticipatory approach to safety. It has been our experience that the involvement of front-line physicians in finding solutions to complex, serious safety problems has contributed considerably to the culture of at our site—and it all starts with incident reporting by physicians.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.
Ethics statements
Patient consent for publication
Ethics approval
The ARECCI (Alberta Research Ethics Community Consensus Initiative) screening tool was applied to this quality improvement (QI) protocol, to determine the category of risk for project participants, and therefore, the appropriate ethical review requirements. Because the ARECCI score for this non-research QI project was within the ‘minimal’ risk category, we forewent formal scrutiny by a Research Ethics Board.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors Conception and design: JP, KS, CD. Procurement of data: JN, TR. Analysis of data: DL, JN. Drafting of the original manuscript: JN. Critical review of the original manuscript: DL, TR, CD. Guarantor of the manuscript: JN.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.