Article Text
PDF
Abstract
Every day in hospitals around the world, millions of interspecialty referrals are made to obtain advice on the optimal care and management of patients. In the UK, the brunt of this work is undertaken by junior doctors with less clinical experience than the specialist colleagues to which they refer. A survey of 283 junior doctors revealed that colleagues were underconfident when making referrals and struggled to know which specialty to contact, how to reach the specialty and what clinical information to include in the referral. More concerningly, 10% of those surveyed had experienced bullying or belittling behaviours and verbal aggression from colleagues when referring.
The aim of this project was to design and implement a referrals toolkit for junior doctors to improve confidence making referrals and time to interspecialty advice, to improve patient care. Process mapping to understand the constituents of good referrals was combined with a failure modes and effects analysis describing how referrals fail to identify areas for intervention.
A specialty referrals guide with all specialty contact information was created at the trust, demonstrating an increase in junior doctor median confidence from 3/5 (n=20) to 5/5 (n=23) (p<0.001); 65% found it quicker to refer with the guide and 81% found an improved time to discharge. A referrals cheat sheet was also created, containing specialty-specific information to be included when making a referral. This has been downloaded over 23 000 times from around the globe. Of survey respondents (n=43), 74% noted improved confidence in making referrals, 26% noted faster time to specialty advice and 19% found a positive impact on patient discharges. Overall, the referrals toolkit has been beneficial for both junior doctors and the patients for which they care and has been accessed by over 50% of new foundation doctors in 2021 and 2022.
- Quality improvement
- Quality improvement methodologies
- Healthcare quality improvement
- Continuous quality improvement
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Quality improvement
- Quality improvement methodologies
- Healthcare quality improvement
- Continuous quality improvement
WHAT IS ALREADY KNOWN ON THIS TOPIC
Every day in hospitals all around the UK and further afield, millions of interspecialty referrals are made to obtain specialist advice by junior doctors who are underconfident in making referrals and struggle with knowing who to contact, how to contact them and what information to include in the referral. Concerningly, many have experienced bullying and belittling behaviours and aggression from colleagues when referring.
WHAT THIS STUDY ADDS
The Doctors Improving Referrals (DIRE) project designed and implemented a referrals toolkit for junior doctors, including a referrals information guide and a referrals cheat sheet, to improve confidence when making referrals and the time to interspecialty advice, and consequently to improve patient care.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Collaboration with junior doctors across the UK and the world to create, adapt and optimise the interventions has helped this project to achieve its aims and it is hoped that through continued publicity, the cheat sheet and referrals information guide will be adopted by more centres worldwide. The ultimate goal is to create a more cohesive and constructive environment for junior doctors in hospitals that promotes teamwork and a continual process of learning.
Introduction
Referrals form a significant part of the workload for junior doctors across all specialties.1 In the UK, referrals are commonly made online or via the phone using a traditional pager system, known colloquially as ‘bleeping’. The duty of making referrals often falls to junior doctors with less clinical experience, who have to refer to more senior colleagues for specialty patient advice. At our trust, it was observed informally that junior doctors faced significant challenges and obstacles in making referrals. They felt that referrals were often unsuccessful without due explanation as to why the referral was rejected, what further information was required and how they could improve the referral for next time.
Ealing hospital is part of London Northwest Healthcare National Health Service (NHS) Trust, a busy district general hospital (DGH) in greater London, that serves a diverse and comorbid local population. The rate of admissions often exceeds physical and staffing capacity and patients are admitted to general medicine wards rather than a specific specialty wards. This necessitates frequent referrals for interspecialty advice. Furthermore, many specialties are off-site at the tertiary centre and so there is often a need for interhospital referrals. This busy DGH environment is reflective of a typical NHS hospital and junior doctor workload.
The results of our local hospital survey demonstrated that junior doctors had low confidence in referring to other specialties, including 35% of respondents who were not-at-all confident or knew only the basics of how to refer. The most common reasons for this included not knowing how to refer to a specialty, what information to include and fear of referral rejection. Additionally, only 40% of those surveyed knew how to access referral information, which has been shown to introduce unnecessary delays in patient care.2
It was hypothesised that these issues with referrals were not only isolated to our trust but in fact a national problem with a significant impact on quality of patient care. The Doctors Improving Referrals (DIRE) project set out to survey junior doctors of all grades from trusts across the UK, to establish what barriers and challenges they faced in making interspecialty referrals and to see if these challenges impacted on patient care. The primary aim of this prospectively registered project was to design an accessible, reproducible and succinct referral toolkit which would lead to self-reported improvements in junior doctor confidence when making referrals, the time to interspecialty advice and achieve clinician reported improvements in patient care.
Background
Referrals constitute a considerable burden of work for junior doctors,1 on a background of an increasing workload—38.6% of junior doctors reported a ‘very heavy/heavy’ workload in 2019, and 45.2% reported that they worked past rostered hours on at least a weekly basis.3 Hence, improving the efficiency of information transfer between specialties via more accurate referrals would benefit junior doctors’ workload and morale. Furthermore, it has financial implications in terms of staying beyond rostered hours, which cost London North West Healthcare NHS Trust over £25 000 in fines in 2019 alone.4 Delays introduced by poor quality referrals may also prolong inpatient stay and has become a target of the NHS Reducing Length of Stay Programme,5 which estimates a cost of £800 million per year for delayed discharges (as estimated by the National Audit Office in 20166).
The ability to communicate effectively with colleagues, escalate appropriately and seek specialist medical advice are among the core skills listed in the General Medical Council (GMC) outcomes for provisionally registered doctors.7 Miscommunication between specialties has been shown to contribute to delays in patient reviews, procedures and discharges that negatively impact patient care.8 Furthermore, the GMC recognises that professional skills including making written and verbal referrals are an essential part of junior doctors’ daily schedules.1 More frequently, e-referral templates are being introduced to structure the information provided to the referral receiver and have reduced the time spent referring by ensuring important clinical information is included.9 E-referrals also enhance the visibility, transparency and reliability of interspecialty referrals.10 However, one way referrals via online templates inhibit the two-way flow of information between referral receiver and referrer. Moreover, when the attention of the specialist is required immediately, many trusts continue to support the pager system to directly contact specialties. This means that competence in making verbal referrals remains a core skill for junior doctors.
Baseline measurement
For this project to be representative of the cohort of junior doctors, it was decided to disseminate the baseline measurement survey nationally via MindTheBleep.com and using junior doctor Facebook groups. The survey was devised using a mixture of questions based on the five point Likert scale as well as free text responses. Figure 1 shows an excerpt from the online questionnaire, including an example of one of the Likert scales used.
An excerpt from the questionnaire disseminated to junior doctors, including an example of one of the Likert scales used. *response required
The national survey received 283 responses from a diverse and representative sample of centres across the UK. Responses were received from all grades of junior doctors, from foundation year 1 (FY1) to registrar; the majority of responses were from FY1 (146 respondents—51.6%), followed by foundation year 2 (FY2) (70 respondents—24.7%) and the remainder were from other senior house officer (SHO) grades, registrars and physician associates.
Twenty-six respondents (9.2%) felt confident making referrals (5 on the Likert scale of confidence) and 95 were fairly confident (33.6%) (4 on the Likert scale). The majority of respondents answered with scores of 1–3, indicating overall lower than anticipated levels of confidence, including those that felt somewhat confident (115 respondents—40.6%), slightly confident (42 respondents—14.8%) or not at all (5 respondents—1.8%).
Regarding barriers to referrals, 115 junior doctors (40.6%) struggled to know what specialty to contact at the time of referral, 168 (59%) were unsure of what information to include, 94 (33%) were unsure how to contact the specialty and 171 (60%) did not understand why they were making the referral.
The penultimate question of the survey considered consequences of referrals. Over half of junior doctors were fearful of making referrals (161 respondents—56.8%) and 192 respondents (67.8%) had experienced rejection of referrals. Particularly, the free text highlighted issues around professional behaviours including bullying or belittling behaviours and verbal aggression in an alarming number of cases (26 junior doctors—9.2%). Figure 2 displays a word diagram of free text quotes describing troubling experiences while making referrals.
Word diagram of quotes from specialists that junior doctors experienced after making referrals.
Impacts on patient care were frequently observed, including delays to obtaining inpatient specialty advice reported by 60%, delayed discharges reported by 50% and dissatisfied patients and/or relatives seen in 20% of respondents. Some poignant quotes about the impact on patient care are listed as follows:
‘Patients are often left waiting to be reviewed by a specialty that delays discharge and can often be done as an outpatient.’
‘Conditions deteriorating, patients re-presenting, delayed diagnosis.’
‘Led to consequence of death and palliation secondary to specialities not wanting to take responsibility of patient care.’
The final question in the survey aimed to promote peer involvement in the inception of interventions. The cohort highlighted that specific information about how to reach each specialty, what information to include when making the referral and strategies for escalation when a referral is rejected should be key parts of a referrals toolkit.
Methodology
The Quality Improvement (QI) team consisted of two foundation doctors and a medical registrar all trained in QI methodology. The project was run over a 6-month period during the 2020–2021 academic year from March to August. It was prospectively registered with London Northwest Healthcare NHS Trust (Registration number IM.EH.20.282).
A number of validated tools were used in the design of the interventions. Process mapping is recommended by the NHS England Institute for Innovation and Improvement and was used in the planning of this project.11 Initially, high level process mapping was performed with a small group of stakeholders to identify key areas for improvement in the referrals process and ensure the project team had a clear overview of the patient referral journey from Accident and Emergency (A&E) admission to discharge. Overall, it was observed that during an inpatient stay, several specialty referrals are required and that referrals from A&E to a broad specialty such as medicine or surgery were achieved promptly. Reasons theorised for this included the time pressures within A&E to make a referral and the heavy senior colleague involvement in deciding the team to which patients are referred, how to make the referral, quick escalation when a referral is rejected and easy access to specialty contact information to offload A&E beds.
The high level process map was used to identify the core constituents of a successful referral. Using the information from the baseline survey, a Failure Modes and Effect Analysis (FMEA) was generated describing in detail where and how each key step of the referrals process may fail. Hierarchical task analysis (HTA) was subsequently used to plan interventions to improve each step of the process.
From the high level mapping and the FMEA, three key themes were identified, to improve the quality of inpatient specialty referrals, as follows: theme 1—access to inpatient specialty contact information; theme 2—what information to include in a specialty referral; theme 3—obtaining further support in the event of referral rejection. The proposed interventions, as identified in the HTA, were continually implemented and reassessed to ensure they performed against these themes.
Strategy
The improvement interventions were implemented through the plan, do, study, act (PDSA) cycle, and the impact on junior doctors and patient care was assessed. PDSA cycles were initially undertaken at a local level, in two different DGHs, before progressing to complete PDSA cycles at a national level. A continuous process of reassessment of the interventions was undertaken at all stages of the project.
PDSA cycle 1
Following feedback from the baseline measurement survey, the first PDSA cycle aimed to provide accessible information on how to reach each specialty. This was intended to improve junior doctor confidence to reach a specialty and reduce the time taken to obtain advice. A freely accessible, online guide with specialty contact information, hosted via Google Sheets (Alphabet, California, USA), was created. Unlike previously described directories or the widely used Induction App (Induction Healthcare, London, UK) the guide detailed inpatient, emergency and outpatient contact information for each specialty at the trust and for services outside the trust if not provided by London Northwest Healthcare directly. The referral guide also included a link to a generic referral letter with prompts for necessary and relevant information when completing outpatient referrals. Figure 3 shows an excerpt from the referrals information guide.
An excerpt from a page of the Ealing specialty referrals guide. ENT, ear, nose and throat; EPRO, online clinical software platform utilised in the hospital trust; ERCP, Endoscopic Retrograde Cholangiopancreatography; GI, gastrointestinal; ICE, Integrated Clinical Environment; MDT, multidisciplinary team; NHS, National Health Service; UGI, upper gastrointestinal.
Links to this document were disseminated among junior doctor communications groups and email lists. Subsequently junior doctors were asked to complete a questionnaire asking about the same questions as at baseline. This illustrated an improvement in confidence of junior doctors in making referrals overall, with more feeling fairly confident or confident than previously. However, it was noted that some specialties were missing entirely, including dermatology and hepatobiliary surgery, and for other specialties information had only been included about inpatient referrals but advice on outpatient referrals was missing. After the addition of this information, colleagues were resurveyed to close the audit loop.
PDSA cycle 2
The second PDSA cycle had two key aims. The first aim was to assess the accessibility of the specialty referrals information guide. As an online link to a google document that had been emailed to all junior doctors in the hospital, the team had envisioned this would demonstrate excellent accessibility. However, the survey demonstrated that junior doctors were finding it very difficult to access the specialty referrals guide, searching through their emails every time to find the link. As a result, a QR code was generated which could be quickly scanned using a smartphone device. The code was printed, laminated and put up in the doctor’s office of A&E, the doctors’ mess, the main reception of each ward and the doctor’s office in each ward. A repeat survey showed that all participants were able to easily access the document.
The second aim of PDSA cycle 2 was to assess the impact of the specialty referrals information sheet on the success of interspecialty referrals and any impact on patient care, with the hypothesis that the intervention had resulted in an improvement. While there was good improvement in confidence making referrals and easy accessibility to the information sheet, many commented that knowing what information to include in the referral was an ongoing challenge. This set the precedent for further change as described in PDSA cycles 4 and 5.
PDSA cycle 3
Owing to the success of the referrals guide at Ealing hospital, the intervention was trialled elsewhere to demonstrate feasibility. After implementation at the centre, feedback from junior doctors demonstrated that the referrals information guide had made referrals quicker, easier, more efficient and reduced delays in patients receiving timely care. During this cycle, feedback highlighted that incorporating multidisciplinary meeting referral information into the guide would be worthwhile, which had not previously been considered as it is not a discrete specialty referral. Subsequently, multidisciplinary team referral information was included in the referral information guides for both trusts.
PDSA cycle 4
Based on the feedback collected through PDSA cycles 1–3, the aim of cycle 4 was to generate a specialty-specific list of clinical history, examination, pathological and radiological findings to include when making a referral. This would act as a checklist for junior doctors before making a referral, and improve the likelihood of referral success. The salient clinical features to include were obtained by canvassing opinion from specialist registrar colleagues nationwide, via the Mind the Bleep website. Once these data were collected, it was reviewed and collated to provide a succinct list of points for each specialty, then the specialties were pooled into one document to form a referrals ‘cheat sheet’ (please see online supplemental appendix 1 for the current version of the cheat sheet). The cheat sheet was then published to the aforementioned online platforms and further feedback sought on its design, which highlighted that specific specialties such as paediatrics and maxillofacial surgery were missing from the sheet, as well as specialty-specific deficiencies that were addressed. Moreover, a section on what to do if your referral is rejected was incorporated to encompass the third theme identified in the FMEA.
Supplemental material
PDSA cycle 5
It was initially anticipated that publication online would be sufficient for distribution of the referrals cheat sheet to junior doctors across the country. However, it was noted that the demographic was not geographically diverse. The aim was to collect more feedback and refine the resource by recognising that referral systems and specialty distinctions may vary across the UK, to make the cheat sheet universally applicable. To achieve this, the cheat sheet was shared with a more extensive list of junior doctor social media groups, including Reddit, Facebook and Twitter social media platforms, as well as email lists. This resulted in feedback from doctors across all areas of the UK, including from specialty doctors (consultants and registrars), leading to further refinement of the cheat sheet.
Results
The impact of the interventions on the primary outcomes of junior doctor confidence, time to interspecialty advice and patient care were assessed separately for the specialty referrals contact guide and the referrals cheat sheet.
For the specialties referrals guide, preintervention and postintervention 5-point Likert scales were used to test junior doctor confidence in making referrals. The preintervention median confidence was 3 (n=20, IQR=1), compared with postintervention confidence of 5 (n=23, IQR 1.5). The Wilcoxon rank sum test for unpaired data demonstrated that this change was significant (p<0.001). Additionally, 65% (n=15) of junior doctors responded saying that they found it quicker to refer to specialities while using the guide, and 81% (n=18) found that the guide had a positive impact in improving discharges on the ward.
To assess the impact of the referrals cheat sheet, a feedback link was distributed to those that downloaded the PDF from mindthebleep.com and consented to share their email addresses. There were 667 individual survey respondents from all the deaneries across the UK from December 2021 to July 2022. The cheat sheet demonstrated global engagement, with 70% of respondents from the UK and another 30% from across the globe, including Europe, North America, South America, Oceania and Asia. Furthermore, the referrals cheat sheet webpage has received 36 060 total views from 19 223 unique users across 6 continents. Fifty per cent of Foundation Year 1 doctors in the 2021–2022 year signed up for a copy of the sheet, and the figure for the 2022–2023 year has already surpassed the previous benchmark.
Of the survey respondents, 43 fed back on the impact of the intervention. Each respondent noted at least one specific benefit which applied to their own practice in the multiple choice question addressing the primary outcomes; 74% noted increased confidence in making referrals, 26% noted faster time to specialty referrals and 19% found it led to quicker patient discharges. Other improvements noted outside of the primary outcomes were that the cheat sheet has helped junior doctors understand why they are making the referral and what information to include, made it easier to identify which specialty to contact and led to referrals being accepted more easily.
Lessons and limitations
The DIRE project collected a vast amount of information on the experiences of junior doctors making referrals in NHS hospital trusts. The baseline data highlighted significant deficiencies in the ability of junior doctors to successfully make a specialty referral that hampered confidence, affected morale and ultimately impacted patient care. Particularly concerning, was that colleagues highlighted a number of bullying, undermining and aggressive behaviours witnessed when making referrals. This is perhaps unsurprising given a recent BMA survey suggesting that two in five doctors have witnessed such behaviours at their workplace in the last year12 and one in five have experienced it,13 resulting in reduced job satisfaction and more time off sick. This was an unexpected finding of the project and the authors certainly hope that the referrals toolkit can be used to improve junior doctors' confidence nationally in making referrals; addressing the core themes of: (1) access to inpatient specialty contact information; (2) information to include in a specialty referral; and (3) obtaining further support in the event of referral rejection.
The strengths of this project lie in the spirit of national and international collaboration, with involvement of doctors from all grades. This allowed us to create specific interventions that addressed the weaknesses in making referrals, as identified by the stakeholders themselves, with input directly from the specialties that was continually enhanced and reassessed through a structured and rigorous PDSA methodology. The feedback demonstrated that colleagues are using the referrals cheat sheet in teaching sessions for medical students and foundation doctors; most notably, a tertiary referrals centre and teaching hospital in central London has incorporated the referrals cheat sheet into its clinical simulation training. The referrals contact information guide has now been adopted by three centres, with further interest in its adoption in another two centres. Furthermore, the project satisfied its primary aims of improving confidence in making referrals, the time for specialty advice and the overall impact on patient care.
There were also limitations in the design and inception of this study. Compared with the number of downloads, the number of completed survey responses were low. Due to the nature of the intervention, it was not possible to obtain compulsory feedback at the time of the download and a survey had to be sent out around a week later. Furthermore, the number of downloads observed is likely a gross underestimate, as once the PDF cheat sheet is downloaded it can be widely and easily shared. Although this may limit the quality of the quantitative data, it enhances the generalisability and accessibility of the intervention and therefore it is also considered a strength of the design. Potential alternative options during the design phase could have included focus groups or other methods of gathering data when it was evident that feedback survey response rates were not as high as anticipated. In order to develop the referrals tool further, focus groups could be used to expand on the specialty advice currently available. This could include a ‘guide to the top three referrals per specialty’, for example, within endocrinology, guides to hyponatraemia, thyroid pathologies and hyperglycaemic emergencies are very different.
Due to the scale of the project, we were unable to generate a detailed guide of specialty referral information for each hospital in the UK, and therefore, were unable to formally assess the combined impact of both interventions. Instead, the impact of the interventions was assessed individually and the project has been widely publicised to encourage colleagues to adopt a similar method of collating specialty contact information. A template has been made freely available to those who have reached out to engage in similar work, so that the project may be continued. Finally, quantitative assessment would have enhanced the quality of our data collection, however, asking busy junior doctors across the country to, for example, time how long it took to obtain inpatient specialty advice is simply not feasible on such a scale. The authors accept that asking doctors directly whether they feel the intervention has improved the time to referral introduces a degree of recall bias. However, the main objective was to improve junior doctor confidence in making referrals and to hope that this would lead to increased efficiency, faster advice and ultimately better patient care.
Sustainability
In order to demonstrate the sustainability of the referrals toolkit, accessibility and utility data were collected from July 2022 to November 2022. During the interim 4-month period, there have been a further 12 370 page views, totalling 48 430 total page views, with an additional 3758 unique users viewing the page. Furthermore, 28 958 Mindthebleep.com newsletter subscribers received a downloaded copy of the referrals cheat sheet. After this further assessment, a total of 59 survey responses were obtained from users ranging from paramedic and medical students, to new FY1 doctors to SHO’s with greater than 2 years’ experience from the UK and abroad. Responses corresponded with all the aforementioned outcomes including knowing which specialty to contact, increased confidence in making specialty referrals, knowing what information to include, understanding why a referral is being made, reducing length of stay of patient. Specific free text responses in the feedback focused on the succinctness of the cheat sheet and how it assisted in structuring referrals, acting as an ‘excellent’ aide-memoire for both students learning how to make referrals and a refresher for junior trainees in hospital.
Conclusion
In summary, the DIRE project identified serious deficiencies and lack of confidence in referring patients for specialty advice. Many junior doctors have experienced bullying, belittling and aggressive behaviours when making referrals, and these factors result in low morale, avoidance behaviours, delayed referrals, delayed discharges and negative impacts on patient care. The project aimed to increase confidence in referring, improve the time to specialty advice and improve the quality of patient care. The referrals toolkit has achieved this through a combination of two different interventions, which improve the accessibility of specialty contact information, detail what clinical information to include in a referral and suggest the next steps to take if a referral is rejected. The international collaboration to create, adapt and optimise the interventions has allowed the project to achieve these aims and it is hoped that through continued publicity, the cheat sheet and referrals information guide will be adopted by more centres worldwide. Further work to improve the referrals process is needed and, particularly, the next step would be to promote cultural shift away from the damaging and demeaning behaviours, towards a more cohesive and constructive environment that promotes teamwork and a continual process of learning.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Ethics statements
Patient consent for publication
Acknowledgments
The authors would like to acknowledge all junior doctors, physician associates and consultants who have taken the time to participate in surveys and provide feedback for both the specialty referrals guide and the referrals cheat sheet. The authors would also like to thank the team at Mindthebleep.com for their assistance in disseminating the referrals cheat sheet and facilitating the spirit of international collaboration.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors Conception: EVT and BRHT. Planning and Development: EVT, BRHT and AD. Data analysis: EVT, BRHT and AD. Initial draft of manuscript: EVT and BRHT. Manuscript writing, review and approval: EVT, BRHT and AD. Guarantor: BRHT.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.