Sharing knowledge, saving time: an online toolbox to aid junior doctors ======================================================================= * James Houston * William Barker * Jonathan Clarke * Ed Mew ## Abstract Junior doctors are too often frustrated by not being able to quickly find information for how to make referrals, book investigations and contact other professionals at hospital. To make matters worse, much of the knowledge gained by doctors throughout the year is lost during the August rotation. There is an unmet need for retaining such knowledge, in order to facilitate a smoother and safer handover. We set up the “Doctors Directory” at our trust; a website run by junior doctors, providing specific, up-to-date information relevant to other junior doctors within the trust. Whilst providing day-to-day information, it also contains a "survival guide" for each hospital firm. We surveyed junior doctors before and after the implementation of this site. 81% of FY1s surveyed have used the site. Of the doctors that had used the site, 94% found it helpful with a mean self reported time saving of 39 minutes a day. Whilst still in its infancy, the site now has mobile access, and has an average of 60 hits a day. Quality improvement projects such as this are readily scalable to other hospitals and have enabled junior doctors to waste less time finding how to do jobs and more time actually getting them done. ## Problem Junior doctors spend a large amount of their time performing administrative tasks within the hospital. They also rotate through multiple different firms every year. Since rotations occur so frequently, there is often a loss of tacit knowledge gained through the previous rotation when they move on to the next job. Learning and re-learning how to perform firm-specific administrative tasks is a significant barrier to working efficiently in a new job. Such tasks include: requesting different tests, using hospital guidelines that are frequently difficult to find, and making referrals to different medical specialties & members of the multi-disciplinary team. Anecdotally, and through our own experience, we felt that more time and stress is often spent finding out how to do such tasks rather than actually doing the tasks themselves. Patient safety is compromised, time is wasted and stress is incurred if these tasks are done incorrectly. In the U.K, this is particularly the case around rotation times, where handover of how to do jobs between doctors is often inadequate. Spending less time on these tasks, would mean more time available on items relevant to the post-graduate curriculum and our learning. ## Background Published research has shown that hospital mortality increases by 6% (1) over the largest rotation time for junior doctors. In 2012 a programme was instigated by Medical Education England (now named Health Education England) to try to facilitate the August handover by providing paid shadowing period for a minimum of four days to new doctors (2). This scheme has become mandatory in 2013. Likewise results from a pilot study in Bristol found that mistakes were reduced by 52% after a week of targeted shadowing and mandatory teaching (3). However no shadowing period can be entirely comprehensive and many new questions will arise during the first days and weeks of a new job. By this point there will be no other junior doctors to shadow and most new trainees will have to find the information out through more time-consuming means. Some trusts use paper guides, but these are frequently lost and are cumbersome to carry around. The learning cycle of a job extends throughout the entire period, and it is difficult to explain and retain information during short formal handover sessions. Much local knowledge is lost between rotating doctors, which only has to be learnt again. It would be helpful to have a repository in which to store some of this local information and knowledge, to facilitate junior doctors in their daily jobs. ## Baseline Measurement For this project, two main baseline measures were considered. The first gauged how confident junior doctors felt after induction. The second examined existing information resources that junior doctors used. While it was not asked in the original baseline survey, an attempt was also made to quantify how much time was spent finding out how to perform routine tasks. Consequently later surveys included a self-reported measure of this recurrent problem. The original survey was taken from new starting junior doctors at the end of their induction. When trying to gauge how new starters felt after induction, junior doctors were surveyed on questions such as whether they knew how to request standard tests, whether they knew what a standard day would involve and whether they knew how to make a referral to different specialities. Should they not know such information, doctors were then asked how they would go about finding such information. This included set options such as asking a senior, asking another junior doctor or finding out using the hospital intranet. Junior doctors were also asked specifically whether they would like more information on their standard day, their on-calls and how to request different tests. The baseline survey found only 12% of doctors agreed with the statement that they knew what their on-call rota would involve with 97% wanting to know more about the rota. A similarly small proportion agreed that they actually knew how to make referrals to medical (12.5%), surgical (9.4%) and other specialities (3.1%). Only 30% knew how to request common tests specific to their firms with 91% wanting to know more about how to request such tests. All of these common tasks might be be expected to be known or at least demanded from a junior doctor in their first couple of days on the job. It appeared that from the results, a massive unmet need of providing such information was required. When trying to find information junior doctors would primarily ask a senior for such information. Specifically, 53% of doctors would ask a senior as first port of call for information regarding how to request a test and 68% of doctors would ask a senior as first port of call on how to make a referral to another team. While senior doctors should always be on hand to assist new joiners, being frequently asked how to do common tasks may well detract from time that they could spend on other clinical duties. There was also no certainty that the senior doctors would actually be able to answer such questions; such local knowledge is often the preserve of previous junior doctors working in that job, who may have now left the hospital. See supplementary file: ds2499.xls - “Questionnaire analysis” ## Design Our intervention involved implementation of the "Doctors Directory". This is an online wiki-based toolbox that serves as an information repository to facilitate completion of junior doctor's administrative tasks. It includes several different components: 1. Handover guides for all FY1 firms starting in August 2. Bleeps of doctors in the hospital both for during the week and out of hours 3. Extension numbers for wards, secretaries and other hospitals 4. Commonly used reference guides specific to the hospital that are now downloadable to desktop computer and mobile phone 5. How to make speciality referrals within the hospital 6. How to request specific investigations within the hospital 7. A resource for teaching and post graduate medical education Users interact in two ways. Most commonly, they use the toolbox to look up administrative details specific to their firms or to find how to request common tests or referrals. We have also tried to make the toolbox sustainable by incorporating our experiences of the shortfalls of trusts' inductions as a non-dynamic entity. Users can interact with the toolbox by helping to update the contained content. The toolbox is in wiki format, making it easily edited. The editing is performed by a handful of dedicated editors who hold a specific editing password which thus reduces the accountability risk. We further enhanced the sustainability of the project by nominating editors to carry the project forward. We have developed a certification process as an incentive to fully completed toolbox sites. The toolbox is, at time of publishing, in place in 40 trusts in the United Kingdom. Our next development plan relates to creating an mobilie phone app which will ensure that junior doctors in trusts with poor wifi connectivity can still use the toolbox. Our timeframe for this app development is over the next six months to April 2014. We have secured enough funds to ensure a server for the toolbox for the next five years but are actively looking for more funding from national competitions to help fund this. This hopefully ensures that the toolbox is an independent entity with no reliance on any resources. ## Strategy Improvements to the directory are continuous so as to maintain up-to-date information to users. In addition to this, we underwent formal PDSA reviews of content, utilisation and assessment of junior doctor opinions of the directory. Our "Plan" stage involved completing baseline surveys of junior doctors and involving keys stakeholders such as senior consultants, hospital IT support, junior doctors and Foundation School administrators. In later stages the "Plan" stage involved planning the implementation of suggested changes brought up from previous PDSA cycles. The "Do" stage involved implementing the site in the trust. The main concern here was to populate the site as completely and quickly as possible, whilst ensuring adequate usage by junior doctors. We found that advertising the site in junior doctor dedicated teaching was effective. The "Study" stage involved analysing uptake of the site. We used Google Analytics to analyse numbers of site hits. We also sent out further surveys to ask junior doctors how much they felt the toolbox had saved them in time and effort. We also listened to word-of-mouth opinion on what the site lacked and what could be improved. The "Act" stage involved adding new content and recruiting new editors. We have gone through three PDSA cycles. From the first cycle we implemented the toolbox. Junior doctors felt that it saved them time and there were some requests for additional features such as embedding commonly used forms into the site. From the second cycle we further edited content which included more information on contacting members of the multidisciplinary team. We also recruited more editors to help manage content. From the third cycle we re-configured the skin of the site, making it more user-friendly. See supplementary file: ds1917.pptx - “PDSA Cycle” ## Post-Measurement Doctors who had just graduated from medical school, known as Foundation Year 1 (FY1) doctors, were thought to have the most potential to benefit from the website. This group of junior doctors were surveyed at a teaching session within one trust that had set up the Doctors Directory. FY1 doctors were asked whether they had used the website and, if they had, whether it was useful and saved them time. Doctors were also asked to self-report how much time they thought the website had saved them. We found that 81.0% (17/21) of FY1 doctors had used the toolbox. Of the doctors that had used the website, 93.8% (15/16) found it useful to access information and 100% (16/16) felt it saved them time. One survey paper was spoilt, hence the number of doctors surveyed in the second and third question is one less than the reported number using the site. The mean self-reported time saved per day through use of the site was found to be 39 minutes. When asked how they would find information about requesting a test, 55% of FY1s would now use the Doctor's Directory as first choice for such information with only 18% asking a senior. Similarly when asked how they would find information regarding how to make a referral 68% of FY1s would now use the Doctors Directory as first choice for such information, with only 18% asking a senior. The site was also shown to be popular from the number of times it was recorded to be accessed. Since launch we noted that approximately 2000 hits have been recorded hits per month. From analysis of this data compared to to baseline survey, we surmised that after implementation of the toolbox, FY1s are less likely to contact senior when finding out how to request a test after 1 month (p = 0.015). FY1s are also less likely to contact a senior when finding out how to make referrals after 1 month (p = 0.001). See supplementary file: ds1502.png - “Graph showing changing actions with Doctors Directory” ## Lessons and Limitations There were some difficulties encountered in the implementation of the project. Firstly we had difficulty getting the hospital IT department and senior colleagues to become more involved. This was crucial, since placing a hyperlink on the hospital intranet would greatly improve access. This issue was overcome by meeting members from the department in person and convincing senior colleagues of the worth of the project by showing them the baseline data. Secondly there was a general reluctance by junior doctors to fill in more forms such as surveys or those detailing their job requirements. This was overcome by general persistence and making doctors realise the mutual benefits of the project. We have planned for the following developments in the future; rolling out the site to different hospitals, applying for longer term funding and developing a smartphone app for mobile content that is available while offline. We are also looking to create similar sites to help other health professionals in the hospital, such as nurses, pharmacists and physiotherapists. We observed the following limitations with the toolbox project. Firstly it was reliant on being up-to-date. Should the data not be correct, then the site would likely waste time rather than save it. Editors were encouraged to update the site continuously throughout the year and have clear communication channels in which people could raise issues. Secondly continuity of the site is difficult, especially when doctors move on to new jobs. This was something we tried to mitigate against by appointing new editors when doctors came for their original shadowing period. It was also helpful to have a permanent senior doctor involved in the project at the hospital. They could then also provide continuity between rotations. Finally, there was a small sample size in our survey. The sample was only of the doctors present at teaching. Doctors who were unable to make teaching sessions often could not make the sessions since they had the most work to do. Thus they perhaps might have been the ones to benefit the most from the website! While this sample involved the majority of FY1 doctors at our hospital and was likely representative of our hospital, it is unclear whether this is representative of all junior doctors across the UK. We are currently compiling data from different sites across the country and hope to publish this data in the future. ## Conclusion Trainee time is wasted finding out how to do common jobs when starting in hospital. The Doctors Directory can save time both to trainees and to senior doctors. Hospitals would benefit by setting up similar systems. The site provides readily accessible, specific information to doctors at no cost to hospitals. With constraints such as the European Working Time Directive, time available during the week is shortened. Thus projects such as this can help reduce time wasted on tasks that have no benefit to training. Free time could then be spent at the bedside, signing off competencies or going home on time! The site has the potential to be set up easily at other hospitals and satellite sites are being set up across the UK (4). In order for the sites to succeed and provide continuity in the long term, further "enablers" must exist at senior levels alongside trainees implementing the projects. ## References 1. Jen MH et al. Early in-hospital mortality following trainee doctors' first day at work. PLoS One. 2009 Sep 23;4(9):e7103 2. Medical Education England [Internet]. London. [cited 2013 Oct 13] Available from: [http://www.mee.nhs.uk/latest\_news/news\_releases/shadowing.aspx](http://www.mee.nhs.uk/latest_news/news_releases/shadowing.aspx) 3. Jaques H. Shadowing by foundation doctors halves errors, pilot shows BMJ Careers [Internet] 2012 Jul [cited 2013 Oct 13] Available from: [http://careers.bmj.com/careers/advice/view-article.html?id=20007862](http://careers.bmj.com/careers/advice/view-article.html?id=20007862) 4. Dr Toolbox [Internet]. London. [cited 2013 Dec 13] Available from: [www.dr-toolbox.com](http://www.dr-toolbox.com) ## Declaration of interests We have read the BMJ Group policy on competing interests and declare the following interests: Dr Toolbox has now been registered as a limited company, with some of the authors holding shares in the company. Data was collected prior to formation of the Company. However, the web site is owned in part by the authors (Barker, Houston, Mew), who seek to provide a free service to NHS doctors, with the support of Healthcare Education England. ## Acknowledgements Dr Jim Buckley This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: * [http://creativecommons.org/licenses/by-nc/2.0/](http://creativecommons.org/licenses/by-nc/2.0/) * [http://creativecommons.org/licenses/by-nc/2.0/legalcode](http://creativecommons.org/licenses/by-nc/2.0/legalcode)