Problem

Local clinical guidelines and policies form a significant part of clinical decision making for junior doctors. Hospitals now have extensive archives containing such policies, often only accessible from static desktop computers. Accessing these can be time consuming for junior doctors, particularly during a busy on-call shift. Access is often limited by cumbersome search functions, slow internet connections, in-use or ‘locked-out’ computers, or simply their inconvenient location.

Junior doctors also need to contact various other locations, people and pagers throughout their working day. This results in extra time spent on-hold to switchboard to be put through to the appropriate number.

The Royal United Hospital Foundation Trust is a 565 bed district general hospital in the south west of England. It currently hosts its clinical guidelines on its own intranet site, accessible only from desktop computers within the hospital. In addition, there is no hospital telephone directory accessible electronically. Instead, numbers are either accessed on each department's intranet page, or through telephoning switchboard.

The project team consisted of an anaesthetic trainee doctor (responsible for App development and outcome measurement) and an acute medicine consultant (responsible for overseeing the project and developing guidelines).

Background

The arrival of the iPhone in 2007 revolutionised the way we access information.1 Since then, the influence of the smartphone has been felt in many industries. Globally, access to information on-the-go is now taken for granted. The healthcare sector is becoming ever more populated with medical smartphone applications (Apps) - including clinical calculators, guideline archives, clinical decision aids, and drug formularies. However, the NHS still lags behind the private sector in providing this new technology.

Smartphones allow healthcare professionals instant access to a wealth of clinical information, without them leaving a patient's bedside. This overcomes the relative paucity of fixed desktop computers available in most clinical areas. Use of mobile devices has been found to positively impact upon clinical decision making, reduce clinical errors, and improve the dissemination of clinical data within healthcare teams.3

A recent UK-based survey found that up to 79% of medical students and junior doctors own a smartphone, with approximately 80% owning a smartphone app relating to their day-to-day work. Well over half of all smartphone owners have an iPhone. The survey concluded that junior doctors were “overwhelmingly enthusiastic to endorse organisational associated apps that help their learning and work activities”.2

Delivery of healthcare is conducted in multiple locations during a working day. Therefore enhanced portability of information is necessary.3^,4 Rapid access to information is often required in certain clinical environments such as the emergency department.3 Systematic reviews have identified that smartphone usage leads to improved decision making, improved working efficiency, reduced clinical errors, greater adherence to standard practice, greater satisfaction with IT services.3

Development of smartphone apps in local environments, to solve specific problems, have been demonstrated.5–7 While larger companies are responding to the need for healthcare apps,8 the availability of tailored / bespoke apps at a local level is still relatively lacking. This is likely due to limited funding and technical capability of healthcare and IT staff in this area.

Here, we describe the development of a clinical smartphone app designed to improve the access of clinical and hospital-specific information. This included clinical guidelines, hospital telephone directory, hospital maps, and ward induction guides for junior doctors.

Baseline measurement

To determine the benefits of our proposed App, we chose to quantify the amount of time the App could save a busy junior doctor in real-terms.

A representative sample of ten junior doctors were recruited to determine the amount of time spent accessing guidelines using using desktop computers. To simulate a real-life situation, each participant was led to a patient's bedside and asked “Please can you find me the guideline to…”. This was completed during normal working hours on several different medical and surgical wards. Participants were asked to locate one commonly used guideline - Management of Hypokalaemia - and a more obscure guideline - Prescribing Amiodarone. Each participant was then timed using both means of accessing the guideline (see results section).

Baseline Measurement (see Table 5):

The average time to access the Hypokalaemia guideline was 76.8 seconds (95% CI 30.6) using a computer. The average time to access the Amiodarone guideline was 142.0 seconds (95% CI 44.8).

Design

SMART Aim:

Our overall aim was to reduce the time taken for clinicians to access guidelines, by providing a smartphone App. This improvement was measured by directly timing how long it takes to access a guideline. The project timeline was intended to be six months from baseline data collection, to allow for the in-house development of the App.

App Design Process:

The App was designed and developed from scratch by a trainee doctor in his spare time. Over the course of a year he became self-taught in iPhone software development. Through talking to other junior doctors, it was evident there was a local need to improve access to clinical policies and telephone numbers. Many junior doctors were already saving these documents to their smartphones or keeping notes of relevant phones numbers. This provided an opportunity to develop a standardised way of people using their smartphones to access this information.

The main goal of the project was to provide access to information within as few steps as possible. The App had to look stylistically clean and work quickly. The tabbed design, also used by many popular Apps, allowed this to be achieved. Users would be instantly familiar with this style of navigation. Each section is always visible and navigable from anywhere within the App.

The Home page contains an embedded Twitter feed. This is a way of communicating targeted information to clinical staff. Rather than group emails (which can easily go unread for several days) this presents users with real-time and up-to-date information each time the App is launched.

The hospital directory section has been through several iterations. Initially it only contained major locations (e.g. Radiology, Stroke unit…). Each item linked to a static map with a single contact number provided for that location. Following feedback cycles, the final iteration allows the user to select between Locations, People, and Baton Bleeps. Each list item links to a generic ‘contact’ page with all available numbers for that particular contact. From user feedback, an additional map display is available for locations.

The focus of the App is the availability of our hospital's medical guidelines archive. The initial design displayed guidelines as a continuous alphabetised list, meaning the user had to scroll past several guidelines to find what they were looking for. The current release now lists guidelines under collapsible sections. This allows the user to see all available categories on one screen. They are also able to ‘expand all’ and ‘collapse all’ if they can't immediately see what they're looking for. Guidelines (and more recently, contacts) can be favourited, allowing users to personalise the App and save searching time.

Guidelines can be easily searched. This, again, allows efficient location of guidelines and contact numbers. Search results are displayed instantly as the user types. The guidelines are tagged with keywords. This attempts to predict what user may search for. For example, a user looking for the Hyperkalaemia guideline may type ‘potassium’. The search results also display exact matches first. For example searching for ‘PE’ returns ‘Pulmonary Embolism’ first before other guidelines (such as Hypercalcaemia). A similar search function is also available for the directory.

It was important that all of the content within the App was available offline. Mobile reception in hospitals is notoriously patchy. The success of the App hinged upon it's speed and reliability. However, the offline content is also liable to frequent change. All guidelines are therefore stored on a private web server along with a version number. The App automatically checks for more recent versions and prompts the user to download new guidelines when appropriate.

Strategy

Outlined below is the process we used to design the app.

PDSA 1:

03/08/14 - Initial development of the underlying mechanics of the App. Each aspect was refined through regular meetings with an acute medicine consultant. This was a continuous cycle of minor improvement, feedback, and re-design.

25/01/15 - Release of App to small group of peers and consultants.

09/02/15 - Collection of feedback from these users using an online survey.

10/02/15 - Implementation of changes. This included adding the feature of bleep responder / directory reverse look-up, alteration of how the maps were displayed, user interface improvements, and fixing search problems.

PDSA 2:

23/02/15 - Release of the App to a single team within the hospital

27/02/15 - Collection of feedback through online surveys and ad hoc through the feedback function within the App.

28/02/15 - Implementation of suggested minor changes.

PDSA 3:

02/03/15 - Release to whole hospital, including junior doctors, consultants, and nursing staff.

09/03/15 - Collection of feedback through online survey.

12/03/15 to Present - Assessment of effectiveness and reception through evaluation of user feedback.

Following development of the app, the same cohort of junior doctors was subsequently timed on accessing the aforementioned guidelines using the App.

PDSA 4:

Users wanted to be able to ‘favourite’ guidelines on their devices to make them more accessible. We implemented this feature on subsequent app versions, allowing users to access these from the home page of the App. A group of ten junior doctors were then timed accessing the previous guidelines using this function. They were first shown the feature and asked to ‘favourite’ the appropriate guidelines.

See supplementary file: ds7243.docx - “Survive On Call Screenshots”

Results

Time taken to access guidelines

PDSA Cycle 3 (see Table 5):

The average time to access the hypokalaemia guideline with the App was 12.4 seconds (95% CI 2.3), versus 76.8 seconds (95% CI 30.6) using a computer. A difference of 64.4 seconds (p = 0.001).

The average time to access the Amiodarone guideline with the App was 25.9 seconds (95% CI 12.9), versus 142.0 seconds (95% CI 44.8) using a computer. A difference of 116.1 seconds (p < 0.001). Three users (30%) had never used the App before. There was no significant difference between these users and those who had used it before (in terms of time to access guidelines).

PDSA Cycle 4 (see Table 5):

The average time to access the Hypokalaemia guideline remained very similar at 11.9 seconds (95% CI 3.05). However, the time to access the Amiodarone guideline improved to 11.2 seconds (95% CI 2.52), meaning there was no statistical difference between these two groups (i.e. p = 0.299). This suggests a improvement in efficiency for this who may use, or anticipate using, more obscure guidelines. For example, a junior doctor rotating to a cardiology rotation may pre-emptively wish to ‘favourite’ cardiology-specific guidelines.

User feedback:

Feedback had been collected using anonymous online questionnaires. In summary, feedback has been tremendously positive at each stage of release.

Release of the App to a small group of colleagues (Beta-testing):

At this stage feedback was qualitative. We asked specifically for good points and points for improvement. Users liked how “logical, intuitive, easy to use” the App was. They commented on it being “Quick to load”, with a “Comprehensive directory”.

Release to the Acute Medicine department junior doctors:

Four out of ten (40%) junior doctors responded to our feedback questionnaire. 100% of the respondents agreed that the App improved the way they accessed guidelines and phone numbers. All agreed that it improved their working efficiency. They rated it as either ‘Good’ or ‘Very good’ in terms of design, ease of use and content (See Table 1).

At this stage, minor improvements were suggested relating to typos or broken links.

Release to all clinical staff within the hospital:

157 users had downloaded the App at upon collection of feedback. This included junior doctors, consultants, and nursing staff. 29 (18% of users at the time) responded to our online feedback survey. The appearance, ease of use and content were again rated as either ‘Good’ or ‘Very Good’ by all respondents (Table 2). Users were also asked to rate their frequency of use and how much time they felt it saved them (Tables 3–4). 96.1% of respondents felt it either saved them time a ‘few times a week’ (53.85%) or ‘significantly saved time every day’ (42.31%).

Our Google Analytics data shows us 338 unique users have used the app since launch (February 22nd 2015 to January 1st 2016). Guidelines have been accessed a total of 11,071 unique times during this period. Based on estimates, using our quantitative data, this may have saved between 196 and 354 working hours over a ten-month period.

See supplementary file: ds7332.pdf - “Tables 1 - 5”

Lessons and limitations

There are significant benefits of providing our users with offline data. These include speed of access and no requirement for an active internet connection. However offline information has the potential to become outdated. This is compared to server-based data, which may be slower to access, but always remains up-to-date.

We feel we have mitigated the drawbacks of offline data, by designing a process whereby we periodically present users with ‘pop-ups’ alerting them to new content updates. The responsibility is then with the user to ensure they keep their own app up-to-date by accepting these updates.

Given the opportunity to repeat the project, we would like to analyse the impact upon on the uptake of local policies and standards. For example, does it improve adherence to local antimicrobial prescribing? This is outside the scope of the current project, but may have significant implications for organisations deciding how to provide access to their clinical policies in the future.

We would also like to analyse the impact upon switchboard and intranet traffic. Both of these were not considered when collecting baseline data. We are currently exploring ways of obtaining such data retrospectively.

Long Term Benefits

The anticipated long term benefits of the App are many. They include improved satisfaction with IT services and improved access to clinical policies, guidelines, and telephone numbers. The App allows busy clinical staff to work more efficiently and access the information they need quicker. This has the potential to improve adherence to local clinical policy and clinical standards.

The sustainability of this App currently depends on two clinicians' work in their spare time. Very little daily maintenance is required, aside from periodically updating guidelines. Organisations could consider hiring in-house App developers to support this emerging field, or training existing staff who may be interested.

Limitations

A small sample size was obtained to analyse the time-saving aspect of the App. However, the large effect demonstrated would likely be unaffected by a larger sample. Furthermore, there was a relatively low response rate to some users feedback questionnaires. Finally, there is a lack of additional baseline data e.g. switchboard activity, frequency of use of intranet guidelines. This hindered our ability to assess further areas in which the App improves working efficiency.

Conclusion

Providing clinical staff with mobile information appears to have significantly improved their satisfaction with how easily they can access clinical guidelines and telephone numbers. They clearly feel it has improved their working efficiency. This has been supported by quantitative measures of actual time saved using the App. The ability to access such information in as little time as possible may be even more pertinent where decision-making is time-critical - for example in Anaesthesia and Emergency Medicine. Further study into these specialties is warranted to determine whether mobile information can impact upon patient safety and clinical outcomes.

Furthermore, by providing clinical policies on a smartphone we may impact a variety of quality improvement projects and audits. This could help improve adherence to specific clinical quality standards and practices.

Based on the authors' experience, many NHS institutions already provide clinical guidelines that can be accessed using desktop computers. It is reasonable to suggest that similar results can be achieved in these institutions. All that would be required is a simple re-formatting of their guidelines to fit a smartphone screen width. One could propose an NHS-wide version of the App. The app could easily detect a user's location. Data could then be provided depending on the user's current place of work. This would benefit many clinicians who often have to adapt their practices to local policy when moving through their training rotations.