Problem

It is a priority for the NHS that care is safe, effective and compassionate. During an unprecedented period of change and increasing financial constraints, it is imperative that staff feel engaged, valued and supported at work, to enable them to deliver high quality care. There is a growing body of evidence showing that engaged staff deliver better care with better patient experience, fewer errors, and lower infection and mortality rates.1–5 Patient experience data are important in raising concerns, but are often only available after the event and so limiting the potential for immediate intervention. They are also highly selective, and may not relate to organisational and staff factors that affect their care.

University Hospital Bristol NHS Foundation Trust (UH Bristol) is a tertiary teaching hospital in the West of England. Staff and patient experience are key parts of the Trust's quality strategy. Prior to this project, information about staff satisfaction and experience was obtained from the annual staff survey and staff exit interviews. The 2014 staff survey indicated that only 57% of staff would recommend it as a place to work. Further analysis revealed that particular areas of concern were local decision-making, effective communication and visible leadership. Nevertheless, a CQC inspection found that staff maintained caring attitudes despite the challenges they faced6. This finding highlighted staff's frustration when unable to deliver the high quality care they wish to provide. Subsequently, a range of strategies were considered by a clinical team to improve staff engagement. In particular, the aim was to develop a mechanism that enabled staff to raise problems and frustrations, highlight positive experiences and encourage local leaders to respond to these. Through the processes described in this report, a real-time feedback mechanism was developed to continually monitor staff experience as close to real-time as possible and facilitate feedback from local leaders. This became known as the Staff Participation Communication and Engagement app (SPEaC-app), also known as Happy App by staff. The overall purpose of the application was to encourage staff to actively engage with managers in improving their working environment and standards of patient care. The aim of the improvement project was to develop and test the SPEaC-app and determine the levels of staff interaction. Our primary outcome was SPEaC-app usage, which was measured by a) the volume of feedback posts made by staff and b) the reasons for those posts. As this was a new intervention there were no previous data against which SPEaC-app data could be compared. A secondary outcome measure was staff engagement, as measured via the national staff survey, although the implementation of the SPEaC-app could not be linked to it directly. Finally, staff satisfaction with the SPEaC-app was assessed through an online survey and staff workshops.

Background

The quality of patient care relies largely on the skills, compassion and enthusiasm of front line staff. Evidence linking staff satisfaction and patient experience7 ,8 shows that high levels of engagement impact positively on both staff and organisations9 and that NHS organisations deliver better quality care when their staff are engaged and involved in decision-making.10 Similarly, organisations with engaged staff deliver better patient experience, fewer errors, lower infection and mortality rates, stronger financial management, higher staff morale and motivation, and less absenteeism and stress.1–5 By contrast, poor staff engagement can lead to disengaged and demoralised staff, resulting in poor quality and potentially unsafe patient care.11 Despite this evidence, levels of staff engagement are highly variable11 and a lack of innovations by NHS Trust boards to increase staff engagement has been reported.12 There is, therefore, a need to develop, test, implement and evaluate initiatives to promote staff engagement within health organisations in order to understand what interventions have the most impact and under what conditions.5 What is clear is that positive leadership and perceived organizational support are crucial to promoting staff engagement.5

The Francis Report recommends using real-time feedback to inform and drive improvements in patient safety and care.13 While strategies to collect staff experience data are not always evaluated or published, some Trusts report shifting away from using annual staff surveys to more frequent ‘pulse surveys’, producing engagement scores. However, this approach does not capture specific factors currently affecting staff and patients and does not allow immediate managerial response.

Over the past two years, UH Bristol clinicians, managers and researchers have worked together to develop, test and evaluate a web-based SPEaC-app which collects real-time, work-related mood feedback from nursing, medical, and other staff groups about problems and frustrations, but also positive experiences relating to their shift. In this report, we describe how this real-time feedback mechanism was developed to improve staff engagement.

Baseline measurement

Throughout this project the team measured (1) SPEaC-app usage data, (2) staff engagement and (3) staff satisfaction with the SPEaC-app.

1. The following SPEaC-app usage data were reviewed weekly by the project team:

 – Number of hits per week by area

 – Number of hits per category across all areas

 – Number of hits per work-related mood icon (positive, negative or neutral) across all areas

1. Staff engagement: The annual staff survey has been used in the NHS since 2003, with specific questions about work engagement being added in 2009. Dimensions of engagement measured are advocacy, involvement and motivation. Prior to this project, the overall staff survey results of 2014 revealed that 74% of staff members at the Trust did not feel that communication between staff and senior managers was good; 54% of staff felt able to make improvements happen in their area of work; and only 57% would recommended it as a place to work.14

2. Staff satisfaction: A brief online survey was distributed to staff to investigate their satisfaction with SPEaC-app. It was delivered initially in the Children's Emergency Department (CED), at two time points during the pilot phase and then to staff in six areas using the SPEaC-app in 2016. Throughout the development, piloting and refining of SPEaC-app, the project team monitored usage via regular workshops with stakeholders. These increased understanding about factors affecting its implementation and uptake.

Design

Our project was underpinned by the Model for Improvement by Langley et al15 which conceptualises improvement as deriving from action, namely the developing, testing, implementing and spreading desired changes. It provides a stepwise, iterative approach to developing specific identifiable improvements, the Plan-Do-Study-Act cycle.

PLAN

An initial workshop was held with staff of all grades including, medical, nursing and administrative staff from a surgical ward and the CED. Staff identified factors affecting staff mood and satisfaction and then discussed methods that could be used to capture and report staff feedback. The principles underlying the methods suggested by staff were the ability to gather, collate and share feedback and agree what action to take. Staff also proposed that feedback should be captured in (or close to) ‘real-time’, be confidential, easy to access and that it could become part of a daily departmental routine. Enthusiasm grew for an IT solution, or an ‘app’ that would encourage staff to express how they were feeling while at work, raise concerns or report any positive or negative issues impacting on patient safety and care. This was not to replace incident reporting. It was agreed that this initiative could offer a sustainable solution with potential for wider use across other hospitals. The UH Bristol Transformation team agreed to support this initiative.

DO

The SPEaC-app was initially developed as a basic interface storing data on an excel spreadsheet. On the user home screen, staff could rate their current mood by selecting either a positive (green), neutral (yellow), or negative (red) face and write a comment to explain why they had picked that mood. Local managers or team leaders, known as ‘administrators’, could down-load the data daily and then feedback to staff on issues raised, reporting the department's overall mood by producing a printed poster each day, with an update of the previous day's comments.

The SPEaC-app was piloted between January and September 2015 in two different areas within UH Bristol, the Children's Emergency Department (CED) and an adult in-patient surgical ward. This initial ‘proof of concept’ phase enabled the project team to develop and test the process, while planning how to scale up the tool. Dedicated terminals were set up specifically to record and explain reasons for their mood and staff in these two areas began to use the SPEaC-app. Staff could use the SPEaC-app as often as they wished during their shift and all their responses were anonymous. In each department, designated local managers, or team leaders reviewed the entries every day. They produced a daily or weekly information and feedback sheet to share the number of positive, neutral and negative reports and produced lists of staff concerns and suggestions, outlining the relevant actions taken. These information and feedback sheets were placed next to the SPEaC-app terminals so that staff could see the actions being taken in response. Positive staff comments were also shared this way. A paediatric in-patient ward began using the SPEaC-app in April 2015. Uptake of the SPEaC-app in the Children's hospital was rapid and sustained. Both these areas had a local manager and matron who supported it and ensured that responses to staff comments were rapid and comprehensive. Uptake on the surgical ward was less consistent, due to difficulties with the location of the terminal and connectivity issues. Feedback was also slower and less comprehensive, which had a negative impact on staff's perception of the SPEaC-app.

Strategy

STUDY

During the first few months of the pilot phase, data were downloaded weekly and shared with research partners at the National Institute for Health Research Collaborations for Leadership in Applied Health Research and Care (NIHR CLAHRC West). The initial 8 weeks of data collection from CED were analysed by a qualitative researcher. Data comprised 486 entries, with an average of 10 entries per day. Using thematic analysis16 every entry was assigned one or more codes. Overlapping codes were then merged and refined and the data was grouped according to common themes for positive and negative codes. These were:

• Positive: good team working; good patient flow; environment & equipment; happy staff; good patient care.

• Negative: equipment & resource issues; environmental problems; short staffed; patient flow and management; interpersonal problems; pay; shift problems; lack of teaching & training.

The number of entries in each theme were counted, in order to report frequencies. The most frequently occurring positive theme (total n=225) was ‘good team working’ (54%; n=121) which captured issues such as good support, communication, leadership and people. The most frequently occurring negative theme (total n= 227) was ‘equipment & resource issues’ (24%; n=55), including both clinical and non-clinical equipment. In a user workshop staff agreed that to improve the usability of SPEaC-app, a menu of categories should be provided to tick as appropriate, followed by a ‘free text’ comments box. The codes from the thematic analysis were used to guide the development of the menu of categories: environment; equipment; patient care; patient flow; pay & conditions; teaching & training; team working; miscellaneous.

In April 2015, a staff workshop captured users' perceptions of facilitators and barriers to using the SPEaC-app. Users highlighted that they valued the anonymity of the app and were motivated by the knowledge that someone would read and act upon their posts. The primary barriers to using the app were time constraints and IT issues, especially when the app ‘crashed’, logged out, or needed rebooting. Overall the feedback from the workshop was that the SPEaC-app was an effective “tool for triggering engagement” and that it effectively “measures the temperature of the department”.

ACT

After the initial pilot phase using the excel version of the SPEaC-app, the project team identified constraints to its usability. As local leaders, known as ‘administrators’ could only access the entries made by staff on one computer, staff entries to the app were not routinely monitored in real time. The project team then developed a SPEaC-app website to improve both the user and administrator (a local manager) experience, and to facilitate two-way communication between staff and administrators by:

• allowing authorised users access to SPEaC-app data in real-time, regardless of their location

• including real time notifications to facilitate more proactive reaction allowing administrators to reply to comments

• enabling users to see the current ‘mood’ in their department.

In June 2015, the first phase of the SPEaC-app website development took place, creating basic functionality for the user to select a mood icon, an appropriate category from a drop down list, and then add comments via free text. Users could also view a list of published comments from their colleagues, responses and mood icons. Managers or leaders were able to log in to an administration page where they could view, edit, reply to and publish comments. The second phase of website development focused on usability. Users could scroll back through the last 100 entries on their homepage and could ‘like’ entries or comments. At this point the website was also made compatible for use with a tablet. Administrators could mark comments as resolved or closed and could produce reports using filters for dates or themes, which enabled correlation with incidents reports, workforce and complaints data. They could also provide a real time ‘mood heat-map’ and escalate concerns raised by their staff, by emailing them to Trust contacts. An announcements box for administrators, displayed on the home screen, allowed them to notify their team of any important local issues.

Workshops with representatives from all areas of the Trust and all staff groups, including clinical, non-clinical, clerical, management and support staff were held over the summer of 2015 to explore engagement. These led to strengthened local plans to improve communications and engagement and confirmed that the SPEaC-app was a tool that would be welcomed across the Trust. It was subsequently supported by the executive and divisional management teams for full roll-out.

Training was delivered to SPEaC-app administrators in four departments in August 2015. Following a period of testing and refining the website (August- September 2015), a combination of dedicated terminals, tablets and kiosks, linking directly to the SPEaC-app website, were gradually installed and trialled from September 2015. By the end of November of that year, eleven areas within the Trust had begun trialling the web-based version of the SPEaC-app.

Results

SPEaC-app usage data:

By November 2016, the SPEaC-app was being actively used in 23 of around 100 areas, around 10% of all staff within the Trust. Between March 2015 and November 2016, 9259 entries were recorded (Table 1); this is an average of 15 entries per day across all areas. Table 1 breaks down the entries according to mood and theme: 45.7% of entries were positive and nearly 40% were negative. The full roll-out showed similar patterns to the pilot for the themes, with ‘team working’ being the most frequent. Figure 1 presents the number of responses each month together with the number of locations actively using the SPEaC-app. The data presented is limited to a) the time period after the SPEaC-app website launch (Sept 15) and b) to those locations that were identified as an area that actively wanted to use the SPEaC-app, and where training was provided to the SPEaC-app administrators in that location by the project team. The number of locations increased over time. However, while response rates through the website increased through the last quarter of 2015, they declined slightly over the first quarter of 2016 and then stayed relatively steady for the rest of that year. Reasons for the slight decline are unclear, but may be related to the tool becoming embedded into daily practice and problems that has locally implementable solutions being resolved more quickly, not requiring repeated input to the SPEaC-app.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Number of locations and average number of hits per location per month

Staff engagement:

The 2015 staff survey identified key areas where staff experiences have improved at the Trust since the 2014 survey. In particular, staff recommending the Trust as a place to work increased from 57% in 2014 to 61% in 2015. Key areas representing significant changes were ‘support from immediate managers’; ‘staff satisfaction with level of responsibility and involvement’ and ‘staff motivation at work’. These improvements cannot be attributed to this new mechanism in their entirety as the Trust had undertaken other measures to improve staff experience. On the other hand, ‘effective team working’ and ‘staff motivation at work’ compared less favourably with other acute trusts in England and no change was noted in ‘the ability to contribute towards improvements at work’.15 Analysis of the 2015 staff survey results identified that encouraging and supporting distributed decision making was key to improving staff engagement.

At staff workshops held specifically to understand the challenges around staff engagement, the SPEaC-app was often cited as an area of good practice by those who had it. Consequently, other areas started to actively request the SPEaC-app for use in their teams.

Staff Satisfaction with the SPEaC-app:

In January 2016, an online survey was conducted in the areas where staff were using the SPEaC-app. This took account of the additional features provided by the new website. Thirty staff (a mix of nursing and medical) from six areas took part in the survey (Acute Medical Unit; Ward 30 - a medical children's ward; Children's Emergency Department; Neonatal Intensive Care Unit: Paediatric Intensive Care Unit and the Surgical and Trauma Assessment Unit). Respondents overwhelmingly reported that it was valuable to be able to feedback about their shift (80%; n=24), with 70% (n=21) finding the responses and feedback now displayed to them on SPEaC-app useful. One participant commented that it is “very valuable as makes me feel listened to by management”. Of the respondents, 43% (n=13) said that they had noticed changes to their area of work as a result of the comments made, although 23% (n=7) responded that they had not noticed changes and the same number were ‘unsure’. Staff reported that they would appreciate having more opportunities to use the SPEaC-app through access from any desktop computer within the Trust via an icon link.

Lessons and limitations

Through the PDSA cycle which involved and sought feedback from users and SPEaC-app administrators, several factors which contribute to successful staff engagement using the SPEaC-app have been identified. In practical terms, usability is determined by staff's access to and the location of the dedicated terminals within each department. The reliability of the internet connection and the ease with which staff can log in affects their willingness to use the SPEaC-app, as does the perceived privacy. Departments where local manager feedback is given to staff regularly, rapidly and comprehensively are more likely to value and use the SPEaC-app. Equally, the SPEaC-app is used most in areas where there is evidence that staff comments are genuinely valued and acted on, leading to tangible change. This suggests that in order to embed the SPEaC-app in new areas, strong and consistent local management is required, alongside initial support from IT.

Two specific impacts of the SPEaC-app on staff engagement at UH Bristol have been identified:

Supporting local engagement with managers and their service delivery teams: It was evident that timely responses by departmental and hospital managers to the concerns raised via the SPEaC-app were crucial in demonstrating managerial commitment to listen to and act on staff feedback, while at the same time generating the trust required for staff to disclose concerns about working conditions, patient safety and quality of care. These two aspects became mutually reinforcing, producing positive impacts for the working environment and patient care. They also led to the SPEaC-app' sustainability across clinical and non-clinical departments.

Stimulating tangible improvements and supporting change: During design workshops, staff commented that many issues did not feel important enough to take to their manager, yet they did not feel able to resolve them themselves. By using the SPEaC-app, staff were able to raise minor problems, which were easily resolvable and potential ‘quick wins’ for improving both staff experience and patient care. For example, a member of staff came forward and offered to deal directly with any comments related to pharmacy stock levels in the CED. At local departmental level, the SPEaC-app facilitated the rapid sharing of experiences and ‘tips’ for making the new soluble prednisolone tablets more palatable to young children. At the organisation level, feedback via SPEaC-app from multiple areas motivated Trust managers to look at the roll out of new blood glucose machines, which was causing widespread frustration.

The process of any change can be perceived as disruptive by staff members, particularly those who are not directly involved in the decision making. During this project several changes led to a spate of negative mood and negative comments on the SPEaC-app. These included:

• the introduction of a new charting and clinical information system on the paediatric intensive care unit

• the introduction of new ways of team working on the Acute Medical Unit

• the introduction of ‘Medusa’, an on-line pharmacy intravenous drug infusion information system in CED.

In these areas, SPEaC-app allowed staff to anonymously voice their concerns or frustrations about service changes and enabled senior managers to respond and sometimes ‘myth bust’ where it was clear that potential concerns were unfounded. On average the ‘unhappiness’ gauged by SPEaC-app, lasted around two weeks while the new processes were embedded.

Measuring improvement in staff engagement was a challenge and with the rapid, ongoing roll-out of the SPEaC-app it is not clear how far improvements in the staff engagement scores, measured via the annual staff survey, can be attributed to its introduction. However, its uptake across the Trust has steadily increased and appears to be sustainable.

Conclusion

The project to improve staff engagement at UH Bristol was guided by a PDSA cycle to design, pilot, refine and improve a real-time staff feedback tool. Staff embraced this concept and engaged with giving real-time feedback once IT issues were resolved. Timely responses by managers and leaders were essential to build trust in the system and to maintain staff engagement over time. Both staff and managers were supportive of the SPEaC-app, demonstrated by the demand for its implementation in many areas within the Trust. By November 2016, the SPEaC-app was being used in 23 areas, where an average of 15 entries are made each day. Its spread across the Trust appears to be sustainable.

The usability is affected by access to, location of, reliability and perceived privacy of the SPEaC-app. It is valued and used most by staff in areas where feedback from local managers or team leaders is regular, rapid and comprehensive and where staff comments are acted upon, leading to tangible change. The SPEaC-app has the potential to support local engagement between managers and their service delivery teams, stimulate tangible improvements in service delivery and support the process of change. Longer term data are needed to determine whether the SPEaC-app can influence other factors including staff turnover, recruitment and retention.