Problem

The Later Life Mental Health Liaison Team (LLMHLT) at the Bristol Royal Infirmary (BRI) is an expanding service with a 38% increase in referral numbers from 2014 to 2015 (542 in 2014 to 746 in 2015). The team provides an in reach service to the hospital dealing with complex cases including dementia, delirium, depression, and all other psychiatric presentations.

As with almost all services, feedback from service users and their families is key to service development and improvement. The rate of feedback the LLMHLT at the BRI received prior to this project was small, averaging feedback from approximately two patients a month from an average of approximately 60 referrals up to August 2015. This data is insufficient to properly contribute to future service development and improvement.

Background

Getting service user feedback is considered an essential part of service provision in the current day NHS.1 ,2 Getting feedback directly from patients in the cohort served by the LLMHLT can be problematic given the high prevalence of cognitive deficits and the associated impact of this on engaging in giving feedback. Prior to this project the LLMHLT at the BRI gave a feedback questionnaire with a postage paid addressed envelope for return to the team, to all service users or their primary carer.

As preparation for this project a range of community and other liaison teams serving later life patients were contacted to understand how they collect feedback from similar cohorts. This research showed different teams use a variety of methods to try and solve the problem described but with no obviously successful solution in place. Such methods include telephoning service users and primary carers, sending out questionnaires after service provision has finished, and asking questions in person to service users or their family.

The Friends and Family Test has become an NHS standard for getting service user feedback. It aims to provide a simple metric for feedback and the guidance is that a follow up question should be used to gain extra information to drive improvement. Response rates across the NHS are expected to be a minimum of 15% and potentially much higher.3

Over the years there have been a plethora of NHS documents produced offering guidance on getting patient feedback. A number of these have been considered in attempting to solve the problem identified by this project.4 ,5 The most recent guidance relevant to this project is the March 2015 update to NHS England's ‘The Friends and Family Test’.6 Medline and BMJ Quality were searched for any projects focusing on this problem in a similar setting but none were found.

Given the challenging nature of getting feedback from this cohort of patients the aim of the project was to get feedback from 15% of patients or carers seen in a month. The aim was to achieve this within six months.

Baseline measurement

The measure was simple, it was the number of service users or carers served by the LLMHLT from whom feedback was received in a given month. This feedback was split into that received by post through the existing system and that received through any additional actions made during the project.

The baseline measurement was feedback from two service users in July 2015 and two service users in August 2015 which represents 3% of service users seen in either of those months.

Design

Having reviewed other Later Life Liaison teams' processes and discussed these with the multi-disciplinary team, the decision was made to implement face to face feedback and remain open to the idea of telephone gathered feedback.

The first implementation of this that represented the first PDSA cycle was done by using the current case load on the third Friday of each month. At the morning handover meeting the patients on the case load that had the cognitive ability such that they may be able to contribute patient feedback were identified. The Psychiatry trainee in the team visited each of these service users on the ward and asked them if they remembered being seen by the LLMHLT. Photos of the team were taken to the bedside to help remind the service users who the team was. A subjective assessment was made by the trainee of whether the service user understood who they were being asked to give feedback about and if they were able to identify the LLMHLT service from other care they had received in the hospital.

If so then they were asked the following two questions:

‘How likely are you to recommend our service to friends and family if they needed similar care and treatment?’

With a Likert scale of; ‘extremely likely’, ‘likely’, ‘neither likely nor unlikely’, ‘unlikely’, ‘extremely unlikely’, ‘don't know’ as answer options.

‘What did you like about the service, and what could we have done better?’

It was hypothesised that this would lead to feedback from a larger number of patients or carers than at baseline. It was also hoped that the existence of this project and discussions around it might cause patient feedback to be at the forefront of the team's mind and that this could lead to more of the existing questionnaires being given to service users or carers when initially seen. In turn it was hoped that this may lead to more questionnaires being received back by post.

Strategy

For each of the PDSA cycles data was collected on how many service users or carers feedback was received from. This feedback was broken down into that received by post through the existing system and that received through any additional actions made during the project.

After the first PDSA cycle it was noted that the intervention used a substantial amount of clinician time. In an attempt to reduce the clinician hours that the intervention required the second PDSA cycle involved using a medical student to collect the patient feedback. This involved the trainee explaining the setting and intervention to the medical student before leaving the medical student to visit each of the service users. It was hoped that this change would maintain the level of feedback but use less clinical time.

Unfortunately the level of feedback received through the intervention dropped in PDSA cycle two compared to one. For this reason the third PDSA cycle involved reverting back to using the trainee to collect feedback instead of a medical student. The third PDSA cycle also involved changing the one A4 document with photos identifying all the team members to an A4 sized photo of each to aid identification in a cohort of patients where eyesight is often a problem. It was hoped that this change would enable more patients to identify the team that feedback was being collected for and therefore lead to more feedback being collected.

The level of feedback received through the intervention increased after PDSA cycle three as hoped and it was notable that there had also been a steady climb in patient feedback received by post.

The fourth PDSA cycle involved doing the same as the third cycle but with an additional intervention. This involved searching notes of the service users who it was assessed were not able to contribute feedback, for contact details of a carer. An attempt was made to contact these carers to collect feedback using the same two questions put to service users. As the LLMHLT is a Liaison team it has always been accepted that the opinions of carers of our service users are very valuable. As stated in NHS England's guidance ‘Families and carers can offer valuable insights, and consideration should be given to capturing their views also’.6 It was hoped that this additional intervention would result in additional feedback being received and this was born out in the results.

Results

The number of service users or carers from whom we received feedback is summarised in table 1 and graph 1 below. The number of service users from whom feedback was collected through the intervention climbed as the PDSA cycles proceeded with a drop when a medical student was used instead of the trainee. After the fourth cycle feedback was gained from nine service users through the intervention. At the same time the number of service users from whom feedback was collected by post through the existing process also climbed as the project proceeded. This lead to a total of 13 and 14 instances of feedback in the final and penultimate months respectively, representing 21 and 18 percent of patients referred in those months.

• Download figure
• Open in new tab
• Download powerpoint

Graph 1

Run chart showing PFC at baseline and after each PSA cycle. PFC = Patient Feedback Collected

The results show a clear correlation between the effort made in collecting feedback and the amount received. As additional time and effort has been spent on interventions to try and get more feedback, more feedback has been received.

Lessons and limitations

The method of sampling the caseload once a month meant that a large number of service users were not asked face to face for feedback. However as was the case before this intervention all service users were left with a questionnaire asking for feedback when they were initially seen by the LLMHLT team. This intervention could be put in place more frequently to get more face to face feedback but that would require more resourcing.

This leads on to another limitations of this intervention, that up to half a day a month of trainee time was required. For this reason it would be useful to collect further data see how sustainable the improvements this project has achieved are. The cost of this time has to be weighed against the benefits. An attempt to utilise medical student time instead of trainee time was tried but caused a reduction in the amount of feedback received. It is unclear why this was and it may be that this was an anomaly and could be tried again.

A qualitative assessment of the feedback has not been undertaken for this project. However it should be acknowledged that a subjective assessment of the service user's ability to remember who the LLMHLT team were was being made and this could have affected the quality and reliability of the feedback. Also it may be that feedback which is received face to face is positively biased. Further work to check this hypothesis may be appropriate to see if these are limitations of the interventions described. If any bias were found this would have to be weighed against not having the feedback at all.

This project has not looked at whether the feedback has been or could be used to drive service change. This work needs to be undertaken in the future and as part of that work it may be appropriate to see if feedback from different sources or captured in different ways varies in how useful it is.

Conclusion

The aim of achieving feedback from 15% of service users has been achieved (21% in the penultimate month, 18% in the final month). A significant amount of the increase comes from an increase in the amount of patient feedback received by post through the existing system. This could be unrelated to the project but it could be reasonably suggested that the focus on patient feedback that this project has brought has increased the number of questionnaires given out at first assessments plus perhaps encouragement to return them. If this is the case then there is a question of how to continue this enthusiasm now that the project has finished. This could be achieved by simply having patient feedback as a standing item on the monthly team meeting agenda.

Although the intervention can claim a real increase in the amount of feedback received it has come at a not inconsiderable time and effort cost, and still only reaches approximately 20%. This firstly highlights the difficulty in getting feedback from service users, particularly perhaps in this cohort. This leads on to the question of whether this cost represents good value and whether the intervention should be continued into the future. Such an analysis is not easy and how much the feedback can be used to drive change is a key factor. In thinking about the value this work can add to the service we are reminded that understanding how best to capture patient feedback, how to use it, and the evidence base for such actions is a much written about, complex subject.7

If the team decide that this is a worthwhile intervention then it could be shared with other similar teams in the locality. The interventions are not site specific so they could be relevant to other Later Life Mental Health Liaison Teams that have the resource available to implement them. They would then need to be evaluated after an initial period of intervention. Any generalisability of the interventions beyond similar types of teams may be appropriate but a thorough consideration of alternatives would need to be undertaken to consider if better options were available with a different cohort of patients.