Article Text
PDF
Abstract
Background In tailoring a quality improvement programme for hospital-based physiotherapy, the original use of video recordings was replaced by using the tracer methodology.
Objective To examine the impact of a tailor-made quality improvement programme addressing patient communication on the professional development of hospital-based physiotherapists, and to evaluate barriers and facilitators as determinants of feasibility of the programme.
Methods A mixed-methods study was conducted. Participants were clustered in groups per hospital and linked with an equally sized group in a nearby hospital. Within the groups, fixed couples carried out a 2-hour tracer by directly observing each other’s daily work routine. This procedure was repeated 6 months later. Data from feedback forms were analysed quantitatively, and a thematic analysis of transcripts from group interviews was conducted.
Results Fifty hospital-based physiotherapists from 16 hospitals participated. They rated the impact of the programme on professional development, on a scale from 1 (much improvement needed) to 5 (no improvement needed), as 3.99 (SD 0.64) after the first tracer and 4.32 (SD 0.63) 6 months later; a mean improvement of 0.33 (95% CI 0.16 to 0.50). Participants scored, on a scale ranging from 1 to 5 on barriers and facilitators (feasibility), a mean of 3.45 (SD 0.95) on determinants of innovation, 3.47 (SD 0.86) on probability to use and 2.63 (SD 1.07) on the user feedback list. All participants emphasised the added value of the tracer methodology and mentioned effects on self-reflection and awareness most.
Conclusions The tailor-made quality improvement programme, based on principles of the tracer methodology, was associated with a significant impact on professional development. Barriers and facilitators as determinants of feasibility of the programme showed the programme being feasible.
- allied health occupations
- continuing education
- continuing professional development
- healthcare quality improvement
- professional competence
- quality improvement methodologies
Data availability statement
Data are available upon reasonable request. The raw data of this study are available from the first author at request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- allied health occupations
- continuing education
- continuing professional development
- healthcare quality improvement
- professional competence
- quality improvement methodologies
Introduction
Quality of care is defined as the degree of similarity between criteria of good care (desirable care) and the practice of care (actual care) at three levels of care organisation: quality of the care provider, quality of the institution and quality of the care system.1 Professional physiotherapy associations are continuously aiming to improve the quality of care provided by its members. The required qualities of the care provider are described in professional competency profiles.2–6
Assessment of clinical performance is a complex skill because clinical performance is highly context specific and cannot be standardised, given the uniqueness of the problem and the context of each patient. It requires professional judgement through observation and interpretation using global quality indicators.7–9 Within Miller’s pyramid, the lower levels of professional competence refer to what a professional knows, knows how to do, and shows how to do in a theoretical or simulated situation. The assessment of these behaviours can be standardised because the content and context are predefined. However, the higher level of Miller’s pyramid—how someone applies these behaviours in clinical practice—can only be assessed by direct observation or tracing professionals in the specific healthcare domain.10
To improve quality among her members, the Royal Dutch Society for Physiotherapy (KNGF) has designed an integral physiotherapy quality system, the Quality Register Physiotherapy Netherlands.11 Part of this system is a quality improvement programme for physiotherapists in primary care, based on peer assessment and feedback. Peer assessment is based on the appraisal of authentic clinical records and video recordings of clinical encounters using quality criteria for clinical performance such as patient communication. According to the Dutch physiotherapy competency profiles of the general and hospital-based physiotherapist, the physiotherapist maintains an effective relationship with the patient and his relatives or others involved, to ensure a high quality of service provided to patients and a high degree of patient satisfaction.12 13 Therefore, the physiotherapist should communicate in a clear, transparent, effective and efficient way during the physiotherapy session.
The quality improvement programme proved to be effective in creating awareness of clinical performance, improving evidence-based practice and patient-centeredness, and increasing motivation to self-direct quality improvement.14 However, this programme is not feasible in the context of hospital-based physiotherapy, because hospital-based physiotherapy is bound to specific regulations regarding patient privacy and protection of personal data. An alternative to video recordings was sought in the use of the tracer methodology, which is also being used in hospital quality systems such as Joint Commission International and Qmentum.15 16 In a tracer, a healthcare professional follows the track—the ‘trace’—of a colleague within the organisation for a certain period in order to identify quality issues.15 Because the literature on quality improvement shows that the tracer methodology is a useful method for assessing quality of care,16–19 we used the tracer methodology to improve patient communication of hospital-based physiotherapists as a tool for learning and professional development.
This study aims to examine the impact of a tailor-made quality improvement programme addressing patient communication on the professional development of hospital-based physiotherapists and to evaluate barriers and facilitators as determinants of the feasibility of implementing the programme.
Methods
Design and setting
This mixed-methods study was conducted from January to November 2019 in a convenience sample of Dutch hospital-based physiotherapists from 16 hospitals. All participants gave informed consent.
Patient and public involvement
Patients or the public were not involved in this study because the opinion of the professional was leading.
Participants
All managers of hospital-based physiotherapy departments in the Netherlands received written study information by email, including goals of the study and contact details of the first two authors, along with an invitation to participate in the study. Hospitals willing to participate were asked to invite a minimum of three and a maximum of four licensed hospital-based physiotherapists from their teams in the study. Participation was voluntary, and all participants provided written informed consent. Participating physiotherapists were clustered in groups per hospital and linked with a group of equal size in a nearby hospital. The tailor-made quality improvement programme consisted of two tracer visits per tracer cycle, one in each hospital (figure 1). The visiting physiotherapists carried out a tracer for 2 hours in which the observed physiotherapist carried out his or her normal work. Then a second appointment was made in the other hospitals within 2 weeks in which these roles were reversed.
Graphical display of the research method.
Development of the quality improvement programme
Because the original quality improvement programme on patient communication was fundamentally adapted to the setting of hospital-based physiotherapy, a pilot was conducted in 2018 with eight hospital-based physiotherapists from two hospitals to test the revised programme’s impact and feasibility. This unpublished pilot generated feedback to improve the provisional programme into a more sophisticated quality improvement as used in this study.
The quality improvement programme
Before the quality improvement programme was conducted, one of the coaches specifically trained by the KNGF for supervising quality improvement programmes introduced the programme to the participants, discussed rules of engagement, explained the use of the tracer feedback list and the tracer methodology, and designated fixed couples of physiotherapists, each from a different hospital for the duration of the study. During the execution of the tracer, the visiting physiotherapists recorded their findings on a tracer feedback list, consisting of seven global quality indicators for patient communication: the patient’s request for help, findings, outcomes, expectations, objectives, action planning and disruptive environmental stimuli (online supplemental appendix 1). Each of these items could be scored on a 5-point Likert scale ranging from 1 (much improvement needed) to 5 (no improvement needed). Randomly, some of the tracers were observed by the coach or one of the two primary researchers (RS, LvH-S), all with extensive experience in hospital-based physiotherapy within various hospitals and therefore were able to assess regular treatment behaviour, who evaluated in the first hour of the tracer to what extent the visiting physiotherapist influenced the treatment behaviour of the observed physiotherapist, to get an indication whether normal treatment behaviour was observed during the tracer, on a scale from 1 (no influence) to 10 (maximal influence).
Supplemental material
Immediately after the tracer, a session was planned in which the visiting physiotherapist provided feedback with comments and explanations to his or her peer. The trained coach facilitated this session. The observed physiotherapist was expected to write an improvement plan according to the feedback.
Follow-up and evaluation
After the second tracer visit, a group interview was conducted on the spot to evaluate the feasibility and self-perceived impact of the programme, moderated by (both trained and experienced interviewers) and supported by the coach. Participants were encouraged to speak freely and to respond to each other. These interviews, planned for 1 hour, were audio-recorded and transcribed verbatim afterwards. A predetermined topic list was used concerning five questions (online supplemental appendix 2).Within 2 weeks after the visits, participants received a questionnaire on barriers and facilitators of implementing the programme as determinants of the feasibility, where we defined feasibility as the probability that the programme in this form could be implemented in practice.20 This questionnaire was based on the Measurement Instrument for Determinants of Innovation (MIDI) questionnaire21–23 and consisted of 26 items, divided into three domains of barriers and facilitators: determinants of innovation, probability to use and used feedback list. For every statement in the list, scores could be given from 1 (completely disagree) to 5 (completely agree).
Subsequently, participants were asked to complete monthly self-assessment checklists on the impact of the programme on quality improvement (online supplemental appendix 3) during 5 months on which they could indicate how their professional performance in communicating with the patient developed in the past month, ranging from 1 (no development) to 10 (maximum development). The six items on the self-assessment checklist were: clarifying the patient’s request for help, formulating the findings in plain language, using results to draw up the treatment plan in consultation with the patient, aligning the mutual expectations, SMART formulation of the expected results in consultation with the patient and clearly formulating the planned actions in consultation with the patient.
Six months after the initial tracer visits, the fixed couples of physiotherapists were asked to visit each other again, using the same procedure and tracer feedback list as in the first tracer cycle. This second tracer cycle was conducted without a coach or an observer and formed the final study activity for participants.
Analysis of impact of the programme
Data from tracer findings using the tracer feedback list with quality indicators for patient communication from all tracers were analysed using SPSS V.25, with standard t-tests for paired samples to compare means and proportions within groups over time. Two-tailed p values of <0.05 were considered statistically significant. The average difference with 95% CIs was separately analysed. Data from the monthly self-assessment checklists were also analysed using SPSS V.25 to compare means and proportions over time. Data from the tracer findings and the self-assessment checklists were examined for correlation using Pearson’s correlation coefficient. Transcripts of the group interviews on impact were checked against the field notes by the two first authors. Thematic analysis was used to study the transcripts, being an appropriate and powerful method to use when seeking to understand a set of experiences, thoughts or behaviours across a data set.24 To encourage trustworthiness, the two primary researchers independently studied and coded eight transcripts. Differences in coding were discussed, and a codebook was created based on consensus. Analysis of transcripts was supported by ATLAS-ti V.8.4.25
Analysis of barriers and facilitators as determinants of feasibility
Data from the questionnaire about barriers and facilitators (feasibility) of the programme were analysed per item and category with SPSS V.25, using standard parametric tests. Transcripts of the group interviews on feasibility themes were checked against the field notes by the two first authors. Transcripts of the feasibility of the programme were analysed as mentioned above, supported by ATLAS-ti V.8.4.25
Results
Participants
Fifty Dutch hospital-based physiotherapists from 16 hospitals participated in the study. Characteristics of the participants are presented in table 1.
Characteristics of participating physiotherapists (n=50) from 16 hospitals
Impact of the programme
Quantitative evaluation
The distribution of tracer feedback list data did not deviate significantly from the standard normal distribution. Participants scored a total average of 3.99 (SD 0.64) on quality indicators for patient communication in the first round of tracers and 4.32 (SD 0.63) 6 months later. Mean difference in overall average scores between the first and second round of tracers was 0.33 (95% CI 0.16 to 0.50, p<0.05). Overview of mean scores for patient communication at T0 and T1 and their differences are presented in table 2.
Overview of mean scores for patient communication at T0 (first tracer cycles) and T1 (second tracer cycles), and their differences (paired sampled t-test)
A closer look at aspects of communication shows that five of seven aspects significantly improved: the patient’s request for help, findings, outcomes, expectations and action planning.
On the monthly self-assessment checklist for evaluating perceived development over the past month, the total average score of participants decreased from the first (T1; mean 6.00, SD 1.69) to the last (T5; mean 5.11, SD 2.81) moment of self-assessment (p>0.05) (table 3).
Mean and change scores of participants on the six items of the self-assessment during 5 monthly measuring moments in between tracer cycles 1 and tracer cycles 2, on a scale from 1 to 10
The correlation between the scores on the used feedback list during the tracer (T1–T0) and the self-assessment scores on the monthly sent list (T5–T1) was very low (r=0.03, p=0.89).
Qualitative evaluation
Analysing the group interviews for perceived impact on professional development, two themes were identified: ‘Peer feedback’ and ‘Learning outcome’.
Peer feedback
Participants mentioned the importance of group composition. Respondents expressed different views on whether peer feedback should be carried out with colleagues from the same or different specialisations, or on whether the goal of peer observation and feedback is learning in breadth or depth. Participants were convinced of the added value of peer feedback for quality improvement. They also indicated that mandatory assessment instead of peer feedback would harm the professional development process.
We sometimes put pelvic physiotherapists and pediatric physiotherapists together and they can ask each other stupid questions. The fact that you can ask stupid questions makes you think differently about your actions. Could be pretty useful. (PT30)
What fascinates me is the peer feedback, the methodical way of acting, which in terms of content is more focused on the profession. That makes me curious. Professional content that you can talk about, how do you do that, and then you can share the knowledge that someone else possesses but you haven't yet. (PT03)
The most commonly mentioned learning effect was self-reflection and awareness. The opportunity to see a colleague from another hospital at work was perceived as very useful. Although it was regularly mentioned that this was a unique opportunity to learn, it was also indicated that too much repetition of the method could lead to saturation. It was stated that the suggestion of assessment can have a negative effect and motivation and that training in feedback skills is an important prerequisite.
And we concluded that we were all lacking a little in giving information about a treatment beforehand. (PT16)
We are probably all open to feedback because we volunteer for it. Maybe giving feedback is not perfect, but you also have people who may not have signed up, who cannot or do not want to receive feedback. Then it is nice to know how best to give feedback, instead of saying “hey, you're doing it wrong”. (PT21)
Barriers and facilitators (feasibility) of the programme
Quantitative evaluation
For the barriers and facilitators (feasibility) of the quality improvement programme, participants scored on a scale from 1 (completely disagree) to 5 (completely agree), a mean of 3.45 (SD 0.95) on determinants of innovation, 3.47 (SD 0.86) on probability to use and 2.63 (SD 1.07) on the user feedback list (table 4).
Scores on the barriers and facilitators (feasibility) questionnaire (n=50)
During 18 tracers, in which influence of the visiting physiotherapist on the observed physiotherapist’s treatment behaviour was scored, a mean influence of 2.61 (SD 2.23) was recorded.
Qualitative evaluation
In eight group interviews, participating hospital-based physiotherapists discussed their views on the tailor-made quality improvement programme. Template analysis resulted in three main themes: ‘Organisation’, ‘Tracer’ and ‘Tracer feedback list’.
Organisation
Most participants felt that the quality improvement programme could be organised more efficiently, with clear instruction in advance, supervision by the coach and support by the department’s manager. They argued that although only a few physiotherapists participated in the programme, it still put a burden on the entire team.
Sometimes it can suddenly be about something that we as physiotherapists find interesting, but then you don’t reach the goal of the intervision. And then it is useful if there is somebody who can steer the process a bit. (PT34)
It’s quite a burden on the whole team. There are four of us gone now. And I know my colleagues are struggling to deal with the patient load. (PT12)
Tracer
All participants emphasised the value of using the tracer methodology as it gave a realistic insight into the daily practice of the observed physiotherapists. Because hospital-based physiotherapists are accustomed to regularly being watched by trainees or employees, they experienced the tracer as creating a safe learning environment allowing prompt feedback. Respondents recognised that the presence of an observer slightly altered their usual work situation.
It’s very direct and safe at the same time. It all becomes very real and because the group is this size, it is pleasant to do. (PT38)
And even though you know what is being judged, you forget that there is someone there. The fact that she was standing there to ‘judge’ me, you just forget. I was busier with my patient, how she was doing, and what was going on with her than I was with my colleague observing what I was doing. On the one hand, it has to do with the fact that you are going to act the way you normally act and on the other hand it has to do with the pressure of work. (PT48)
Tracer feedback list
Participants viewed the tracer feedback list as an applicable instrument and as a good guide for the tracer, although some remarked that the list should be filled out after completing the tracer. The biggest point of criticism regarding the tracer feedback list was its incompleteness because it did not cover all aspects of patient communication. Specifically, aspects of non-verbal communication were missing, such as considering the status of consciousness of the patient in, for example, the intensive care or neurology department. Comments were also made that the list could be a more convenient step-by-step guide, with better use of keywords, explanation of abbreviations used and use of the concept of treatment goal instead of request for help.
Such a form is a nice guide, a kind of format, but I would still like to see it worded differently. More specifically, that you have more of a list where you can tick some boxes. (PT27)
The form is an example. It is not an assessment form. It does not lead to a score. It should lead to feedback. It is a means to get feedback. If that is not enough, you have to do it another way. (PT28)
Discussion
Major findings
This study shows that a tailor-made quality improvement programme for hospital-based physiotherapists on patient communication has an impact on professional development. Participation in the programme was associated with a statistically significant increase in reported patient communication quality. This improvement was seen in five aspects of patient communication: the patient’s request for help, findings, outcomes, expectations and action planning. The presence of an observer during the tracer appeared to have almost no influence on the natural treatment behaviour of the participants, so there seems to be indication that normal treatment behaviour was observed during the tracer. Most participants were convinced of the added value of peer observation and feedback, through self-reflection and awareness.
The quantitative data on the MIDI questionnaire indicate that the offered quality improvement programme is in general feasible, where the used feedback list appears to be the largest barrier to using the programme. Important facilitators for the programme are a clear instruction in advance, supervision by a coach and support by the manager of the department.
Because the quality improvement programme consisted of two inter-related parts, tracer methodology and a monthly self-reflection questionnaire, it is hard to say which of these two interventions contributed to what amount to the results of the study. The very low correlation between the scores on the used feedback list and the self-assessment list, which we cannot properly explain at the moment, makes this point even more difficult to interpret.
Relation with similar studies
Comparing this study with equivalent studies in the literature is difficult because research on peer review is fragmented and has been limited to small-scale projects. Peer assessment and feedback (where we also include the use of the tracer method, on the understanding that the latter is then used formative rather than summative as is customary) on professional performance can be provided in several ways with different effects.26 Two randomised controlled trials showed that peer assessments were significantly more effective than group discussions in improving quality and in contributing to self-awareness among professionals.27 28 In agreement with the results of this study, an evaluation of a peer group model of supervision among allied healthcare workers reported improved skill development.29 Also, in a primary care setting, both self-assessment and peer assessment were shown to be effective in improving the physiotherapist’s clinical performance.30 In line with our findings, experienced physiotherapists perceived, observing colleagues while doing their job, to be the most powerful learning process that enabled them to develop their clinical expertise further.31 The results of this study thus support and extend previous findings of the potential value of peer observation and feedback as a quality improvement strategy.32 A meta-review of Ivers et al33 showed that feedback is more effective if the source is a colleague or supervisor, if it is given more than once, if the feedback is provided both in writing and orally, and if it contains concrete goals and an action plan. Although in our study feedback was only given once, compliance with these other features of effectiveness was met. It was also important that the feedback was provided by a licensed colleague from another hospital. Studies of feedback acceptance and its impact on subsequent professional development showed that feedback is better accepted and used if the provider is considered reliable and credible by the feedback recipient.34 35 In general, peer observation and feedback are seen as an innovative concept with the potential to use as a strategy for continuing professional development, where creating a feasible programme and a supportive environment to be able to do this properly is challenging.36–40 Especially these two findings are emphasised by our study.
Meaning and relevance of the findings
In our study, we observe an improvement of 7.6%, from 3.99 to 4.32 on the used scale, which is higher than the 4.3% that is found on average for audit and feedback effects.33 These findings of the impact of a quality improvement programme on patient communication are of significance for national boards of physiotherapy and other stakeholders in physiotherapy services. The results show that a tailor-made quality programme for hospital-based physiotherapists stimulates the development of their professional competence. Key component of the feasibility and relevance of this programme is that it is easy to apply in hospitals because it is linked to the tracer methodology that is already known in most hospitals.
Strengths and limitations of the study
Although exact figures and characteristics about the approximately 2000 hospital-based physiotherapists working in the Netherlands are lacking, it is the opinion of the board of the Dutch Association for Physiotherapy in Hospitals, based on their knowledge and experience of contacting Dutch hospitals, that the participants in this study, covering approximately one-fifth of all hospitals in the Netherlands, are representative of the overall hospital-based physiotherapy workforce. Whether the findings are also generalisable to hospital-based physiotherapists in other countries should be explored in further studies.
Granting potential members of the target group an important role in the development process of the quality improvement programme by joint scoring and evaluating assures that updating the programme with their data will result in a more successful programme. Research has shown that this sort of bottom-up quality improvement initiative might hold better and more sustainable results than external, top-down regulations.41–43 This is because shared social and professional norms are important predictors of behaviour change.44 45 The use of a mixed-methods design also adds value to this study: using the qualitative results clarifies the quantitative results of the study.46 A key limitation of the study is that neither the used feedback list during the tracer rounds or the self-reflection list has been formally tested for reliability and validity. Also, participants criticised some points of the used feedback list, which may affect its validity to some extent. Furthermore, the eight conducted group interviews were not anonymous, participants may have felt restrained to speak freely or one individual’s opinion can be over-represented, and voluntary participation (motivated participants) may distort the results. Also, the observed improvement in patient communication skills was only based on participating physiotherapists’ assessing each other and themselves, and may therefore have been subject to social desirability bias. Bias could also have occurred due to the impact of the impossibility of blinding the assessment.
Suggestions for further research
The feedback and self-assessment forms used in the study should be further adjusted and tested in a follow-up study for their clinimetric properties. To discover whether and in what amount the used tracer methodology or the monthly self-assessment list was responsible for the positive test results, the effect of both should be further investigated separately and in combination, of which examples can already be found in the literature.47 48 Further studies using independent and more objective assessment of communication skills are needed to substantiate our findings
Conclusion
A tailor-made quality improvement programme for patient communication of hospital-based physiotherapists showed a significant and relevant impact on participants’ communication skills through self-reflection and awareness. Barriers and facilitators of the programme as determinants of feasibility showed the programme being feasible.
On the qualitative components, this study was reported following the Consolidated criteria for Reporting Qualitative research.49 The entire study was reported following Revised Standards for Quality Improvement Reporting Excellence V.2.0.50
Data availability statement
Data are available upon reasonable request. The raw data of this study are available from the first author at request.
Ethics statements
Ethics approval
According to Dutch regulations, this study was considered exempt from review by the Medical Ethics Review Committee of Isala Hospital, Zwolle, the Netherlands and registered under number WMO 181127.
Acknowledgments
The authors would like to thank James Keogh, lecturer at Saxion University of Applied Sciences, for his contribution to translating quotes from the qualitative research from Dutch into authentic English.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors RS designed the work, collected, analysed and interpreted the data, and wrote the draft of the manuscript. LvH-S collected, analysed and interpreted the data, provided critical feedback on drafts of the manuscript, and finally approved this version to be published. TJH contributed to the design and methodology, provided critical feedback on drafts of the manuscript, and finally approved this version to be published. MM contributed to the analysis of the write-up of the study results, the development of the manuscript, and finally approved this version to be published. PB and PvdW contributed to the design and methodology, provided critical feedback on drafts of the manuscript, supervised the study, and finally approved this version to be published.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.