You are here

Article Text

Article menu

Download PDFPDF

XML
Original research
Development of quality indicators for departments of hospital-based physiotherapy: a modified Delphi study
  1. Rudi A Steenbruggen1,2,
  2. Roel van Oorsouw3,
  3. Marjo Maas1,4,
  4. Thomas J Hoogeboom1,
  5. Paul Brand5,6,
  6. Philip van der Wees1
  1. 1IQ Healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
  2. 2School of Health, Saxion University of Applied Sciences, Enschede, The Netherlands
  3. 3Physiotherapy, Radboudumc, Nijmegen, The Netherlands
  4. 4Institute of Allied Health Studies, HAN University of Applied Sciences, Nijmegen, The Netherlands
  5. 5Medical Education, Isala Hospitals, Zwolle, The Netherlands
  6. 6Clinical Education, UMCG, Groningen, The Netherlands
  1. Correspondence to Rudi A Steenbruggen; r.a.steenbruggen{at}saxion.nl

Abstract

Background International hospital accreditation instruments, such as Joint Commission International (JCI) and Qmentum, focus mainly on hospital policy and procedures and do not specifically cover a profession such as hospital-based physiotherapy. This justifies the need for a quality system to which hospital-based physiotherapy can better identify, based on a common framework of quality indicators for effective quality management.

Objective This study aimed to identify the most important quality indicators of a hospital-based physiotherapy department in the eyes of hospital-based physiotherapists and their managers.

Methods Based on input from three focus groups and a structured literature review, a first set of quality indicators for hospital physiotherapy was assembled. After checking this set for duplicates and for overlap with JCI and Qmentum, it formed the starting point of a modified Delphi procedure. In two rounds, 17 hospital-based physiotherapy experts rated the quality indicators on relevance through online surveys. In a final consensus meeting, quality indicators were established, classified in quality themes and operationalised by describing for each theme the rationale, specifications, domain and type of indicator.

Results Three focus groups provided 120 potential indicators, which were complemented with 18 potential indicators based on literature. After duplicate and overlap check and the Delphi procedure, these 138 potential indicators were reduced to a set of 56 quality indicators for hospital-based physiotherapy. Finally, these 56 indicators were condensed into 7 composite indicators, each representing a quality theme based on definitions of the European Foundation for Quality Management.

Conclusion A set of 56 quality indicators, condensed into 7 composite indicators each representing a quality theme, was developed to assess the quality of a hospital-based physiotherapy department.

  • continuous quality improvement
  • healthcare quality improvement
  • hospital medicine
  • quality improvement
  • standards of care
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjoq-2019-000812

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Hospital-based physiotherapy can play a significant role in the multidisciplinary treatment of hospitalised patients by focusing primarily on functional mobility as an important part of the patient’s functional health condition, before, during and after hospitalisation.1 Quality of hospital-based physiotherapy can be defined as the degree of similarity between criteria of good care (desirable care) and the practice of care (actual care).2 This degree of similarity can be quantified through quality indicators which can be classified into structure, process and outcome indicators.2–4 In hospital care, quality indicators are being used as a tool in quality improvement cycles, for example, to decrease morbidity and mortality, or to qualify for a recognised quality approval such as Joint Commission International (JCI) or Qmentum.5–8 These globally expanding accreditation instruments focus mainly on hospital policy and procedures and do not specifically cover a profession like hospital-based physiotherapy.7 8

    To determine whether the practice of care provided by hospital-based physiotherapists meets the criteria of desirable care and to stimulate continuous quality improvement, a quality system is required. Such a quality system typically consists of four components: a professional profile with core competencies, a system of standards and guidelines, external accountability and systematic quality control.9 Because accreditation instruments such as JCI and Qmentum cover only the medical and nursing staff as recognisable individual disciplines,7 8 these instruments do not allow systematic quality control of hospital-based physiotherapy departments. This justifies the need for a tailored quality system for hospital-based physiotherapy. To assess the quality of the provided hospital physiotherapy care, the alignment between performance, strategy, vision and desired outcomes needs to be established.10 Profession-specific quality assessment feedback can help physiotherapists to identify areas of professional practice that need improvement. This process has been recommended as an essential component in raising the standards of hospital-based physiotherapy care.11 12

    Because organisational restructuring due to financial issues is common in multidisciplinary hospital care, a quality system for hospital-based physiotherapy should be sufficiently flexible towards organisational changes and the associated changing roles of hospital physiotherapists. This continuously changing role places an increased emphasis on higher level skills in patient care and interprofessional communication and collaboration skills, and highlights the ongoing importance of professionalism.13 When hospital restructuring takes place, physiotherapists, as part of allied healthcare, need their own conceptual model to describe the effect of hospital restructuring on their professional role.14 15 Hospitals nowadays struggle how to organise these changing roles: a move to programme management from a traditional department structure affects the professional practice of physiotherapists, reporting both positive and negative effects on professional affect, professional practice and patient care.16

    A quality system for an individual profession such as hospital-based physiotherapy should be based on a common framework for effective quality management to which this specific profession can better identify, and which is independent of hospital restructuring. Because such a system is lacking, the aim of this study is to identify the most important indicators for the quality of a hospital-based physiotherapy department in the eyes of hospital-based physiotherapists and their managers, as a first step towards establishing a valid and useful quality system for the profession.

    Methods

    A modified RAND/UCLA (RAND Corporation/University of California at Los Angeles) Appropriateness Method Delphi study was used to establish a list of the key quality indicators of hospital-based physiotherapy.17 The study was conducted from May 2017 to January 2019 in a convenience sample of Dutch hospital-based physiotherapists and their managers, taken from the member databases of the Dutch Association of Physical Therapy in Hospitals (NVZF) and the Dutch Association of Managers of Physical Therapy (VLF). According to Dutch regulations, ethical review was not required because there were not any patients or interventions involved in this study.

    Patient and public involvement

    No patients or public were involved in this study because the opinion of the professional was the main topic. The results of the study will be disseminated to the members of the Delphi Panel and members of the NVZF and VLF during the annual congress of the NVZF.

    The study consisted of three stages.

    Stage 1: focus groups

    During a scheduled meeting in May 2017, all VLF members were informed that focus groups on quality indicators for hospital-based physiotherapy would be part of the meeting’s programme. Attendees expressing interest in participating in the focus groups were divided into three equally sized focus groups. Each focus group interview was moderated by an NVZF board member experienced in the subject of quality of hospital-based physiotherapy. Every moderator received detailed moderation instructions from one of the primary researchers and was supported by a research team member who made field notes during the interviews. The focus group interviews were semistructured following a topic list (online supplementary appendix 1), concentrating on two main questions:

    Supplemental material

    1. Which do you think are the most important elements that demonstrate the quality of a hospital-based physiotherapy department?

    2. What do you think is a good and short definition of quality of a hospital-based physiotherapy department?

    Respondents were encouraged to speak freely and to respond to each other. The interviews, planned for 1 hour, were audio recorded and transcribed. The transcripts were checked against the field notes and then returned for a member check. The two primary researchers independently analysed the transcripts using Atlas.ti V.7,18 19 to identify all the quality topics that were proposed by the participants during the group interviews. Differences were resolved by discussion until consensus was reached.

    Stage 2: literature search

    A structured literature search was conducted to complement the initial set of quality indicators from stage 1. PubMed was searched in February 2018 using the following search string: ‘((‘Physical Therapy Department, Hospital’ [Mesh]) AND ((((‘Quality of Health Care’ [Mesh] OR ‘Quality Assurance, Health Care’ [Mesh]))) OR quality [Title/Abstract])) AND ((((‘Organization and Administration’ [Mesh] OR ‘organization and administration’ [Subheading]))) OR ((Organization*[tiab] OR Administrat*[tiab])))’. The two primary researchers independently reviewed the retrieved titles and abstracts, and the full text of potentially relevant articles. The reference lists of retrieved full-text articles were hand-searched for potentially relevant articles. Relevant articles were independently hand-searched for additional quality indicators. Differences were discussed until the two primary researchers agreed on a set of additional quality indicators which were added to the set obtained in stage 1. The resulting set of indicators was checked for duplicates and for overlap with JCI and Qmentum quality indicators and classified by following the EFQM (the European Foundation for Quality Management) Excellence Model (figure 1).20

    Figure 1
    Figure 1

    The EFQM (the European Foundation for Quality Management) excellence model (Source: www.efqm.org).

    Stage 3: Delphi rounds

    All 180 NVZF members, hospital-based physiotherapists and managers of hospital-based physiotherapy received written study information by email, along with an invitation to join the Delphi panel. Members who responded positively to the invitation took part in the Delphi panel. The Delphi procedure consisted of two online survey rounds and a consensus meeting.

    In the first two online survey rounds, the IQ (Scientific Center for Quality of Healthcare) Consensus Tool was used.21 This tool supports groups in achieving consensus, based on the Modified RAND Delphi Method. Each quality indicator was valued by the group members using a nine-point Likert scale, ranging from very low to very high value. The tool combines the highest tertile, median and top 3 scores to arrive at a ‘selection’, ‘discussion’ or ‘no selection’ score, using the Campbell criteria.22 Group members were also asked whether indicators showed overlap with known JCI or Qmentum quality indicators and if they thought indicators were missing.

    In preparation for the consensus meeting, the research team put together the first draft, assembling the indicators by theme. The meeting started with a presentation on the study purpose and setup, followed by a full-day group discussion moderated by the first author who is highly experienced in running meetings in similarly sized groups. Throughout the meeting, changes to the developing indicator set were projected on a screen in real time. The meeting was audio recorded, and comments and changes were recorded by a research team member.

    First, the consensus panel members were asked to confirm the quality indicators they had selected or deleted in the two online survey rounds. Second, they discussed whether the indicators with the label ‘discussion’ should be included or deleted. All selected indicators were condensed into composite indicators, each representing a quality theme based on definitions of the EFQM. These composite indicators were operationalised by describing the rationale, specifications, domain (following EFQM) and type of indicator (structure, process or outcome). When differences in opinion hampered the process, decisions were made through consensus. When unanimous consensus appeared impossible, voting by hand raising took place, where a >75% majority was needed for adopting a proposal for amendment. If this majority was not attained, the proposal was rejected.

    One week after the meeting, the resulting quality indicator list was sent to the participants by email, to allow feedback on the text. This feedback was processed by the two primary researchers, leading to the final set of themes and indicators.

    Results

    Focus groups

    The quarterly VLF meeting in May 2017 was attended by 26 of 70 members. Characteristics of the participants are presented in table 1. Attending members were divided into three focus groups.

    View this table:
    Table 1

    Group member characteristics

    Analysis of the three focus groups’ transcripts resulted in 120 potential quality indicators for hospital-based physiotherapy.

    Literature search

    The PubMed search yielded 163 hits, 12 of which were considered relevant after reading title and abstract.11–16 23–28 These 12 articles were searched for quality indicators not found during stage 1. The resulting 18 indicators were added to the 120 potential quality indicators for hospital-based physiotherapy departments obtained in stage 1, providing a total of 138 potential indicators. Deleting duplicates and indicators present in JCI and Qmentum resulted in a list of 103 potential quality indicators of hospital-based physiotherapy (online supplementary appendix 2). These were classified according to the EFQM model.

    Delphi rounds

    The Delphi panel consisted of 17 of 180 NVZF members. Characteristics of the participants are presented in table 1.

    In the first online survey round (October 2018) with a 100% response, 43 quality indicators were selected, 35 were labelled as ‘discussion’ and 25 were not selected. None of the group members indicated missing quality indicators or overlaps with JCI or Qmentum indicators.

    In the second online survey round (November 2018) with a 100% response, the 35 ‘discussion’ indicators were reoffered to the group members. The members selected 13 indicators to be included, 7 were labelled as ‘discussion’ and 15 were not selected.

    During the consensus meeting in January 2019, with a 94% attendance of group members, consensus was reached to definitively not select the seven indicators which were labelled as ‘discussion’. With this decision, 56 quality indicators for hospital-based physiotherapy were set (figure 2). The participants agreed that these 56 quality indicators could be grouped into seven themes (composite indicators, table 2). Every theme with the specific indicators was operationalised by describing the rationale, specifications, domain (according to EFQM) and type of indicator (structure, process or outcome) (online supplementary appendix 3).

    Figure 2
    Figure 2

    Number of quality indicators for hospital-based physiotherapy during the three stages of the process.

    View this table:
    Table 2

    The 56 quality indicators for hospital-based physiotherapy sorted into 7 composite Indicators

    Discussion

    Major findings

    This study aimed to identify important quality indicators of a hospital-based physiotherapy department in the eyes of hospital-based physiotherapists and their managers providing an inside-out perspective. A RAND-modified Delphi procedure resulted in 56 indicators condensed into 7 composite indicators each representing a quality theme: (1) culture of continuous learning, improvement and open dialogue; (2) promotion of staff expertise that is consistent with the demand for care; (3) using a planning and control cycle to work on achieving its goals in the short, medium and long term, with a policy plan that fits within the frameworks of organisational policy; (4) forming an integral part of the overall patient and hospital process; (5) implementing a patient-oriented policy; (6) systematically ensuring that the physiotherapeutic interventions undertaken by employees are of the highest possible quality; (7) collecting feedback on performance from stakeholders and staff and taking action that is based on this feedback.

    Meaning of the findings

    These composite quality indicators can serve as the first step towards a quality system for hospital-based physiotherapy, meeting the hospital-based physiotherapy’s need of such a system. With these findings, based on a common framework, a foundation could be laid for a method of quality improvement of hospital-based physiotherapy in the Dutch situation. This method could rely on an assessment procedure in which the specific profession of hospital-based physiotherapy recognises itself better than in systems such as JCI or Qmentum. Assessment of professional performance, including both clinical and organisational performance, can be applied for summative or formative purposes. Summative assessments are used to decide on academic progress, certification or accreditation (such as JCI and Qmentum). Formative assessments are used to support continuous learning and quality improvement.29 Most indicators found in this study are structure elements, a few are process indicators and none are outcome indicators. This suggests that a formative assessment would be better in place right now.

    Relation with similar studies

    To our knowledge, this is the first set of quality indicators for hospital-based physiotherapy. A known set of quality indicators for a specific discipline in Dutch hospitals is one used by obstetric caregivers. The user experience of this set shows high awareness of and reasonable contribution to quality indicators,30 which supports the similar use of quality indicators for hospital-based physiotherapy. Learning from nursing initiatives concerning the use of quality indicators, it is also important that leaders empower their staff in the process of transforming towards a higher quality delivery system.31

    Strengths of the study

    This study has several strengths. Although the set of indicators was developed from the perspective of Dutch hospital-based physiotherapy departments, we consciously used established indicator development methodology—that is, the RAND-modified Delphi procedure—to improve its validity and generalisability.17 32 33 The response rates to the online surveys were high with 100% in the two online rounds, and 94% attendance of group members during the consensus meeting. Because members of the focus groups and the Delphi Panel represent approximately 30% of the total amount of hospitals in the Netherlands, in which organisational positioning of a department of hospital-based physiotherapy can differ strongly, we assume that the presented list of quality themes and indicators is sufficiently flexible towards organisational changes and changing roles of hospital-based physiotherapy. Research has shown that bottom-up quality improvement initiatives, such as communities of practice and professional networks focusing on collaborative learning, might hold better and more sustainable results than external, top-down regulations,34–36 because shared social and professional norms are important predictors of behaviour change.37 38 This is the main reason why we chose a bottom-up approach for our study.

    Limitations

    We acknowledge the following limitations. Although Delphi consensus groups can produce collective answers, the achieved consensus is not necessarily accurate; bias can occur in the consensus meeting because one individual’s opinion can be overrepresented.39 Also, the group size of 17 members was slightly larger than the ideal Delphi group size of 5–11 members.40 Since the consensus meeting was not anonymous, respondents may have felt restrained to speak freely. Although we consciously invited representatives of hospital-based physiotherapy, their scope may be limited affecting the validity and generalisability. Stakeholders from outside the profession of hospital-based physiotherapy were not involved in the development process and the set of quality indicators has not yet been subjected to external review.41

    Suggestions for further research

    The presented list of quality indicators, classified in quality themes, should be enriched in future research. By including key stakeholders of hospital-based physiotherapy such as patients, medical specialists and hospital management, an outside-in perspective could be provided. Combining the results from both the inside and outside perspectives could provide the ideal mix of indicators for good quality of hospital-based physiotherapy. In addition, our set of quality indicators should be further assessed for reliability, validity and acceptability. Reviewing the list by national and international hospital-based physiotherapy specialists could contribute to these points and to the issue of generalisation. It is conceivable that after these steps, a foundation could be laid for a method of quality improvement of hospital-based physiotherapy, at least in the Dutch situation.

    Conclusions

    This study presents a set of 56 quality indicators, condensed into 7 composite indicators each representing a quality theme based on definitions of the EFQM. These indicators are important and relevant to Dutch hospital-based physiotherapy departments and their managers. To our knowledge, this is the first time such a quality set for hospital-based physiotherapy has been presented. By involving relevant stakeholders and external reviewers, this set can be further assessed on reliability, validity and acceptability, laying a foundation for a method of quality improvement of hospital-based physiotherapy.

    This study was reported following the Consolidated criteria for Reporting Qualitative research (COREQ).42

    Acknowledgments

    The authors would like to express their appreciation to the members of the Delphi Panel for their important contribution to this study.

    References

    1. ↵
      1. Euwes M,
      2. Engelbert RHH,
      3. Geleijn E, et al
      . Beroepscompetentieprofiel ziekenhuisfysiotherapeut. Amersfoort; NL: NVZF, 2010: 25–40.
    2. ↵
      1. Donabedian A
      . The definition of quality and approaches to its assessment: Vol 1. explorations in quality assessment and monitoring. Ann Arber, MI: Health Administration Press, 1980.
    3. ↵
      1. Donabedian A
      . The quality of care. How can it be assessed? JAMA 1988;260:1743–8.doi:10.1001/jama.260.12.1743pmid:http://www.ncbi.nlm.nih.gov/pubmed/3045356
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Salzer MS,
      2. Nixon CT,
      3. Schut LJ, et al
      . Validating quality indicators. quality as relationship between structure, process, and outcome. Eval Rev 1997;21:292–309.doi:10.1177/0193841X9702100302pmid:http://www.ncbi.nlm.nih.gov/pubmed/10183280
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Wright J,
      2. Dugdale B,
      3. Hammond I, et al
      . Learning from death: a hospital mortality reduction programme. J R Soc Med 2006;99:303–8.doi:10.1177/014107680609900617pmid:http://www.ncbi.nlm.nih.gov/pubmed/16738373
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Haynes AB,
      2. Weiser TG,
      3. Berry WR, et al
      . A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9.doi:10.1056/NEJMsa0810119pmid:http://www.ncbi.nlm.nih.gov/pubmed/19144931
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. The Joint Commission
      . The joint Commission big book of policies and procedures for hospitals. Oak Brook, Ill: Joint Commission Resources, 2019.
    8. ↵
      The Qmentum accreditation program. Available: https://accreditation.ca/intl-en/accreditation/qmentum/ [Accessed 16 Apr 2019].
    9. ↵
      1. Klazinga N,
      2. Lombarts K,
      3. van Everdingen J
      . Quality management in medical specialties: the use of channels and dikes in improving health care in the Netherlands. Jt Comm J Qual Improv 1998;24:240–50.doi:10.1016/S1070-3241(16)30378-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/9626617
      OpenUrlPubMed
    10. ↵
      1. Jones R,
      2. Jenkins F
      . Key tools and techniques in management and leadership of the allied health professions. Milton Keynes: Radcliffe Publishing Ltd, 2011: 1–29.
    11. ↵
      1. Hammond R,
      2. Lennon S,
      3. Walker MF, et al
      . Changing occupational therapy and physiotherapy practice through guidelines and audit in the United Kingdom. Clin Rehabil 2005;19:365–71.doi:10.1191/0269215505cr784oapmid:http://www.ncbi.nlm.nih.gov/pubmed/15929504
      OpenUrlCrossRefPubMed
    12. ↵
      1. Coulson-Thomas C
      . Re-Engineering hospital and healthcare processes. Health Estate J 1997;51:14–15.pmid:http://www.ncbi.nlm.nih.gov/pubmed/10174371
      OpenUrlPubMed
    13. ↵
      1. Lopopolo RB
      . Hospital restructuring and the changing nature of the physical therapist's role. Phys Ther 1999;79:171–85.doi:10.1093/ptj/79.2.171pmid:http://www.ncbi.nlm.nih.gov/pubmed/10029057
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Lopopolo RB
      . The effect of hospital restructuring on the role of physical therapists in acute care. Phys Ther 1997;77:918–32.doi:10.1093/ptj/77.9.918pmid:http://www.ncbi.nlm.nih.gov/pubmed/9291949
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Lopopolo RB
      . Development of the professional role behaviors survey (probes). Phys Ther 2001;81:1317–27.doi:10.1093/ptj/81.7.1317pmid:http://www.ncbi.nlm.nih.gov/pubmed/11444995
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Miller PA,
      2. Solomon P
      . The influence of a move to program management on physical therapist practice. Phys Ther 2002;82:449–58.doi:10.1093/ptj/82.5.449pmid:http://www.ncbi.nlm.nih.gov/pubmed/11991798
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Fitch K,
      2. Bernstein SJ,
      3. Aguilar MD, et al
      . The RAND/UCLA appropriateness method user's manual. Santa Monica CA: RAND, 2001: 10–53.
    18. ↵
      1. Olson K,
      2. Young RA,
      3. Schultz IZ
      . Handbook of qualitative health research for evidence-based practice. New York NY: Springer, 2015.
    19. ↵
      1. Friese S
      . Qualitative data analysis with ATLAS.ti. London: Sage Publications Ltd, 2012.
    20. ↵
      At EFQM, our purpose is to improve the performance of organisations and their ability to manage change and transformation. Available: https://www.efqm.org/ [Accessed 23 Apr 2019].
    21. ↵
      IQ consensus tool. Available: https://www.iqhealthcare.nl/nl/kennisbank/tools/iq-consensus-tool/ [Accessed 23 Apr 2019].
    22. ↵
      1. Campbell SM,
      2. Roland MO,
      3. Quayle JA, et al
      . Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they? J Public Health Med 1998;20:414–21.doi:10.1093/oxfordjournals.pubmed.a024796pmid:http://www.ncbi.nlm.nih.gov/pubmed/9923948
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Braithwaite J,
      2. Westbrook M
      . Rethinking clinical organisational structures: an attitude survey of doctors, nurses and allied health staff in clinical directorates. J Health Serv Res Policy 2005;10:10–17.doi:10.1177/135581960501000105pmid:http://www.ncbi.nlm.nih.gov/pubmed/15667699
      OpenUrlCrossRefPubMed
    24. ↵
      1. Hanekom SD,
      2. Louw Q,
      3. Coetzee A
      . The way in which a physiotherapy service is structured can improve patient outcome from a surgical intensive care: a controlled clinical trial. Crit Care 2012;16:R230.doi:10.1186/cc11894pmid:http://www.ncbi.nlm.nih.gov/pubmed/23232109
      OpenUrlCrossRefPubMed
    25. ↵
      1. Adamson BJ,
      2. Lincoln MA,
      3. Cant RV
      . An analysis of managerial skills for the current and future health care environment. J Allied Health 2000;29:203–13.pmid:http://www.ncbi.nlm.nih.gov/pubmed/11147186
      OpenUrlPubMed
    26. ↵
      1. Ziviani J,
      2. Wilkinson SA,
      3. Hinchliffe F, et al
      . Mapping allied health evidence-based practice: providing a basis for organisational realignment. Aust Health Rev 2015;39:295–302.doi:10.1071/AH14161pmid:http://www.ncbi.nlm.nih.gov/pubmed/25935069
      OpenUrlPubMed
    27. ↵
      1. Wilson S
      . Instructional insights: restructuring the fieldwork experience: one clinic's journey. Occup Ther Health Care 2017;31:365–73.doi:10.1080/07380577.2017.1381895pmid:http://www.ncbi.nlm.nih.gov/pubmed/29039722
      OpenUrlPubMed
    28. ↵
      1. Turner-Stokes L,
      2. Williams H,
      3. Abraham R, et al
      . Clinical standards for inpatient specialist rehabilitation services in the UK. Clin Rehabil 2000;14:468–80.doi:10.1191/0269215500cr349oapmid:http://www.ncbi.nlm.nih.gov/pubmed/11043872
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Black P,
      2. Wiliam D
      . Developing the theory of formative assessment. Educ Asse Eval Acc 2009;21:5–31.doi:10.1007/s11092-008-9068-5
      OpenUrl
    30. ↵
      1. Cellissen E,
      2. Franx A,
      3. Roes KCB
      . Use of quality indicators by obstetric caregivers in the Netherlands: a descriptive study. Eur J Obstet Gynecol Reprod Biol 2017;211:177–81.doi:10.1016/j.ejogrb.2017.02.016pmid:http://www.ncbi.nlm.nih.gov/pubmed/28284930
      OpenUrlPubMed
    31. ↵
      1. Butler GA,
      2. Hupp DS
      . Pediatric quality and safety: a nursing perspective. Pediatr Clin North Am 2016;63:329–39.doi:10.1016/j.pcl.2015.11.005pmid:http://www.ncbi.nlm.nih.gov/pubmed/27017039
      OpenUrlCrossRefPubMed
    32. ↵
      1. Hasson F,
      2. Keeney S,
      3. McKenna H
      . Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008–15.pmid:http://www.ncbi.nlm.nih.gov/pubmed/11095242
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Boulkedid R,
      2. Abdoul H,
      3. Loustau M, et al
      . Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011;6:e20476–9.doi:10.1371/journal.pone.0020476pmid:http://www.ncbi.nlm.nih.gov/pubmed/21694759
      OpenUrlCrossRefPubMed
    34. ↵
      1. Grol R
      . Quality improvement by peer review in primary care: a practical guide. Qual Health Care 1994;3:147–52.doi:10.1136/qshc.3.3.147pmid:http://www.ncbi.nlm.nih.gov/pubmed/10139412
      OpenUrlFREE Full Text
    35. ↵
      1. Butterfield R,
      2. McCormick B,
      3. Anderson R, et al
      . Quality of NHS care and external pathway peer review, 2012. Available: https://www.chseo.org.uk/downloads/report3-peerreview.pdf [Accessed 10 Jun 2019].
    36. ↵
      1. Pronovost PJ,
      2. Hudson DW
      . Improving healthcare quality through organisational peer-to-peer assessment: lessons from the nuclear power industry. BMJ Qual Saf 2012;21:872–5.doi:10.1136/bmjqs-2011-000470pmid:http://www.ncbi.nlm.nih.gov/pubmed/22562877
      OpenUrlAbstract/FREE Full Text
    37. ↵
      1. Prochaska JO,
      2. Redding CA,
      3. Evers KE
      . Health behavior and health education. Hoboken NJ: Wiley & Sons, 2008: 97–121.
    38. ↵
      1. Ajzen I
      . Nature and operation of attitudes. Annu Rev Psychol 2001;52:27–58.doi:10.1146/annurev.psych.52.1.27pmid:http://www.ncbi.nlm.nih.gov/pubmed/11148298
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Howell A-M,
      2. Burns EM,
      3. Hull L, et al
      . International recommendations for national patient safety incident reporting systems: an expert Delphi consensus-building process. BMJ Qual Saf 2017;26:150–63.doi:10.1136/bmjqs-2015-004456pmid:http://www.ncbi.nlm.nih.gov/pubmed/26902254
      OpenUrlAbstract/FREE Full Text
    40. ↵
      1. Waggoner J,
      2. Carline JD,
      3. Durning SJ
      . Is there a consensus on consensus methodology? descriptions and recommendations for future consensus research. Acad Med 2016;91:663–8.doi:10.1097/ACM.0000000000001092pmid:http://www.ncbi.nlm.nih.gov/pubmed/26796090
      OpenUrlPubMed
    41. ↵
      1. Abrahamyan L,
      2. Boom N,
      3. Donovan LR, et al
      . An international environmental scan of quality indicators for cardiovascular care. Can J Cardiol 2012;28:110–8.doi:10.1016/j.cjca.2011.09.019pmid:http://www.ncbi.nlm.nih.gov/pubmed/22154233
      OpenUrlCrossRefPubMed
    42. ↵
      1. Tong A,
      2. Sainsbury P,
      3. Craig J
      . Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349–57.doi:10.1093/intqhc/mzm042pmid:http://www.ncbi.nlm.nih.gov/pubmed/17872937
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors RS designed the work, collected, analysed and interpreted the data, and wrote the draft of the manuscript. RvO collected, analysed and interpreted the data, provided critical feedback on drafts of the manuscript and finally approved this version to be published. TJH and MM contributed to the analysis of the write up of the study results, the development of the manuscript and finally approved this version to be published. PB and PvdW contributed to the design and methodology, provided critical feedback on drafts of the manuscript, supervised the study and finally approved this version to be published.

    • Funding This study was funded by the Dutch Association of Physical Therapy in Hospitals (NVZF).

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available on reasonable request. Data consist only of deidentified participant data during two stages of the Delphi procedure. After a reasonable request to the first author, these data are available.