Article Text
PDF
Abstract
Introduction Emergency medical regulation is a risky activity. In France, emergency medical societies have proposed activity and performance indicators, but their lists are non-exhaustive, unstructured and used heterogeneously among emergency medical call centres (Centres de Réception et de Régulation des Appels, CRRA). Our objective was to build by means of regional stakeholder consensus an operational quality dashboard for CRRAs.
Methods We conducted an observational step in a French CRRA from June to September 2018 and at the same time listed existing activity and quality indicators through a rapid international literature review. We adapted and classified all indicators identified in a structured table. We prioritised them from April to September 2019 by seeking consensus with one regulator physician and one medical regulation assistant from the 13 CRRAs of the largest French region. We used an adapted Delphi method with a prioritisation scale from 1 to 9.
Results The rapid review of literature included 33 studies among the 414 identified and, with the first observational step, resulted in a list of 360 quality indicators covering the following areas: material resources, human resources, quality approach, call handling and postcall support. 15 of the 26 members participated in the entire process. Seventy indicators were considered as priorities with strong agreement among participants. We built an operational dashboard of quality indicators deemed high priority and provided 70 descriptive indicator sheets.
Conclusion Our study allowed to build an operational quality dashboard for CRRAs as a ready-to-use support for an internal audit, for prioritisation of quality approach actions and for national and international benchmarking.
- emergency department
- prehospital care
- healthcare quality improvement
- patient safety
- telemedicine
Data availability statement
Data are available upon reasonable request. Data available include survey results and spreadsheets. Other documents available are survey items and newsletter sent. Data will be available immediately following publication, with no end date. Data available from LA (ID ORCID: http://orcid.org/0000-0002-5964-288X).
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Emergency medicine is, by its nature and uncertain environment, a risky activity.1 2 More specifically, medical regulation has the daily challenge of responding appropriately and effectively to emergency medical calls; when necessary, it must send the appropriate resource as fast as possible. This particular exercise is recognised as telemedicine and is based on validated procedures and recommendations.1 3–5
Urgent call management and emergency medical aid organisation are specific to each country. Some have adopted a single emergency number, such as the UK with 999,6 the USA with 9117 and Europe with 112.8 Depending on the type of emergency, an appropriate unit is sent and transmissions are made with the nearest emergency service when necessary. However, many countries still have three separate numbers for emergency, depending on whether the emergency comes under urgent medical aid, police or fire department; each of them has its own call centre and mobile units.9
This is the case in France, where emergency medical calls are centralised into emergency medical call centres, known as CRRAs (Centres de Réception et de Régulation des Appels).10 Operational 24/7 via the call number ‘15’, a medical regulation assistant (ARM) receives calls before the expertise of a regulator doctor (MR), who can be either an emergency physician or a general practitioner, depending on the type of emergency to manage.3 The CRRA is an integral part of the emergency medical assistance service called SAMU (Service d’Aide Médicale Urgente) in France, whose mission is to make decisions on how to best handle various medical situations. These can range from giving a simple telephone advice to sending mobile emergency and resuscitation unit, to quickly bringing sick persons or victims to appropriate hospitals while providing first aid.10
Medical regulation practices are associated with serious adverse events. These incidents are specific and particularly complex due to the telephone treatment of medical calls and the intervention of non-medical services (firefighters, medical transport, police).1
The recent French context, marked by the ‘tragedy of Strasbourg’ in December 2017,11 as well as several complaints for non-assistance to people in danger relayed in the French press, has highlighted this complexity as well as the lack of security in the work of agents handling calls. In addition, terrorist attacks underline the role of medical regulation in health response coordination.
The number of calls to SAMU keeps increasing (about 31 million calls for 67 million inhabitants in France in 2017; +20% between 2013 and 201712), which shows high expectations from the public1 13 and increases the pressure felt by the agents to offer effective and quality service and ensure the ‘right care’.13
Nevertheless, in France, no formalised and harmonised quality approach exists in CRRAs. Medical regulation does not appear in certification processes.1 Emergency medical societies have proposed activity and performance indicators, but the resulting lists are non-exhaustive, unstructured3–5 and their use heterogeneous between CRRAs.1
International literature on the effectiveness and precision of the responses provided is quite rich, especially concerning cardiorespiratory arrests,14–17 but very poor in terms of quality approach and evaluation. Studies exist on quality and indicators identification concerning prehospital urgent care18 19 or emergency services,20 but not concerning the previous step, which is the reception and regulation of urgent calls. Performance seems to take precedence over global quality.
As a result, it seems urgent to propose a structured list of quality indicators for medical regulation, concerning human and material resources, activity, performance and quality of calls, in the form of a dashboard. This tool is designed to follow the progress of the quality approach, assess it, identify areas for improvement and pilot actions.21 With this in mind, considering stakeholder opinion and experience in seeking consensus seems necessary in developing and prioritising a set of quality indicators. The objective of this work is to build an operational quality dashboard for CRRAs by searching for consensus on a regional scale.
Materials and methods
Indicator development phase
An observational step was conducted from June 2018 to September 2018 in a French emergency department at the Poitiers University Hospital by the public health team. Two public health residents (LA, JG) came to the CRRA as observers for 2 days to better understand the functions of the SAMU and to identify the risks related to the practice of regulation. Observations consisted of a tour of the premises, presentation of the operation of the SAMU and of the CRRA team, and monitoring of regulation activity through the practice of double listening calls with ARMs and MRs.
In parallel, a rapid international review of literature22 was carried out to identify all existing quality and activity indicators. A rapid review is a form of knowledge synthesis in which components of the systematic review process known as ‘evidence summaries’ are streamlined to produce information in a short period of time. Although numerous rapid reviews have been produced, their varied methodologies can diversely impact research results.23 A rapid review protocol was compiled based on the ‘framework of rapid review methods’.24
The databases queried were PubMed, ScienceDirect, Cochrane Library and Cairn. A first exploratory phase of these databases helped refine the keywords. The search equations are presented in table 1.
Databases and search equations for the rapid review of literature
The following were the inclusion criteria:
Studies published in English or French.
Studies published since 1 January 1999 (in France, the development of quality procedures follows the ordinance of 24 April 1996 establishing an accreditation procedure implemented since 199925).
All types of studies.
The following were the exclusion criteria:
Title and abstract published in language other than English and French.
Title and abstract off subject.
Irrelevant population or irrelevant outcomes in full text, that is, not meeting the objective (identify quality indicators).
Articles were selected first after reading the title and the abstract by two reviewers (LA, JG), then the full text by one reviewer (LA). In case of disagreement on a selection, an agreement was achieved after discussion between the two reviewers.
Other sources consisted of recommendations and repositories published by various French emergency medicine societies and associations, as well as by the French National Authority for Health (Haute Autorité de Santé, HAS), and the General Inspection of the Administration and General Inspection of Social Affairs report.26
At the end of this preliminary step, all the recommendations and indicators concerning the SAMU regulation were listed in a table using the framework with structure, process and outcome indicators.27 Some indicators were developed by the public health team following the observational phase and after discussion with CRRA stakeholders and the French Society of Emergency Medicine (Société Française de Médecine d’Urgence, SFMU). Some indicators from the recommendations were adapted (reformulated or clarified) for better understanding and greater consistency.
Indicator prioritisation phase
An interventional step was conducted from April 2019 to September 2019 in the largest French region (the Nouvelle-Aquitaine region with 6 million inhabitants, or 10% of the French population) by the public health team of Poitiers University Hospital.
The listed quality indicators were prioritised using the HAS validated consensus method ‘Recommendations by formalized consensus’ (December 2010, updated March 2015).28 This approach is an adapted method from the Delphi one.29 Briefly, the HAS consensus method consists of an individual rating in two rounds, separated by an intermediate meeting. The method involves three groups: one steering group, one rating group and one reading group. A questionnaire is distributed with a discrete numerical scale graduated from 1 to 9 next to each item: a value of 1 means that the respondent considers the proposal totally inappropriate (or not indicated or not acceptable); a value of 9 means that the respondent considers the proposal totally appropriate (or indicated or acceptable); values of 2–8 reflect possible intermediate situations; and a value of 5 corresponds to respondent indecision.
Patient and public involvement
In this study, the steering group was made up of the medical manager of SAMU 86 (HD-F), a doctor from SAMU 86 involved in the quality process (JB), the head of the public health department of the University Hospital of Poitiers and coordinator of care risk management (VM), and a public health resident from Poitiers University Hospital (LA). The rating group consisted of volunteer ARMs and MRs from the 13 SAMUs of the Nouvelle-Aquitaine region. The sample consisted of one MR and one referral ARM, with at least 15 years of experience, for each CRRA. The reading group was extended to all CRRA agents in the region.
First, the approach was presented to the 13 SAMUs of the region in order to obtain their consent and their commitment to participate in the entire prioritisation process.
The questionnaire was drawn up from the exhaustive table of indicators, which was built from the development phase (observational step and rapid review of literature). The design was neat in order to facilitate participant acceptance and increase response rate. To preserve anonymity, the questionnaire was sent to the medical manager of each SAMU, who then forwarded it to the rating MRs and ARMs.
The rating was done individually via an online survey site, SurveyMonkey, and participants were given 2 weeks to answer the survey. To be sent, the questionnaire had to be completed in full, but the filling could be done in several times. Responses were anonymous.
The rating criterion used was the priority given to the indicator. Each item had to be rated on a scale of 1 to 9: 1 if the item was considered non-priority and 9 if it was considered absolute priority. A final item allowed participants to comment or suggest a new indicator. The results obtained were analysed according to the HAS method. The proposal was deemed priority when the median value was ≥7 and all ratings ≥5; non-priority when the median value was ≤3.5 and all ratings ≤5; and uncertain when the median value was between 4 and 6.5 (indecision) or when there was no consensus between the rating group members (all other situations). Proposals with strong (ratings in the range 7–9) or relative (ratings in the range 5–9) agreement for prioritisation were accepted as such, without discussion in the meeting or submission to the second round of rating.
At the end of the first round, a results restitution was planned, as well as a discussion with the rating group about items that did not reach consensus. These were submitted for a second round of rating, but only to members who had attended the discussion, according to the HAS validated method.28 The response and analysis methods remained unchanged.
At the end, all results as well as a process summary were returned to all SAMUs in the region via a newsletter sent by email. A structured descriptive sheet30 was drawn up for each prioritised indicator, which contains the title, objective, complete definition of its numerator and denominator if applicable, data collection modalities, details on the interpretation of the results and characteristics. Finally an operational dashboard was established based on the indicators deemed to be priorities to be able to give visibility of quality within the CRRAs through the results of each indicator: "yes", "no", "partially". This dashboard is in the form of an Excel document containing one tab per theme to be covered, then one line per indicator and its result opposite, with the possibility to directly access the descriptive sheet for each item. Every year it will be completed to follow the evolution of the quality approach within the CRRAs. This entire step was carried out by the steering group.
Results
Indicator development phase
The rapid review of literature included 33 studies in our work, from the initial 414 that resulted from various searches (figure 1).
Flow chart of the rapid review of literature for the development and prioritisation of quality indicators in emergency medical call centres.
The preliminary phase of observation, discussion with stakeholders and literature review identified 360 quality indicators concerning SAMU regulation.
All were classified in a table structured into five themes according to the field concerned: material resources, human resources, quality approach, call handling and postcall support (table 2). These five subjects were chosen according to the themes found in all the indicators listed, as well as by relying on the repositories of the association ‘SAMU-emergencies of France’3 5; we can thus find what allows a CRRA to function (human and material resources), the indicators which optimise this operation and the safety of the CRRA (quality approach), as well as the indicators directly linked to the activity of the CRRA (performance about calls and support for victims).
Distribution of quality indicators for emergency medical call centres, listed in the initial table built from observation and rapid review of literature
Indicator prioritisation phase
After presenting the approach and the prioritisation process by sending an electronic document to the Nouvelle-Aquitaine SAMUs, 13 MRs and 13 ARMs agreed to participate (table 3).
Characteristics of respondents for prioritisation process in the 13 CRRAs of the Nouvelle-Aquitaine region
The first round of rating took place between 30 April and 23 May 2019, lasting approximately 3 weeks. The response rate was 77%; only one SAMU was not represented among the respondents (table 3).
At the end of the first round, 18.9% of the items were deemed priority with strong agreement, 33.6% priority with relative agreement, 15.6% uncertain due to indecision and 31.9% uncertain due to lack of consensus (table 4; detailed data not shown).
Results of analysis according to the HAS method used for prioritisation of quality indicators in emergency medical call centres
Before the second round, some items were reformulated by the steering group due to ambiguous or unclear formulation, according to the HAS method.28 Moreover three new items were integrated following the proposals of the participants.
The 171 items which did not obtain consensus in the first round as well as the 3 new items were submitted for a new rating to the 20 respondents from the first round.
The second round of rating took place between 25 June and 23 September 2019, for around 13 weeks. The response rate was 75%; nine SAMUs were represented (table 3). At the end of the second round, 1.1% of the items were deemed priority with strong agreement, 20.1% priority with relative agreement, 31.6% uncertain due to indecision and 47.1% uncertain due to lack of consensus (table 4; detailed data not shown).
Finally, out of the 26 participants included in the consensus approach, 15 participated in the whole process, representing a response rate of 58%. All in all, 70 items (19.3%) were considered priority with strong agreement, 156 (42.9%) priority with relative agreement, 55 (15.2%) uncertain due to indecision and 82 (22.6%) uncertain due to absent consensus (table 4).
At the end of the prioritisation process, a newsletter was returned to the 13 SAMUs presenting the results and prospects. A dashboard of indicators deemed high priority has been established (online supplemental table 1). Seventy descriptive sheets were written. The reading group contributed by simplifying some of them (structural indicators), reformulating some objectives and redrawing some items to improve the feasibility of data collection.
Supplemental material
Discussion
Main results
To our knowledge, this is the first study on the development and prioritisation of quality indicators for emergency medical call centres. There were 360 indicators initially listed, thereby constituting the first structured and exhaustive list in France for prehospital emergency regulation.
The large number of indicators shows the complexity of medical regulation activity, its many risks at all levels (material and human resources, call management and quality approach) and the need to prevent them in order to improve quality of care. Such a large quantity of indicators cannot be realistically accepted, implemented and evaluated, hence the need to prioritise them for progressive implementation in CRRAs. Finally, 70 indicators were deemed to be high priority with strong agreement. This significant reduction in the number of indicators allowed creation of an operational dashboard and the writing of an indicator descriptive sheet for each.
The 70 prioritised indicators concern conventions, procedures and protocols, methods for effector triggering, exceptional health situations, equipment, call recording, training, the roles and number of concerned professionals, securing their work, relations with firefighters, regulation file keeping and call transfers. These emerging themes are coherent with the current context and stakeholder concerns (ARMs and MRs) and illustrate the need to secure regulation activity. It is interesting to note that no quantified performance indicator was found among the 70 highest priority indicators.
Place in the literature
In contrast, many works have been done about performance, in particular with regard to cardiopulmonary arrests,14–17 with the aim of showing how to improve the efficiency and accuracy of the response provided by emergency call centres.31 Agent occupation rate seemed to be the most important factor contributing to service quality as well as all quantitative indicators.
However, quality does not stop with performance. On the contrary, quality and safety of the structure, resources and processes can improve performance, as explained by Giroud in his publication.32 However, few indicators exist in the literature to assess these dimensions. In our study, the diversity of listed indicators and the choice of prioritised indicators confirm this need to consider regulation activity in its entirety, as well as human factors, in view of achieving more secure activity. Moreover, the notion of human factors has appeared in recent literature: for example, Rawshani et al33 concluded that there was a higher risk of not prioritising life-threatening patients during lunchtime. Work by Hamelin and Arzalier34 goes in the same direction; analysis of complaint files showed that considering human factors and fatigue is essential and underlines the importance of reporting on internal dysfunctions. The impact of feedback on performance was explored by Clawson et al,35 which showed that regular and objective information to emergency medical dispatchers about their performance significantly improved their rigour. Professional practice evaluation, feedback and adverse event analysis must be encouraged in order to achieve a positive culture of error.36 37 The role of human factors was also studied in telephone exchanges by Riou et al,38 who analysed the words used to initiate cardiopulmonary resuscitation (CPR) assisted by the dispatcher and their association with caller agreement. The quality of telephone communication should be formalised and evaluated.39 40
Securing the work of agents seems to be a major concern at present, as confirmed in our study by the strong consensus obtained for this proposal. In this complete item, well-being at work is taken into consideration. Indeed, a link has been demonstrated between job satisfaction and performance41; here again, human factors are important.
Thus our study seems to fill a gap in the exhaustiveness of the risk factors associated with CRRA activity, taking into consideration the human factor.
Strengths and weaknesses
This work was carried out in connection with the French Society of Emergency Medicine and was based on a validated method considering peer opinion, thereby strengthening its validity. The search for consensus has confirmed the need to formalise a quality approach as well as the relevance of the indicators. We initially obtained strong adhesion following presentation of the approach, and second the results gave no non-priority indicators and an average of >5 out of 9 (priority) for 96.7% of them. All in all 62.3% of the indicators obtained a strong or relative prioritisation consensus.
The first round of rating required two general reminders by email and a 1-week extension to obtain the maximum number of responses. The participation rate was suitable (77%). For a better questionnaire acceptability and an optimal response rate, the choice was made to adapt the results analysis of the first round, by accepting as such the proposals that have obtained a strong and relative agreement. By resubmitting to the rating only uncertain proposals (for which there was indecision or absence of consensus), the second round would be less heavy.
Furthermore, the possibility for the participants to express themselves freely at the end of the questionnaire was important for proposal integration or modification before the second round.
The second phase took place during the summer period, explaining the two general reminders and the two individual reminders, as well as the time limit for closing the 13-week survey. It nevertheless obtained a participation rate of 75%. Out of the 174 items resubmitted for rating, 37 finally obtained agreement. We can think that when each participant knows the median obtained by the group for each item, it becomes possible for some participants to review their judgement and reposition themselves accordingly.
Finally, only 58% of the participants went through the prioritisation process. However, Nouvelle-Aquitaine is the largest region in France, with 12 departments and 13 SAMUs, which allowed for 15 representative peers to participate in the whole process: doctors and ARMs, who are the main participants in regulation. The length of questionnaire is probably largely responsible, but it was necessary to be as exhaustive as possible from the start in order to be relevant in reduction by prioritisation. Because of this length constraint, it was possible to respond to the questionnaire in several phases and an effort was made on its presentation. Otherwise, the duration of the process (more than 4 months in total) as well as the vacation period during which it took place certainly favoured the number of ‘lost to follow-up’.
It was not possible to hold a meeting with the rating group between the two rounds, which can constitute a potential bias. The aim of the meeting was to return the results of the first round and to discuss the proposals deemed uncertain. Given constraints in each participant’s schedule, it seemed preferable to return the results by email so that everyone could benefit from the same feedback. To do this, each participant of the first round received an overall summary of the results of the analysis as well as a file with their own answers. The median obtained by the group was shown below each item on the second questionnaire so that each participant could situate himself in relation to the rest of the group during the second round, according to the HAS method. In the absence of discussion, comments and proposals brought up during the first round were considered and partly integrated into the questionnaire, after consultation with the steering group.
Implications
This work will facilitate the gradual integration of a quality approach within CRRAs. From now on, each SAMU in the region will be able to assess the quality and safety of its regulation activity. Using the dashboard, or a computerised mapping tool, will enable organisation of a periodic internal audit. The detailed descriptive sheets constitute an operational tool to also benchmark with other CRRAs in order to help each other out and improve everyone’s practice. Ultimately, once the installation is successful, the indicators that were prioritised with relative agreement could gradually be integrated until exhaustive implementation of the 363 indicators is achieved in a few years.
Moreover, this dashboard could be integrated into the quality account of regional establishments as well as in certification procedures. These indicators could also be integrated into the indicators for improving quality and safety of care (IPAQSS).42
Finally, this work could facilitate CRRA inclusion in the IFAQ procedure (Financial Incentive to Improve Quality).43
Areas for improvement
The search for consensus was carried out on a regional scale, but this approach could be extrapolated to national and international scale to increase the power of the study, as well as to standardise practices in terms of quality and safety and to do benchmarking. Moreover, we could integrate other actors of medical emergency, such as paramedics, nurses and general practitioners.
Initiatives are already emerging to strengthen medical regulation quality within CRRAs. Training with diploma was set up in 2019 and is henceforth compulsory to exercise the profession of medical regulation assistant.44 Telemedicine is also growing, with CPR assisted by phone or smartphone, remote video consultations to assess severity of patient illness or remove doubts, etc; these are valuable aids for regulation.
Furthermore, the Information System-SAMU (SI-SAMU) programme is being deployed in French territories and aims to make the information system and telecommunications means used by emergency medical services more reliable and secure. This centralised and national portal is intended to be used by all SAMUs.45
Conclusion
In conclusion, our study allowed identification of all quality indicators for SAMU regulation and prioritising them by means of regional stakeholder consensus in order to propose an operational dashboard for progressive implementation in CRRAs. In the future, we could extend the prioritisation process on a larger scale and include other CRRA agents to strengthen the power of the study and standardise practices.
Data availability statement
Data are available upon reasonable request. Data available include survey results and spreadsheets. Other documents available are survey items and newsletter sent. Data will be available immediately following publication, with no end date. Data available from LA (ID ORCID: http://orcid.org/0000-0002-5964-288X).
Ethics statements
Ethics approval
An ethical approval is not necessary for this study according to the French legislative framework since the study does not handle personal data.
Acknowledgments
The authors of this paper acknowledge the contributions of the ARMs and MRs from all the emergency medical call centres (SAMUs) of the Nouvelle-Aquitaine region.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors LA conceived and designed the study, designed the data collection tools, proceeded with data collection, conducted the research, analysed the data, and drafted and revised the paper. JB designed the data collection tool, and drafted and revised the paper. JG proceeded with data collection, and drafted and revised the paper. OM revised the paper. HD-F monitored the data collection, revised the paper and supervised the research. VM designed the study, revised the paper and supervised the research.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.