Article Text

Quality and performance indicators in Portuguese anatomical pathology laboratories: a panel validation by qualitative Delphi technique
  1. Ana Paulino1,2,
  2. Ana Rita Pedro2,3,
  3. Ruben Roque4,
  4. Sónia Dias2,3
  1. 1Anatomical Pathology Department, Centro Hospitalar de Lisboa Ocidental EPE, Lisboa, Portugal
  2. 2NOVA National School of Public Health, Public Health Research Centre, Universidade Nova de Lisboa, Lisboa, Portugal
  3. 3Comprehensive Health Research Centre, Universidade NOVA de Lisboa, Lisboa, Portugal
  4. 4Anatomical Pathology Department, Portuguese Institute of Oncology, Lisbon, Portugal
  1. Correspondence to Dr Ana Paulino; anaisabelpaulino{at}gmail.com

Abstract

Background In laboratory medicine, quality and performance indicators (QPIs) are essential tools to ensure the quality of healthcare services and patient safety. QPIs allow comparison of outcomes, favouring accountability and transparency. Internationally, there are some QPI evaluation models, but the fact that they are paid limits their dissemination in smaller/poorer laboratories. In Portugal, each laboratory defines its own QPIs, with no uniformity between institutions. The development of a free QPI panel suitable for anatomical pathology laboratories (APLs) would allow for quality assessment and improvement.

Objective To develop a consensual and validated QPI panel suitable for Portuguese APLs.

Methods The study was developed in two stages. First, a bibliographic review was carried out, selecting the adequate QPIs. Afterwards, these QPIs were evaluated by experts through the Delphi method, where they could also suggest other pertinent QPIs.

Results By the end of the Delphi method, there was a consensus on 64 QPIs (31 for ‘structure’, 30 for ‘process’ and 3 for ‘result’). The consensual QPIs covered all phases of the total test cycle. The lack of specific anatomical pathology QPIs in the bibliography was noticeable. There was greater consensus on ‘process’ and ‘result’ QPIs than on ‘structure’. This was supported by the bibliography, where the first ones were more valued. Nevertheless, it is important to monitor all the main laboratory processes, prioritising the evaluation of QPIs with greater impact on healthcare quality and patient safety. These results should allow APLs to identify the causes behind poor performance and improve their services.

Conclusions This panel is a valuable tool for APLs, contributing to quality awareness. It can be the first step towards the development of a free benchmarking quality programme in Portugal, encouraging competitiveness and cost-efficiency.

  • Pathology
  • Quality improvement
  • Performance measures
  • Quality improvement methodologies
  • Laboratory medicine

Data availability statement

No data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What is already known on this topic

  • Quality and performance indicators (QPIs) are essential tools to ensure the quality of healthcare services and patient safety.

  • There are some paid QPI programmes, but besides their widespread use they are limited by their costs and only a few are focused on anatomical pathology laboratories.

What this study adds

  • Through a bibliographic review and a specialists Delphi panel, it was possible to define a set of QPIs validated for Portuguese anatomical pathology laboratories.

How this study might affect research, practice or policy

  • To strengthen, disseminate and operationalise the project, it is now essential to articulate with professional associations and specialty colleges.

  • This is the first step towards the development of a free benchmarking quality programme in Portugal, encouraging competitiveness and cost-efficiency.

Introduction

Medical laboratories are responsible for 70% of clinical decisions.1 Therefore, laboratory errors can have a significant impact on patients’ outcome and increase direct and indirect costs.2 In Portugal, concerns over the quality of anatomical pathology laboratories (APLs) led to an increase in certified/accredited institutions over the years.

The referential ‘ISO 15189:2012’ defines quality and performance indicators (QPIs) as ‘how well an organization meets the needs and requirements of users and the quality of all operational processes’. QPIs are essential tools to guarantee the quality of healthcare services and patient safety.2 They favour transparency in laboratory services by boosting improvement strategies, monitoring, benchmarking and accountability.2 3 When developing QPIs, the practical context they apply to needs to be considered in order to clearly define goals, acceptable values, and data collection and analysis methodologies.4 Quality evaluation through indicators demands a systematic approach to ensure reproductivity and validity.

To increase efficiency, a QPI panel should report to all phases of the total test cycle: preanalytical, analytical and postanalytical. The vast number of procedures and variables in laboratory activities can hinder this goal and therefore laboratories should focus on tasks with greater impact on patients.5 6 In the last decades, due to the implementation of effective quality strategies (ie, external quality programmes, guidelines and recommendations), the analytical phase registered a steep decline in the error rate.2 7 This is the most regulated and standardised phase in medical laboratories, unlike the preanalytical and postanalytical phases which are considered more vulnerable and associated with greater risks to patients.8 The preanalytical phase is characterised by a complex responsibility network and several interfaces between different services and professionals. According to Roque et al,9 most errors at this stage are clinical errors and include obtaining the sample from the wrong patient, inadequate surgical procedures or incorrect sample identification. At the preanalytical phase, the incomplete description of the patient’s clinical history was shown to affect the accuracy of the reports.10 Errors at this stage can seriously affect the viability of the sample and consequently the final diagnosis.11 In the postanalytical phase, the turnaround times, critical notification and result interpretation errors were identified as weaknesses by the patients.12 The implementation of laboratory information systems (LIS) and control checkpoints between vulnerable activities is the most used strategy to mitigate preanalytical and postanalytical errors.7

Internationally, there are QPI-based quality programmes, like the ones developed by the Royal College of Pathologists (RCP) or the College of American Pathologists (CAP). However, there is no consensus on a common terminology: each model has different criteria, objectives, data collection, processing and result analysis methodologies.4 13 These programmes are also paid, a fact that tends to decrease their dissemination.

In clinical laboratories, entities from countries such as Australia, Brazil or China developed national QPI panels mostly focused on the postanalytical phase. In 2008, the International Federation of Clinical Chemistry and Laboratory Medicine created the ‘Laboratory error and patient safety’ group, aiming to develop a QPI panel suitable for clinical laboratories and promote error reduction. Over the last decade, this group identified a reliable QPI set for the total test cycle, harmonising criteria and procedures, drawing on an international panel of experts.

In Portugal, each APL defines its own QPI panel, which does not favour uniformity or a high level of universal quality.14 In 2016, the Portuguese ‘Anatomical Pathology Referral Network Report’ defined some indicators concerning human resources, production, care level, training and research. These were to be periodically monitored to identify imbalances or inequality in the network, but so far no evaluation has been released.

Within this framework, this study aims to develop a quality and performance evaluation model suitable for Portuguese APLs through identification and consensus validation of a QPI panel, representing all the main laboratory phases (preanalytical, analytical and postanalytical).

Methods

The study had two main stages:

  • First stage: QPI identification through a comprehensive and systematic bibliographic review.

  • Second stage: QPI submission to an expert panel and consensus assessment through a four-round Delphi method; in the first round, experts could also suggest indicators to submit to their peers in subsequent rounds.

First stage: QPI identification

Between January and June 2020, the PubMed and Scopus platforms and the CAP, the RCP and the Royal College of Pathologists of Australasia official websites were used to identify laboratory QPIs, using the expressions ‘anatomical pathology’, ‘benchmarking’, ‘performance’, ‘indicator’, ‘key performance indicators’, ‘KPI’, ‘laboratory’, ‘pathology’ and ‘quality’. After an abstract analysis, a total of 20 papers and 2 official QPI documents describing QPIs and pertinent information for their characterisation were selected. The papers’ complete reading led to an initial list of 313 items. The search ended when theoretical saturation was achieved. From this first list, all QPIs referring to laboratory fields other than anatomical pathology (101 items) and repeated/overlapped QPIs (188 items) were eliminated, reducing the list to 24 items. These were classified according to Donabedian’s trilogy: 6 topics referring to ‘structure’, 17 regarding ‘process’ and 1 related to ‘result’.

Regarding the ‘structure’ items, the studies only mentioned categories for their evaluation, not suggesting indicators per se, so these needed to be specified. Based on the previously read papers, 25 qualitatively assessable ‘structure’ indicators were added to the QPI list. As the experts would have the opportunity to comment on the QPIs, these indicators would serve as a starting point for the first analysis and critique. The final QPI panel was reviewed and validated by three independent anatomical pathology quality specialists, who did not present any problems or objections. The data collection online platform, the questionnaire structure and the vocabulary adequacy were also tested.

Second stage: QPI validation

Consensus methods aim to achieve a generalised agreement on a controversial issue, as experts suggest solutions to a proposed problem, according to their experience, in a structured environment.15 The Delphi method is a qualitative technique used to systematically obtain critical inputs from a group of experts, collecting and clarifying their experiences and sharing the results through a series of questionnaires interspersed with feedback. The technique is based on four fundamental features: anonymity of the participants, interaction, controlled feedback and statistical aggregation of group response.16 17

All methodological criteria were defined (table 1) prior to the implementation of the technique, as a rigorous design is essential for reliable and reproducible results. Following the Delphi method’s aim to identify a consensus, each QPI was considered consensual if the percentage of concordance in any classification of a five-range Likert scale was equal or over 80%. When consensus was achieved, the QPI was withdrawn from the next rounds. The specialist could also comment on every QPI, and during the first round QPI suggestions were also welcomed.

Table 1

Delphi criteria

There were four rounds of questionnaires, based on other healthcare Delphi models and according to the available time. The specialists were invited to participate if they met all the following criteria: (1) a pathologist, healthcare professional or anatomical pathology technician; (2) work in an APL for at least 5 years; (3) work in a certified and/or accredited institution for at least 2 years (APL or hospital); and (4) have QPI assessment experience.

As the Delphi technique highly depends on the participation of external experts, one of the study’s greatest concerns was the insufficient number of recruited experts or their dropout during successive rounds. To mitigate this situation, some strategies were implemented:

  • Use of an online platform, accessible anywhere and anytime.

  • Selection of a completely anonymous and confidential methodology.

  • In rounds 2, 3 and 4 the previous results were presented at the same time as the QPI to be reclassified, motivating an immediate answer.

  • Each expert had an individual password to access the questionnaires, allowing the identification of lacking answers and their consequent reminder.

Patient and public involvement

Patients were not involved in the study.

Results

In the first stage of the study, the bibliographic review led to the identification of 43 QPIs, included in the first questionnaire submitted to the experts:

  • 25 ‘structure’ QPIs (categorised by human resources, workload, LIS, facilities, work accidents, external quality evaluation programmes).

  • 17 ‘process’ QPIs.

  • 1 ‘result’ QPI.

The second stage concerned the Delphi technique application and monitoring.

Participation and dropout rates

A 70% participation rate is required to guarantee methodological accuracy,18 a rate that was achieved in every round (table 2). The bibliography also suggests a maximum dropout rate of 20% between rounds. This value was slightly higher (21.9%) during the first round; however, the remaining rounds had dropout rates below 16%. The participants’ dropout is usually a direct consequence of long questionnaires or lack of interest. Consequently, it is vital to start the technique with a significant number of experts, ensuring that the number of participants in each round stays within the recommended values (at least 8–12 participants).19

Table 2

Participation per round

Experts’ and institutions’ characterisation

Most of the specialists were anatomical pathology technicians (79%–88% in all rounds) and the remaining were pathologists (12%–21% in all rounds). Most of them have worked in the field for 5–19 years. From all the participants, only 9%–13% did not have any health quality training. Those who had training largely chose in-hospital sessions (less timely and costly). One participant was trained by a certifying entity, two had health quality postgraduation diplomas and two were certified healthcare auditors.

In the first two rounds, professionals from 13 different institutions participated in the study: 38% from private APLs, 16% from public–private partnerships and 46% from public laboratories. In the last rounds, with the withdrawal of some participants, the representativeness of private laboratories and public–private partnerships decreased to 30% and 10%, respectively. As for public institutions, they represented 60% of the participants at this point.

All the institutions with APLs had some type of certification/accreditation, "ISO 9001" being the most popular. The Joint Commission International, the Caspe Healthcare Knowledge Systems, the Andalusian Agency for Healthcare Quality accreditation models and the "ISO 14001/ISO 45001" certification models were also identified. In addition, some specific accreditations for cancer diagnosis monitoring were also present, namely those provided by the European Society of Breast Cancer Specialists or the Organisation of European Cancer Institute.

Regarding the APLs, 92% had some type of certification/accreditation. Overall, the models overlapped with those held by the institutions, except for the "ISO/IEC 17025" accreditation and specific accreditations granted by recognised international entities (not specified due to privacy issues regarding the laboratories).

Delphi technique

In the first round, the experts were presented with 43 QPIs. Of these, they reached a consensus on 5 (11.6%), further suggesting 44 new indicators to be evaluated in the following rounds. The second round had the highest number of QPIs, and consensus was reached on 10 of 82 (12.2% of the total of this round). During the third round, there was a general agreement regarding 25 of 72 indicators (34.7% of the total of the round). Finally, the last round had the highest number of consensual QPIs: of the 47 QPIs, there was a consensus on 24 (51% of the total). The feedback presented throughout the rounds brought the opinions closer together, allowing consensus achievement. About 97% of the consensual QPIs (62 QPIs) were rated ‘5 - totally relevant in a QPI laboratory comparative model’, 1.5% (1 QPI) was rated ‘4 - Very relevant in a QPI laboratory comparative model’ and 1.5% (1 QPI) was rated ‘3 - Relevant in a QPI laboratory comparative model’.

‘Structure’ QPIs

The ‘human resources’ category performed the worst, as there was no general agreement regarding any of the QPIs. The experts did not agree with the direct comparison of the total number of pathologists, technicians or assistants, noting that the ratio between different groups of laboratory professionals should not be compared, since it depends directly on the needs of each laboratory. There was also no consensus regarding the average age of laboratory professionals or the number of professionals per laboratory module. The experts argued that the current binomial qualitative individual evaluation model, used in Portuguese public services, does not allow for comparison of personnel performance. In the ‘workload’ category, specialists reached a consensus on 8 of 10 QPIs. According to the experts, these criteria can be evaluated by the number of paraffin blocks, complex specimens received, diagnostic points and number of laboratory professionals. They also highlighted the importance of using the diagnostic point system (which evaluates the pathologist’s workload according to complexity and typology) instead of the billing codification system, as it better reflects the complexity and work involved in the processing of each sample (time and cost). The evaluation of the amount of grossing and first screening performed by specialised technicians was also consensual, as well as the amount of outsourced work. In the ‘technologies and information’ category, there was a consensus on eight of nine QPIs. The availability of electronic requests and specific LIS, the use of bar codes/quick response (QR) codes for traceability, and the number of non-conformities related to LIS problems were valued by the experts. They clearly chose QPIs oriented to monitoring, evaluating and reducing errors. Other consensual QPIs included the availability of image and sound systems in grossing rooms, which simplify and facilitate diagnostic interpretation and quality control. The digital pathology QPI was one of the most discussed, particularly its potential impact regarding algorithm development and ‘digital diagnosis’ enabling. However, the experts acknowledge that it requires a large investment and deep workflow changes.

Of the 10 QPIs presented for the ‘facilities’ category, a general agreement was reached on 7, namely those referring to the facility area per employee, the evaluation of noise level and air extraction, luminosity, and ergonomic conditions. In addition, the experts emphasised the importance of defining and evaluating the laboratory workflow, which should adapt to each laboratory and become as efficient as possible.

In the ‘work accidents’ group, three QPIs obtained consensus. The analysis of the number of work accidents with or without medical assistance was found to be essential, as was the analysis of the accidents and their cause–effect parameters in order to implement preventive/corrective measures and mitigate their occurrence.

In the ‘external quality assessment programmes’ category, there was a general agreement on two of three QPIs presented. The analysis of the results of the external quality evaluation programmes as well as the comparison of deviations from the general average for the same models were considered relevant. Experts even suggested the development of a programme for technical and medical quality evaluation in Portuguese APLs.

Some of the QPIs suggested by the specialists in the first round did not fit into the ‘structure’ categories so they were grouped in the ‘others’ category. Of the five QPIs presented, there was a consensus on three: two related to equipment maintenance and one regarding stock management.

‘Process’ indicators

Of the 32 QPIs presented, a consensus was achieved on only 2. The experts agreed with the QPI regarding specimen reception non-conformities, acknowledging that sample identification errors were frequent and highlighting the need to clearly inform non-laboratory professionals and clinicians about laboratory procedures to reduce errors in the preanalytical phase. They also emphasised the importance of patient/sample identification procedures and error-reducing oriented checkpoints.

The discussion further led to the conclusion that monitoring turnaround times in all types of samples, in addition to those considered urgent, is vital, as is the evaluation of false negatives in second cytological screening, "atypical squamous cell-squamous intraepithelial lesion" (ASC-SIL) ratio in cervical-vaginal cytology, the comparison of intraoperative versus definite diagnoses, and the comparison of primary diagnosis versus second opinion. In addition, it is essential to ensure adequate corrective and preventive measures when diagnostic discrepancies are identified. There was a general agreement among the specialists regarding the evaluation of some technical laboratory quality parameters (like sampling quality, slides repetitions, re-embedding, immunocytochemistry and histochemistry quality evaluation, or fine needle aspiration inconclusive diagnosis), the importance of the pathologist’s presence in multidisciplinary therapeutic decision consultations and the number of cases that are discussed in these meetings. The retrospective review of closed cases was considered relevant for the identification of diagnostic non-conformities.

‘Outcome’ indicators

The QPIs regarding ‘result’ were met with general agreement. Two of the indicators referred to customer and employee satisfaction, while the third focused on the importance of complaint analysis. The final QPIs are found in online supplemental table.

Supplemental material

Discussion

Over the past few years, some work groups have been developing and validating a QPI panel for clinical laboratories, raising awareness on the importance of monitoring and analysing QPIs and laboratory errors as tools of continuous quality improvement.2 7 8 However, robust QPI and error analysis models were not found in APLs and the QPI documents published by international colleges are not specific to anatomical pathology. As Ferreira20 defends, clinical pathology works mostly with analytical results, making it easier to develop and integrate quantitative control tests.20 This reality contrasts with an anatomical pathology diagnosis that requires clinical correlation, interpretation and differential diagnosis, which are subjective components.

The experts evaluated a total of 87 QPIs, reaching a consensus on 64 of them.

Tangible ‘structure’ QPIs were hard to identify in the literature, perhaps due to laboratory variability. Since laboratories manage their workloads according to their characteristics, care provider typologies and specialties,21 each expert will naturally value the ‘structure’ QPIs differently according to their reality. In addition, these QPIs are usually less controlled by laboratory professionals and more dependent on management decisions and investments, so it did not come as a surprise that this category presented the worst consensus rate.

In the ‘process’ QPIs, the consensual indicators represent different modules within the APL: histology, cytology and immunocytochemistry, among others. In the first round, the indicators regarding where there was a quickest consensus were those more frequently identified in the literature. Most of these QPIs have already been extensively studied and their impact on patient outcome and diagnosis feasibility is known.4 9 21 22 The experts also highlighted the importance of indicators related to diagnostic accuracy and sensitivity, as well as interpretive variability.

In the ‘result’ category, specialists reached consensus on all three items, and the following comments focused on the importance of an organisational culture based on quality. Laboratories and institutions need to sensitise their employees to the importance of satisfaction surveys, also encouraging substantiated complaints. Communication between services is crucial to the understanding between laboratory and clinicians/clients, further leading to safer and more efficient care delivery.

The ‘process’ and ‘result’ QPIs are significant tools in the quality and performance evaluation of a laboratory and are highly valued in the literature. However, in the result analysis, structural factors may be fundamental to the interpretation and justification of the obtained values.

Shahangian and Snyder5 concluded that due to the complexity of the procedures involved in the total test cycle it can be difficult to implement QPIs suitable for all laboratory phases.6 Although the final QPI panel has items in all test cycle phases, the discrepancies regarding its distribution are notorious. Most of the QPIs end up being considered transversal to all laboratory phases, partly due to the weight of the ‘structure’ indicators in the total volume of QPIs. The analytical phase has the highest number of QPIs. These refer only to tasks performed inside the laboratory, directly evaluating the work of APL professionals. In the last decade, with the implementation of guidelines and quality certification/accreditation programmes, there has been a more careful and efficient management of all analytical processes, with a notorious decrease in error rates.2 7 However, the preanalytical and postanalytical phases are the most sensitive regarding patient safety, registering a higher percentage of errors.2

International studies on laboratory medicine (mostly clinical pathology laboratories) advocate that errors affecting the preanalytical phase may correspond to 53%–70% of the total errors recorded in the laboratory.13 23 They also document error rates between 0.25% and 24% of the total volume of specimens received.13 23 In Portugal, Roque et al identified an error rate of 3.1% during the specimen’s reception in APLs.9 They concluded that the lack of information about the sample’s type, the absence of request order and the lack of clinical information were the main reasons for non-compliance. The authors pointed out the importance of effectively informing clinical services about the correct preanalytical procedures and recommended a checklist implementation, defending that the professional’s accountability is a key factor in the improvement of results. The ‘construction of a reporting system and shared databases’ could also represent a good strategy for performance comparisons between hospitals and could help achieve better results.9

Regarding the postanalytical phase, clear communication between the laboratory and the clinicians, not only verbal but also written, is essential. Previous studies concluded that in 30% of the reports issued there were misconceptions regarding interpretation.24 Ferreira20 conducted a review of histological slides concerning errors in the analytical and postanalytical phases, concluding that there is a 12.9%–15.1% error rate, occasionally with serious consequences for patients.20 As for indicators relating to turnaround times, they should be carefully monitored. Although short turnaround times may be important to initiate therapy sooner, Ferreira20 notes that shorter responses may be associated with higher error rates in surgical specimens. It is essential to balance the diagnostic quality and the turnaround time to ensure reliable outcomes. Considering the impact of laboratory errors on patients, monitoring the QPIs representative of the main processes within the laboratory is crucial to ensure patient safety.

Before implementation, a generalised discussion about each QPI is fundamental, enabling its integration into the reality of most laboratories. Although the final product is identical, each laboratory has its working methodology and QPIs need to fit the largest number of scenarios. This discussion aims to increase the credibility and reliability of QPIs by standardising terminology, numerators and denominators, inclusion and exclusion criteria and data collection, and processing methods.13 In addition, it is necessary to ensure LIS can provide the necessary data.

The final QPI list is quite extensive and it is important to prioritise the QPIs to implement, choosing primarily those which have a greater impact on the safety and quality of the services provided.1 This task will facilitate the integration of QPIs into laboratories’ daily routines and gradually motivate professionals for their analysis.

The results should lead to a critical reflection by each laboratory, enabling the identification of the causes behind poor performance and the implementation of a quality improvement strategy. This type of methodologies can even lead to discussions about the best working practices, stimulating methodological changes based on scientific evidence.25 This process must adapt to new procedures and technologies, based on continuous quality improvement and results.

Limitations

In the last rounds, consensus increased considerably. This may be one of the technique’s perverse effects: experts may feel pressured to reach a consensus, not truly revealing their opinion.19 However, given that this technique is time-consuming, if there was not a genuine interest of the experts, they would not participate. Considering the response rate was within the recommended values, it is assumed that the specialists’ answers resulted from their real interest in the subject and that the possible bias generated was reduced. Nevertheless, throughout the implementation of the Delphi method, the experts’ comments generated a constructive discussion, which contributed to the achievement of consensus on many QPIs.

The number of participants was always within the recommended values, although a greater number of specialists would benefit the results.19 It would also have been interesting to have more pathologists among the participants.

The QPI classification, according to Donabedian, and its allocation in each phase of the total test cycle were challenging. The bibliography presented different interpretations, so the most common categorisation was used.

To guarantee the anonymity of experts and institutions, some certification/accreditation models were not mentioned.

This study is just the first step to develop a QPI set for APLs. To develop these theoretical conclusions, it is essential to articulate with professional associations and specialty colleges in order to strengthen, disseminate and operationalise the project. These are the next goals of the authors.

Conclusion

We were able to identify a set of QPIs validated by a panel of experts and applicable to public or private APLs. This was the first step of a project that aims to develop a laboratory benchmarking model which will contribute to quality awareness and improvement, compelling institutions to be more competitive and cost-effective. There is still a long road ahead, but only a generalised commitment to patient safety and to the pursuit of better outcomes can be effective in continuous quality improvement.

Data availability statement

No data are available.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and was approved by the Ethics Commission of Ocidental Lisbon Hospitalar Center, EPE (Lisbon, Portugal) (Clinical Studies National Register Number (Portugal): 20170700050). Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The authors would like to thank all experts involved for their contribution.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors AP, the main author, developed the idea, designed the project, carried out all the methodological aspects of the study, analysed and interpreted the data, and wrote the draft. AP was responsible for the final draft version submitted for publication and is the project guarantor. ARP and RR were the main advisors, oriented the study from the beginning and reviewed the final version of the project. SD was also an important advisor and the final reviewer.

  • Funding The present publication was funded by Fundação Ciência e Tecnologia, IP national support through CHRC (UIDP/04923/2020).

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.