Article Text
PDF
Abstract
Purpose The purpose of this study was to examine the scope, quality dimensions and treatment aspects covered by existing quality indicators (QIs) for the somatic diseases bronchial asthma, atopic eczema, otitis media and tonsillitis as well as the psychiatric disorders attention deficit hyperactivity disorder (ADHD), depression and conduct disorder in paediatrics.
Methods QIs were identified through an analysis of the guidelines and a systematic search of literature and indicator databases. Subsequently, two researchers independently assigned the QIs to the quality dimensions according to Donabedian and Organisation for Economic Cooperation and Development (OECD) and to the content categories covering the treatment process.
Results We found 1268 QIs for bronchial asthma, 335 QIs for depression, 199 QIs for ADHD, 115 QIs for otitis media, 72 QIs for conduct disorder, 52 QIs for tonsillitis and 50 QIs for atopic eczema. Of these, 78% focused on process quality, 20% on outcome quality and 2% on structural quality. Using OECD criteria, 72% of the QIs were assigned to effectiveness, 17% to patient-centredness, 11% to patient safety and 1% to efficiency. The QIs covered the following categories: diagnostics (30%), therapy (38%), patient-reported outcome measures/ observer-reported outcome measures/patient-reported experience measures (in sum 11%), health monitoring (11%) and office management (11%).
Conclusion Most QIs focused on the dimensions of effectiveness and process quality, and on the categories of diagnostics and therapy, with outcome-focused and patient-focused QIs being under-represented. Possible reasons for this striking imbalance could be the easier measurability and clearer assignment of accountability in comparison to the QIs of outcome quality, patient-centredness and patient safety. To produce a more balanced picture of the quality of healthcare, the future development of QIs should prioritise the currently under-represented dimensions.
- Quality measurement
- Quality improvement
- Primary care
- Paediatrics
- Patient-centred care
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
High-quality healthcare plays a crucial role in the diagnosis and management of children and adolescents with health problems. While there exists a vast range of quality indicators for the assessment of healthcare in paediatrics, an up-to-date overview of the scope and quality dimensions covered by them is lacking.
WHAT THIS STUDY ADDS
Our analysis found 2091 quality indicators (QIs) designed to assess healthcare quality in children and adolescents for the diseases asthma, atopic eczema, otitis media, tonsillitis, attention deficit hyperactivity disorder, depression and conduct disorder. The majority of QIs focus on measuring process quality and effectiveness, while clinical outcome, patient-centredness and patient safety are only covered to a much lesser extent.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This study demonstrates a particular need for the development of QIs focusing on outcome quality, patient-centredness and patient safety in order to ensure that child and adolescent healthcare quality is measured in a balanced and comprehensive way.
Introduction
In the field of medicine, quality indicators (QIs) are used globally to capture the construct of quality that is not directly measurable. They are considered as metrics used to quantify measures for monitoring and assessing the quality of key leadership, management, clinical and supportive functions.1 Quality is indirectly represented by numerical ratios, and can identify potential deficits by showing high or low values.2 QIs are assigned to quality dimensions to assess their balance and content focus. For this purpose, various frameworks for measuring quality of care (QoC) are presented in the literature. Donabedian (1966) distinguished between structural, process and outcome quality, and influenced evaluation in healthcare worldwide.3 Structural quality includes all the resources necessary for the provision of healthcare services (eg, equipment of the doctor’s office). Process quality captures all components of the performance of a treatment process (eg, counselling) and outcome quality defines changes in health status (eg, reduction in hypertension).3 4 Since then, various frameworks have been developed.5–10 All of them share the core dimensions of effectiveness, patient safety and patient-centredness.4 This distinction was elaborated in the Health Care Quality Indicator project (2006) of Organisation for Economic Cooperation and Development (OECD), which aimed to develop indicators for international comparisons of healthcare quality. Effectiveness is defined as the extent to which desirable outcomes are achieved according to evidence-based and guideline-compliant healthcare services. Patient safety captures the occurrence of preventable adverse events such as side effects or harm.8 One example is the overuse of antibiotics, often owing to diagnostic uncertainty, which leads to an increased occurrence of antimicrobial resistance.11 According to the OECD, the patient-centredness dimension reflects whether the patient is at the centre of healthcare and its related experiences are considered and included in treatment.8 12 Further dimensions, such as appropriateness, continuity and access to treatment, can be subordinate to these three core dimensions.4
High QoC plays a particularly significant role in the treatment of children and adolescents. In this age group, the risk of adverse events such as consequential damages and chronicity of disease due to inadequate diagnosis or treatment is particularly high.13 To measure the QoC, individual QIs or QI sets for routine paediatric care have been published by experts and organisations. For example, Ewald et al14 developed a set of 42 QIs to assess the quality of paediatric primary care in Europe as part of the COSI PPC-EU study (Core set of quality indicators for Paediatric primary care practices in Europe). Another example is the Care Track Kids study, for which Homaira et al15 designed a QI set for 17 common diseases, such as bronchiolitis, to assess the quality of paediatric primary care in Australia. To date, two reviews have compiled and examined QIs for assessing healthcare of children and adolescents. Beal et al16 identified 19 QI sets and 396 individual QIs designed for use in paediatrics and assigned them to quality dimensions according to the Institute of Medicine; 14% of the QIs for patient safety, 59% for effectiveness, 32% for patient-centredness and 33% for timeliness. In 2009, another study by Kavanagh et al17 assigned 606 QIs from seven QI sets to the children from USA and UK for quality dimensions according to Donabedian. Of these, 97% and 3% represented process and outcome quality, respectively, while none showed structural quality.
It is unclear how many QIs on paediatric care have been developed and published since then and which aspects of treatment and quality dimensions they cover. To identify the scope as well as the focus and potential deficits of existing QIs, further analyses and an update of the state of knowledge on QIs for common paediatric diseases is mandatory. To cover a broad spectrum of common diseases in childhood and adolescence, we selected asthma, atopic eczema, otitis media, tonsillitis, attention deficit hyperactivity disorder (ADHD), depression and conduct disorder (CD) for this study. Thus, both somatic and psychiatric conditions as well as chronic and acute diseases were examined, as these are some of the most common childhood and adolescent diseases.18–20
Therefore, in this study, we explored the number of QIs that exist for the treatment of asthma, atopic eczema, otitis media, tonsillitis, ADHD, depression and CD in children and adolescents and their quality dimensions. The analyses took place as part of a project that systematically evaluated the QoC for outpatient paediatric patients in Germany.13
Methods
The methodological approach consisted of two steps that are explained in detail as follows:
Step 1: review of published QIs
First, a review was conducted using indicator, guideline and literature databases. The indicators were entered and summarised in the register. The results were quantitatively synthesised using numerical counts.
Indicator database search
The search in indicator databases (eg, AHRQ and NICE) focused on paediatric QIs for the seven diseases. Only freely accessible English and German databases were included in this study.
Guideline analysis
Guidelines from German organisations (eg, Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften e. V. (AWMF)) and international guideline databases (eg, Guidelines International Network (GIN)) were analysed for possible indicators. Only explicitly defined QIs (indicator statements, nominators and denominators) were included in the register.
Literature review
A systematic literature search was conducted for the seven diseases in the electronic databases including PubMed, Web of Science, Cochrane Library and Embase for QIs published by October 2020 and for psychiatric disorders in PsycINFO and PSYNDEX published by April 2021. First, a potential search string was developed independently by two researchers, tested and operationalised by joint consensus. The systematic search string for QI ((quality indicator healthcare) AND (indicator*[Title/Abstract])) was combined with disease-specific search terms. An example of the Boolean operators that were used for the individual diseases in PubMed are as follows: (asthma), (atopic eczema) OR (atopic neurodermatitis) OR(atopic dermatitis), (otitis media), (angina tonsillaris) OR (tonsilliti*), (attention deficit disorder) OR (adhd), (depress*), (oppositional defiant disorder) OR (conduct disorder). Literature review restricted to paediatric cases was not implemented in advance. After duplicates were removed, title/abstract screening for irrelevant literature was independently performed by two individuals. For the remaining studies, the full texts were screened and re-evaluated for relevance. In this process, literature pertaining exclusively to adults was excluded. Subsequent grey research expanded the search. In all three steps, the disease patterns in line with the inclusion criteria and publication languages other than German or English, were excluded.
Step 2: assigning QIs to dimensions and categories
To analyse the scope and balance of the QIs more precisely, content and quality dimensions were assigned to the QIs. For this purpose, the Donabedian framework with the dimensions of process, outcome and structural quality as well as effectiveness, patient-centredness and patient safety according to the OECD were employed. For QIs that capture the costs of treatment, the dimension ‘efficiency’ was added.
The categories ‘health monitoring’, ‘diagnostics’ and ‘therapy’ were selected based on the content structure of the German national healthcare guidelines. The category ‘health monitoring’ was used for the assignment of QIs, which record epidemiological measures (eg, prevalence), frequencies of the use of medical services and associated costs. In addition, the categories ‘Office management’, for processes in the practice and organisational aspects, and ‘patient-reported outcome measures/observer-reported outcome measures/patient-reported-experience measures (PROM/ObsROM/PREM)’ were added. Patient-reported outcomes (PROs) refer to self-reported health status information such as quality of life, symptoms or treatment effects.21 If patients are too young, ill or cognitively impaired, observer-reported outcomes (ObsROs) can be used. These outcomes are based on the observations of someone other than the patient; mostly parents or caregivers.22 Patient-reported experiences (PREs) describe patients’ perceptions of their treatment experiences.21
Each indicator was assigned to one dimension each of Donabedian and OECD, and one content category. As PROS and ObsROs are necessary for the diagnosis and treatment of psychiatric disorders, some psychiatric QIs were assigned to more than one category. Based on an exact definition of the categories and dimensions with corresponding indicators as an example, the assignment to the categories and dimensions was made by two persons independently of each other. The assignments were then compared, divergent results discussed and then jointly reassigned. If no agreement could be reached, an indicator expert was consulted in order to reach a consensus.
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Results
Systematic literature analysis
A total of 2091 QIs identified through systematic research were accepted as relevant for this review (figure 1). Table 1 shows the distribution of the QIs sorted by disease and references. The search of literature databases proved to be particularly productive as 1722 QIs were identified. In contrast, only 31 QIs were identified based on the guidelines. The highest number of QIs were covered for asthma (1268 QIs) and the lowest for atopic eczema (50 QIs) and tonsillitis (52 QIs).
Overview of identified quality indicators by disease and source
Flow chart of quality indicators identified, screened and included. (ADHD, attention deficit hyperactivity disorder; CD, conduct disorder; QIs, quality indicators)
Assigning categories and dimensions
The dimensions and content categories were assigned to the individual QIs. Table 2 shows examples. First, the QIs were assigned to quality dimensions according to Donabedian (see table 3). More than three-quarters (77.8%) of the total 1627 QIs addressed process quality. A total of 425 (20.3%) and 39 (1.9%) QIs captured the outcome and structural qualities, respectively. Otitis media and tonsillitis showed the highest proportion of QIs measuring process quality (95.7% or 90.4%) and the lowest for outcome quality (4.3% or 9.6%) compared with other diseases. No QIs for structural quality were identified for either of the diseases. CD showed the lowest proportion of QIs for process quality (59.7%) and the highest for outcome quality (36.1%).
Examples of categories and dimensions of quality indicators
Overview of quality indicators by disease and dimension according to Donabedian
Table 4 reports the results of the assignment of the QIs to the quality dimensions according to OECD and efficiency. A total of 1496 QIs, which is slightly less than three-quarters (71.5%), covered the quality dimension of effectiveness, 352 QIs (16.8%) for patient-centredness and 227 QIs (10.9%) for patient safety. The remaining 16 QIs (0.8%) captured efficiency. The lowest proportion of QIs addressing effectiveness was CD (58.3%), while asthma (76.1%) and otitis media (74.8%) showed the highest proportion in this dimension. Regarding patient-centredness, CD (33.3%) and depression (28.4%) showed the highest proportions of QIs; however, tonsillitis (7.7%) and otitis media (0.9%) showed the lowest. Furthermore, tonsillitis and otitis media had the highest proportions of QIs for patient safety (tonsillitis 26.9%; otitis media 24.3%) but asthma (8.7%) and CD (8.3%) showed the lowest. Among seven diseases, asthma is the only disease whose identified QIs also captured efficiency (1.3%).
Overview of quality indicators by disease and dimension according to OECD
Table 5 lists the assignments of the categories. Most QIs (n=841, 37.7%) covered aspects of therapy, while 660 QIs (29.6%) addressed diagnostics. A total of 254 QIs (11.4%) were assigned to the PROM/ObsROM/PREM category, 237 QIs (10.6%) to health monitoring and 239 QIs (10.7%) to office management.
Overview of quality indictators by disease and category
ADHD showed the highest proportion (49.8%) of QIs for diagnostics, while there were low proportions for otitis media (11.3%) and tonsillitis (17.3%). In the therapy category, the lowest proportion of QIs was found for ADHD (17.5%) and the highest for otitis media (79.1%) and tonsillitis (69.2%). The proportion of QIs addressing PROM/OBsROM/PREMs was higher for the psychiatric disorders ADHD (21, 1%), depression (19.0%) and CD (28.1%) than those for somatic disorders. In this category, the chronic somatic diseases asthma and atopic eczema accounted for 7.3% and 12.0% of their QIs, respectively, while no corresponding QI was identified for otitis media and tonsillitis. Regarding health monitoring, asthma (14.5%) and CD (16.9%) presented the highest proportions, while atopic eczema (2.0%) presented the lowest. The two somatic chronic diseases asthma and atopic eczema showed the highest QI proportions of all diseases in the office management category (12.9% and 16.0%, respectively).
Discussion
This review updates the state of knowledge on QIs for common paediatric diseases, allowing for the selection of indicators for evaluating paediatric healthcare. The QIs can be used to analyse patient records with regard to the QoC provided and to derive opportunities for improving care. The comprehensive systematic analysis focuses on asthma, atopic eczema, otitis media, tonsillitis, ADHD, depression and CD and the number of QIs and their aspects that are present for these diseases in childhood and adolescence.
The findings revealed many differences in the numbers of QIs. Almost two-thirds of all QIs were associated with asthma. This comparably high number can be attributed to various factors. For several decades, the prevalence and mortality in adults and children due to asthma has increased worldwide, and are associated with significant suffering and costs.23 Asthma has long been recognised as a global problem, and preventive efforts and improvements in its treatment are mandatory.24 25 Furthermore, the high number of QIs may also be due to frequently changed recommendations in the treatment and revised guidelines (eg, changes in classification schemes).
The assignments of QIs to quality dimensions show a similar focus as shown in reviews by Kavanagh et al and Beal et al. Kavanagh et al reported the highest proportion of QIs capturing process quality. In contrast, there are currently more QIs on outcome and structural quality. In both the reviews, most QIs captured effectiveness, and the smallest proportion captured patient safety after efficiency. Beal et al also examined QIs for timeliness in addition to the three OECD dimensions, which covered one-third of the QIs and the efficiency dimension.
One reason for the high proportion of QIs capturing process quality or effectiveness could be the definite assignment of accountability.18 Both dimensions ask for the implementation of specific measures that are as evidence-based as possible. These are usually assigned to a specific group of healthcare providers, such as paediatricians. This assignment simplifies the interpretation of the quality measures using indicators. Consequently, concrete needs for action can be derived to achieve better quality in the future. For QIs addressing outcome quality, patient-centredness and patient safety, the determination of accountability is often unclear. In these cases, it depends on the results (eg, controlled asthma) or the involvement of the patient in the treatment process, which is associated with various factors such as the collaboration of multiple care providers, the patient’s or parents’ willingness to participate, existing comorbidities, and psychological and socioeconomic factors.26 Owing to the lack of clear accountability, complex analyses are necessary to interpret the results of the corresponding QIs. Because QIs addressing process quality or effectiveness are easier to measure, the high proportion of QIs for these two dimensions were found in previous reviews and in this study.
Disease-specific differences were also identified by analysing the dimensions. Acute somatic diseases including otitis media and tonsillitis showed the lowest balance of QIs. The high proportions of QIs for therapy category and the quality dimensions, such as process quality and patient safety, can be explained by the course of treatment for the two diseases. Diagnosis is comparatively less complex than that in the case of chronic diseases, and treatment is mostly an exclusive drug therapy. Therefore, the focus of QIs is on the prescription of drugs and the associated possible risks and adverse events (eg, antibiotic resistance). Due to missing or very low proportions of QIs capturing patient-centredness and PROM/OBsROM/PREMs, a clear deficit is evident. However, psychiatric disorders showed higher proportions of QIs for PROM/ObsROM/PREM and patient-centredness than those for somatic disorders. This is due to the fact that reports from the patient or its social environment form the basis of diagnosis and course of therapy (eg, via standardised questionnaires). Thus, with reference to OECD dimensions and categories, the QIs for psychiatric conditions are more balanced than those for somatic diseases.
Divergent focuses are understandable because of the different types of illnesses (acute and chronic, somatic and psychiatric) with different diagnostic procedures and forms of treatment. Nevertheless, to comprehensively assess the quality of the entire course of treatment, there should be QIs for every disease for all dimensions and categories mentioned in this study. On this basis, a disease-specific selection can be made without running the risk that a particularly significant category or dimension is not assessed simply because there are no QIs for it. To achieve high QoC, it is important to include patients’ well-being and experiences in the treatment, not only for psychiatric diseases but also for somatic diseases. Therefore, QIs for PROM/ObsROM/PREMs should be included in the future, especially for tonsillitis and otitis media. In general, it is not important to focus exclusively on effectiveness and process quality; the ICHOM initiative is exemplary in this regard (www.ichom.org). Thus, the described problem of accountability for the outcome quality, patient-centredness and patient safety dimensions must be addressed. Overcoming the imbalance in terms of the quality dimensions covered by QIs is a complex task for which various solutions are conceivable. Examples include the creation of standardised questionnaires to capture PROs and experiences of somatic conditions, patient participation in the development of QIs, structured recording of patient safety incidents and structural changes such as an electronic health record shared by all providers.
This study has several limitations. First, the development of a search string focused exclusively on the term ‘QI’. The very early specification might have resulted in a lack of relevant articles not being found. Further searches could potentially add similar search terms such as ‘healthcare quality measure(s)’ or ‘quality measure(s)’ or performance measure(s). This weakness is partially compensated for by the fact that we have researched not only peer-reviewed publications but also the grey literature. This approach is due to the fact that QIs are often developed by official organisations that are responsible for monitoring the QoC and do not publish their QIs in peer-reviewed journals. In addition, QIs are found as part of guideline reports, which are also often not published in this form. Therefore, searching grey literature for a comprehensive account of existing QIs for a condition is always indicated. Second, the results are limited to only seven diseases and do not cover the entire spectrum of child and adolescent care. Therefore, it is possible that the distribution of QIs may differ for other diseases with similar prevalence rates, such as allergic rhinitis or anxiety disorders.
Third, with respect to the overall analysis, it should be noted that it is dominated by the asthma QIs, which represented almost two-thirds of all QIs found. Therefore, the distribution of QIs across categories and dimensions in the overall analysis is also skewed by the asthma QIs. However, the present individual analyses allow us to assess the distribution of QIs found per disease among the categories and dimensions.
Conclusion
This analysis investigated the quantity and scope of existing QIs for the treatment of common diseases in children and adolescents and updated the research status. The QIs analysed were found to be focused on process quality and effectiveness owing to the easier measurability and interpretation of the results. However, to assess quality comprehensively, the dimensions of outcome quality, patient-centredness and patient safety should also be considered. Greater attention should be paid to these dimensions in the development of future QIs. For this purpose, concepts should be promoted to simplify the measurability and subsequent evaluation of QIs.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
References
Footnotes
Contributors Project design (CB and MG); Data research (CM); Data processing (CM, TM and TN). Analysis of data (CM and TM); Supervision (MG). Drafting of manuscript (TM); Manuscript revisions (CB, CM and MG), Guarantor (MG)
Funding The work is supported by Gemeinsamer Bundesausschuss (G-BA) (grant number: 01VSF19035).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.