Article Text
PDF
Abstract
Objective To develop a set of quality indicators for the diagnosis and antibiotic treatment of adult patients with suspected urinary tract infections in general practice.
Design A Research and Development/University of California Los Angeles appropriateness method was used.
Setting Danish general practice.
Subjects A panel of nine experts, mainly general practitioners, was asked to rate the relevance of 27 preliminary quality indicators. The set of indicators was based on the most recent Danish guidelines for the management of patients with suspected urinary tract infection. An online meeting was held to resolve misinterpretations and achieve consensus.
Main outcome measures The experts were asked to rate the indicators on a nine-point Likert scale. Consensus of appropriateness was reached if the overall panel median rating was 7–9 with agreement. Agreement was defined as: no more than one expert rated the indicator outside the three-point region (1–3, 4–6 and 7–9) containing the median.
Results A total of 23 of the 27 proposed quality indicators attained consensus. One additional indicator was proposed by the panel of experts, leading to a final set of 24 quality indicators. All indicators focusing on the diagnostic process achieved consensus of appropriateness, while the experts agreed on three quarters of the proposed quality indicators concerning either the treatment decision or the choice of antibiotics.
Conclusion This set of quality indicators may be used to strengthen general practice’s focus on the management of patients with a possible urinary tract infection and to identify potential quality problems.
- antibiotic management
- continuous quality improvement
- general practice
- performance measures
- quality measurement
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- antibiotic management
- continuous quality improvement
- general practice
- performance measures
- quality measurement
WHAT IS ALREADY KNOWN ON THIS TOPIC
To improve antibiotic use in general practice, it is necessary to focus on both the diagnostic process and the prescribing patterns.
WHAT THIS STUDY ADDS
A total of 24 quality indicators for the management of patients with suspected urinary tract infections have been developed.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This set of indicators may be used to identify potential quality problems as a basis for reflection and opportunities for improvements.
Introduction
Antibiotic use is associated with the emergence of resistant bacteria, which is considered to be a major threat to human health worldwide.1 It is estimated that around 700 000 people die annually due to an infection that no longer can be successfully treated with antibiotics.2 The overall antibiotic use in Denmark is low compared with many other European countries, although higher than in some of the other Nordic countries.3
In Denmark, antibiotic consumption has decreased over the last decade, however, a large use of antibiotics, including a relatively high use of quinolones, has recently been found in the elderly population.4 About 80% of antibiotics are prescribed in general practice with urinary tract infections (UTIs) being one of the most common indications for antibiotic prescribing, particularly for women and elderly patients.2
In order to improve the quality of antibiotic use for UTIs in general practice, it is necessary to identify any quality problems.
Use of quality indicators (QIs) is a structured method allowing thorough detection of potential quality problems. QIs are defined as ‘specific and measureable elements of practice that can be used to assess the quality of care’,5 as they provide a quantitative measure of quality. Indicators are often constructed as a proportion6; defined with a numerator (number of patients receiving a specific investigation or treatment) and a denominator (number of patients included in the quality assessment). Quality is multidimensional and the development and interpretation of an indicator is not always straightforward. It is important to keep in mind that QIs do not provide definitive answers but indicate problems or good quality.
QIs for the management of patients with suspected UTI have previously been developed. Some of these indicators are developed for use in outpatient care7; others in hospitals8 and some are adapted to a specific group of patients.9 However, no QIs for outpatient care encompass the diagnostic process even though it is well known that a rational decision to prescribe antibiotics is based on a proper diagnosis.
Despite a clear demand,10 11 no QIs comprising both the diagnostic process and the decision to prescribe antibiotics for UTI, have so far been developed for use in general practice.
The aim of this project was to develop a set of QIs for the diagnosis and antibiotic treatment of patients aged ≥18 years with suspected UTI in general practice.
Methods
A Research and Development (RAND)/University of California Los Angeles (UCLA) appropriateness method was used for the development of the indicators. This method was established in the 1980s by the RAND health organisation12 in corporation with UCLA.13 It is a consensus method described as the only thoroughly tested systematic method combining evidence with expert opinion.14 The RAND/UCLA appropriateness method is widely used for the development of QIs in healthcare systems,15 16 and comprises four steps:
Development of preliminary QIs based on scientific evidence and guidelines.
First assessment of proposed QIs by a panel of experts using an emailed fact sheet (round 1).
Consensus meeting in which the panellists discuss the indicators.
Second assessment of QIs during the meeting (round 2).
Preparatory phase
A national research team comprising four researchers/physicians (including three of the authors) was established. The assignment of the research team was to generate a set of proposed QIs as well as to prepare and lead the consensus process.
The team identified all evidence-based Danish guidelines for the management of patients with suspected UTI in general practice. Only guidelines available online were included. Several main themes for quality measurement were identified from the guidelines. Whenever guidelines presented conflicting recommendations the most recent recommendation was used. The recommendations were operationalised as preliminary QIs with accompanying standards. Further evidence-based literature was sought whenever needed to complete the development of each of the preliminary QIs. This set of indicators was divided into three quality domains focusing on either; the diagnostic process (eg, the assessment of specific symptoms, signs and test results), the treatment decision (eg, to prescribe or withhold antibiotics) or the choice of antibiotics prescribed (eg, pivmecillinam, sulfamethizole, nitrofurantoin, trimethoprim or ciprofloxacin). The indicators comprised both lower and upper UTI for patients aged ≥18 years. Some QIs concerned only specific patient groups, such as catheter users and pregnant women. Each proposed QI was developed with an accompanying standard (acceptable range) to encompass the optimal performance addressed by that indicator. Standards were based on most recent national guidelines. Low quality of the care provided is indicated if a performance falls outside the standard range. Furthermore, a registration chart (online supplemental appendix 1) to collect data for quality investigation was developed to match the proposed indicators.
Supplemental material
Patients or the public were not involved in the design, conduct, reporting or dissemination plans of this research.
The expert panel
A broad national nine-person panel representing all five regions of healthcare administration of Denmark was established. The panel comprised seven general practitioners (GPs), one microbiologist and one infectious disease specialist. Seven women and two men participated (please see the Acknowledgement section for further details about the panel members). To be deemed as an expert, the person needed to have profound knowledge and experience within the management of UTI, general practice and/or quality assessment. The experts were purposively sampled to ensure participants from diverse geographical areas and different professional positions.
Round 1
A fact sheet was prepared for each proposed QI including the definition of the QI, standard and domain, relevant patient population and evidence with reference to relevant literature. Fact sheets with the 27 proposed QIs were distributed to all experts via email. The experts were asked to rate each indicator on a 9-point Likert scale ranging from 1 (completely disagree) to 9 (completely agree). Each indicator had to be rated for the relevance of measuring the quality of a health professional’s management of patients with suspected UTI in general practice. The experts were provided with links to relevant guidelines17–23 and encouraged to use this evidence-based information when assessing the indicators. The experts were asked to rate each indicator with the accompanying standard as an integral unit. However, if the experts disagreed with the standard they were asked to explain the disagreement and rate the QI separately.
Round 2
All nine experts attended the 2.5 hours online meeting held in November 2020, 7 days after the first round of rating was completed. One expert (a microbiologist) was unable to attend the entire meeting, but completed the process through postmeeting rating.
Each expert was provided with feedback on their own ratings by means of a bar chart showing the distribution of ratings from the first round, with the expert’s own rating marked in the figure. Only QIs that did not reach consensus during the first round were discussed at the online meeting. Each of these indicators was discussed and experts were encouraged to propose new QIs or rephrase the already existing ones. The discussion was facilitated by a moderator (LTS) from the research team and evidence-based literature was cited whenever appropriate. Finally, the experts were asked to rate these indicators again at the meeting.
Analysis
For each QI, medians of the Likert scores were calculated and the indicators were classified into three levels of appropriateness based on the recommendation from the RAND team13: (1) appropriate (accepted and not further assessed) was defined as a panel median of 7–9 with agreement; (2) uncertain (and included in the next assessment round) was defined as a panel median of 4–6 or any median without agreement and (3) inappropriate (excluded and not further assessed) was defined as a panel median of 1–3 with agreement. Agreement was defined as: no more than one expert rated the indicator outside the three-point region (1–3, 4–6 and 7–9) containing the median.
Those indicators classified as appropriate from either the first or the second round were included in the final set of indicators.
Results
A total of 16 QIs reached consensus after the first round of ratings. The remaining 11 indicators were discussed and reassessed at the following online meeting. Two QIs (QI9 and QI16, tables 1 and 2) were rephrased at the meeting and one additional indicator (QI17) was proposed by the panel of experts.
Medians for 11 quality indicators (QI) and accompanying standards focusing on the diagnostic process of urinary tract infections (UTIs)
Medians for nine quality indicators (QI) and accompanying standards focusing on the treatment decision for urinary tract infections (UTI)
Following consensus of appropriateness, the final set comprised 24 QIs for the management of patients with suspected UTI in general practice.
Consensus of appropriateness was attained for all of the 11 proposed QIs focusing on the diagnostic process (table 1). For example, the experts agreed on the relevance of measuring the performance of urine culture and susceptibility testing for patients diagnosed with complicated lower UTI (QI4). The importance of assessing the use of urine culture and susceptibility testing was likewise agreed on when referring to patients with pyelonephritis and specific patient groups (pregnant women or catheter users) who might have a UTI (QI6, QI10 and QI11). The panel of experts also agreed on the relevance of measuring the appearance of UTI symptoms and high probability of bacteriuria (a positive urinary dipstick and/or a positive microscopy) when patients were diagnosed with UTI (QI2).
Consensus of appropriateness was attained for six of the eight proposed QIs focusing on the treatment decision (table 2). The experts agreed on the relevance of measuring antibiotic treatment for catheter users with suspected UTI (QI16), and concurrently proposed an additional indicator measuring change of catheter for catheter users with symptoms of UTI (QI17).
Consensus of appropriateness was attained for six of the eight QIs focusing on the choice of antibiotics prescribed (table 3).
Medians for eight quality indicators (QI) and accompanying standards focusing on the choice of antibiotics prescribed for urinary tract infections (UTIs)
The experts agreed on the relevance of measuring the use of pivmecillinam as first choice antibiotic for treatment of patients with suspected lower or upper UTI and for pregnant women with a possible UTI (QI19, QI21 and QI24). The panel also agreed on the relevance of measuring the use of ciprofloxacin for patients with either lower or upper UTI, and for catheter users with suspected UTI (QI20, QI22 and QI23).
Three standards were modified during the consensus process. For example, the proposed standard for the indicator concerning patients with suspected lower UTI treated with ciprofloxacin was set to 0%–10%, but changed to 0%–5% according to the expert’s recommendation and consensus (QI20).
Discussion
Main findings
A panel of Danish experts agreed on a total of 24 QIs for the management of patients with suspected UTI in general practice. All QIs focusing on the diagnostic process achieved consensus of appropriateness, while the experts agreed on three quarters of the proposed QIs concerning either the treatment decision or the choice of antibiotics.
Strengths and limitations
Previous studies have demonstrated the need for validated QIs, developed by an appropriate methodology, to be able to assess the diagnostic process and the treatment decision for patients with suspected UTI in general practice.10 11
This study adhered to a widely recognised, systematic method combining evidence with expert opinion ensuring a transparent and scientific process for the development of indicators.
During the consensus process, the experts evaluated the face validity of the proposed QIs, that is, if the indicators reflected the quality issue that was intended to be measured.24 Moreover, the QIs were based on guidelines and scientific evidence, which are considered to provide QIs with content validity.14 Thus, validation of each QI was an inherent part of the development process. The feasibility of the QIs has been evaluated by application of the indicators to relevant data.25
Previous studies have demonstrated that adherence to guidelines on antibiotic use is protective against treatment failure and mortality.26 27 This set of QIs is based on the most updated evidence-based national guidelines for UTI.
All indicators only concerned day 1, that is, the day of the first contact to general practice. Consequently, no indicators were designed to include, for example, the result of a urine culture. We chose this design because most antibiotics for UTI are prescribed at day 1.28 Information about patient’s symptoms and signs and results of point-of-care tests are likewise available at day 1. Accordingly, we believe that insight into the care provided at day 1 is sufficient to generate useful and comprehensive knowledge about the quality of care for patients with suspected UTI in general practice.
One expert attended only 1 hour of the consensus meeting. However, this expert was provided with comprehensive information from the last part of the meeting before asked to rate the indictors for the second time.
Comparisons with other studies
QIs for the management of patients with suspected UTI have previously been developed.7–9 However, to our knowledge, the indicators from this study are the first set of indicators developed for use in general practice, comprising both the diagnostic process and the decision to prescribe antibiotics for UTI.
The indicator (QIB) that evaluated if patients with high probability of uncomplicated lower UTI (≥1 symptom of lower UTI and a positive urinary dipstick and/or a positive microscopy) were treated with antibiotics, did not obtain consensus. It is well documented that antibiotic treatment is superior to non-antibiotic treatment regimens in terms of achieving bacteriological cure and symptomatic relief.29 However, the results on the risk of complications such as pyelonephritis are debated. Some studies have documented an increased risk of pyelonephritis when ibuprofen was prescribed instead of antibiotics.30 Meanwhile previous studies have found very few cases of complications and no increased risk of pyelonephritis when treatment with pivmecillinam was compared with placebo.31 32 The European Surveillance of Antimicrobial Consumption Network managed to develop a similar indicator: percentage of female patients older than 18 years with cystitis/other urinary infection prescribed antibiotics for systemic use, standard range 80%–100%, although the indicator did not reach consensus for all dimensions during the consensus process.7
The result of the urinary dipstick was included in several of the indicators proposed. Diagnostic accuracy for UTI improves considerably when symptoms and signs are combined with the result of the dipsticks test.33 Dipstick results are also included in a model to predict antibiotic prescriptions for UTI.34
Several studies have explored the link between the non-specific symptom confusion and UTI, however, the association is not clearly documented.11 35
Interestingly, two of the QIs in our set of proposed indicators were rephrased (QI9 and QI16). The rephrasing of both indicators involved a deletion of the proposed specific symptom: emerging confusion in catheter users with suspected UTI. The rephrased indicators ended up with the inclusion of only four specific symptoms for catheter users with suspected UTI: (1) fever, (2) shivering, (3) flank pain and (4) systemically unwell.
Examination of a urine sample is not recommended in patients without UTI symptoms and guidelines recommend no antibiotic treatment for asymptomatic bacteriuria.36 37 Nonetheless, it is very likely that, in every day general practice it may happen anyway. The experts agreed on the indicator concerning urinalysis in patients with no UTI symptoms (QI1). Contrarily, the indicator that evaluated if patients without UTI symptoms but with a high probability of bacteriuria (positive urinary dipstick and/or a positive microscopy) were treated with antibiotics, did not reach consensus (QIA). Some of the experts did not agree on QIA because it included two not recommended actions: (1) to examine urine in patients without UTI symptoms and (2) to treat patients without UTI symptoms with antibiotics.
Perspectives
A set of 24 QIs for the diagnosis and treatment of patients with suspected UTI in general practice has been developed. Studies have shown that GPs in general have a positive attitude towards the use of QIs.38 This set of indicators may be used to strengthen GPs’ focus on their management of patients with suspected UTI and identify potential quality problems. The indicators are not only applicable to Danish general practice but may be applied to for example general practices in the Nordic countries with antibiotic use, resistance pattern and practice setting similar to Danish conditions.
In Denmark, such as many other countries, general practices do not have detailed systematic data registration systems. Therefore, application of indicators to measure and improve patient care is currently a time-consuming activity, hampering a systematic application of these indicators.
Since 2018, Danish GPs have been joined in ‘quality clusters’ for quality discussion and support.39 This set of QIs can ideally be applied to data and used as a basis for reflection and discussion of opportunities for improvements in these ‘quality clusters’.
Furthermore, the indicators can advantageously be used for an intervention programme aiming at improving the quality of the diagnostic approach and antibiotic use for patients with suspected UTI.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Ethics statements
Patient consent for publication
Acknowledgments
The authors would like to thank the panel of experts for their valuable contribution to this study: Dorte Bojer, Sidsel Böcher, Mette Fredberg-Greth, Niels Frimodt-Møller, Gregers Hansen-Nord, Anne Holm, Isik Somuncu Johansen, Berit Lassen and Gitte Krogh Madsen.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors All authors have made substantial contributions to the conception, design, analysis or interpretation of data for the work. The corresponding author, LTS, drafted the first manuscript and all other authors revised the entire manuscript critically and approved the final version for publication. LTS also served as the guarantor for the study. VDS has contributed to study statistics. The experts have contributed valuable in terms of their assessment of the quality indicators.
Funding This study was funded by Kvalitets- og efteruddannelsesudvalget (KEU) in the Capital Region of Denmark and the Section of General Practice, Department of Public Health, University of Copenhagen, Denmark, to whom we extend our gratitude. No award/grant number.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.