Article Text
PDF
Abstract
Chronic obstructive pulmonary disease (COPD) is the third leading cause of death worldwide and is estimated to be the leading cause of death in the next 15 years. Patients with COPD suffer from persistent chronic cough, sputum production and exacerbations leading to deteriorating lung function, worsening quality of life and loss of independence. While evidence-based interventions exist to improve the well-being of patients with COPD, incorporation of these interventions into routine clinical care is challenging. Chronic Obstructive Pulmonary Disease Coordinated Access to Reduce Exacerbations (COPD CARE) is a team-based, coordinated care transitions service integrating evidence-based interventions for COPD management within the patient care delivery model to reduce readmissions. This evaluation considers the process of scaling the COPD CARE service across medical facilities using an implementation package designed for service expansion. The implementation package was developed at the United States Veterans Health Administration and implemented at two medical centres. Core dissemination and implementation science methods were applied to guide design and delivery of the implementation package.
The aims of this evaluation were to (1) evaluate the impact of the implementation package on use of evidence-based interventions for COPD management and (2) explore clinician perceptions of the implementation package. This prospective mixed-methods quality improvement project included two Plan Do Check Act (PDCA) cycles conducted over a 24-month period. Electronic health record data demonstrated significant improvements in the count of evidence-based interventions incorporated into routine clinical care after training completion (p<0.001), offering preliminary effectiveness of the package to improve uptake of best practices for COPD management. Clinician perceptions of the implementation package, measured by questionnaire at multiple time points, demonstrated significant improvements for all scales at the end of the final PDCA cycle. Clinicians described the implementation package as positively impacting clinician confidence, interprofessional collaboration and patient care delivery.
- Quality improvement
- Pharmacists
- Medical education
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. Not Applicable.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Interventions to improve outcomes for patients with chronic obstructive pulmonary disease (COPD) have been shown to be effective and safe; however, widespread uptake of such best practices across the healthcare system remains a challenge. The Chronic Obstructive Pulmonary Disease Coordinated Access to Reduce Exacerbations (COPD CARE) service integrates evidence-based interventions using a team-based approach to care delivery immediately following hospital or emergency department discharge and has been shown to reduce COPD readmission rates and improve timely access to primary care follow-up.
WHAT THIS STUDY ADDS
This quality improvement evaluation demonstrates the effectiveness of an implementation package to improve the rapid uptake of a COPD service. The implementation package led to increased utilisation of COPD best practices and greater clinician self-efficacy.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This evaluation informs future refinement of the COPD implementation package to more effectively scale up the COPD care service. Lessons learnt regarding logistical barriers and facilitators to implementing the COPD CARE service are relevant to health system leaders, practitioners or researchers seeking to scale up similar practice models using an implementation package.
Problem
Chronic obstructive pulmonary disease (COPD) is the third leading cause of death globally, with 3.17 million deaths in 2015 alone.1 2 Within the USA, over 15 million Americans have been diagnosed with COPD, resulting in one COPD-related death every 4 min.3 The projected economic burden for treating COPD in the USA in 2020 was expected to be $49 billion USD, a 53% increase from 10 years prior.4 In addition, the current global pandemic has disproportionally impacted patients living with COPD with pre-existing disease diagnosis identified as the strongest predictive comorbidity for both severe disease and ICU admission in patients infected with COVID-19.5
COPD is especially burdensome among US Veterans, with a threefold greater prevalence of COPD among Veterans as compared with civilians.6 Moreover, one in four US Veterans are rehospitalised for COPD within 1 year of a COPD exacerbation, leading to greater risk for disease progression and worsening health outcomes.7 8 Within 1 year of a severe exacerbation, the mortality rate from COPD in the US Veteran population is 21%, increasing to a 55% mortality rate within 5 years.7
While COPD is progressive and fatal, it is treatable, which can improve patient’s quality of life. Global, evidence-based guidance for COPD management emphasises an integrated team-based approach to care delivery. Clinical practice guidelines describe the importance of implementing evidence-based interventions such as disease education, inhaler technique assessment, management of comorbidities and the provision of a written COPD action plan to improve symptom control.9 Chronic Obstructive Pulmonary Disease Coordinated Access to Reduce Exacerbations (COPD CARE) is a service that integrates evidence-based interventions using a team-based approach to care delivery immediately following hospital or emergency-department discharge. The service has been shown to reduce COPD readmission rates and improve timely access to primary care follow-up.10
While interventions to improve outcomes for patients with COPD have been shown to be effective and safe, improving widespread uptake of best practices across the healthcare system remains a challenge.11 Interventions exploring how COPD best practices can be incorporated into routine care delivery are needed to improve the quality of life of patients living with COPD. Implementation packages are an approach to enhance adoption of COPD best practices; however, to the authors’ knowledge, limited information exists describing their design or impact on care delivery in the USA.12 Evaluations emphasising the impact of implementation packages designed to promote the widespread adoption, implementation and sustained use of COPD best practices are critical to optimise COPD management across healthcare systems.
Background
The COPD CARE service was designed to promote incorporation of best practices into routine clinical care through interprofessional, team-based collaboration for Veterans with COPD with recent hospitalisation for COPD exacerabations. The service integrates the clinical pharmacist practitioner (CPP), registered nurse (RN) and primary care provider (PCP) to deliver team-based care during patient transitions from the hospital or emergency department to home.10 COPD CARE is delivered as a bundled service with a series of structured patient contact opportunities completed by members of the interprofessional team. The added structure and coordination in care delivery that is unique to care bundles allows for COPD best practices to be implemented at various time points during patient care transitions.
Within COPD CARE, patients receive a COPD Wellness Visit, which is an interprofessional visit with the CPP and RN where symptoms are assessed, patient inhaler technique is reviewed, medications are optimised and referrals are placed as applicable to help patients quit smoking, improve mobility and physical fitness and to help promote appropriate nutrition. The CPP within COPD CARE is credentialed and privileged to prescribe medication therapy and coordinate necessary referrals, which allows for timely actions to be taken to improve patient outcomes postdischarge. The RN extensively focuses on patient education and emphasises use of a written COPD Action Plan to provide patients with steps to take when COPD symptoms flare. The PCP serves as a team lead and coordinates a patient care plan after the care transition is completed.
The COPD CARE service has been shown to reduce COPD 30-day readmission rates to 0% while increasing patient access to timely care posthospital discharge.10 COPD CARE was selected as one of 11 national promising practices across the US Veterans Health Administration (VA) national healthcare system through the Diffusion of Excellence Program,13 which aims to expand practices identified by frontline employees across the VA.14–17
While COPD CARE incorporates best practices and has shown to be effective, disseminating this service to additional VA medical centres (VAMCs) is a challenging task. There are 171 VAMCs across the USA each with their own unique characteristics, including different levels of facility complexity, patient demographics, specialisation capacity and service delivery models during care transitions.14 Given these challenges to disseminating COPD CARE to fit each local VA setting, there is a definitive need for a focused, methodical approach to scale this service across institutions.
Design
This evaluation presents the impact of an implementation package to scale the COPD CARE service during patient care transitions within two VAMCs. Through completion of two Plan Do Check Act (PDCA) cycles,18 19 this evaluation aimed to (1) evaluate how training influenced COPD care delivery during a 6-month period and (2) explore clinician perceptions of the interprofessional training implementation package using a mixed-methods approach. Evaluation outcomes will be used to inform further refinement of the COPD implementation package, with the goal of implementing the COPD CARE service for a national implementation effort to other VMACs.20
Implementation packages have been shown to be effective at promoting adoption of best practices into clinical care and are described in the field of dissemination and implementation (D&I) science,20–22 which promotes the use of theories, models and frameworks aimed at facilitating adoption of evidence-based practices across health settings.23 24 This evaluation applied a D&I knowledge translation framework called Replicating Effective Programmes (REP) to guide the implementation and evalution of the COPD CARE implementation package.22 The REP framework was originally developed by the US Centers for Disease Control and Prevention to promote expansion of HIV prevention interventions into practice and has since been applied to promote a diverse array of best practices across health systems.25
The REP framework was selected due to its emphasis on promoting learner self-efficacy, which is a critical predictor of future behaviour.26 Within the context of COPD CARE, the future behaviour desired is the adoption of evidence-based COPD best practices into the clinic. The REP framework is comprised of four phases, the first of which considers the local setting where implementation will occur (phase 1: preconditions), followed by design of an interprofessional training package (phase 2: preimplementation), implementation of COPD CARE training (phase 3: implementation) and evaluation of opportunities for further training refinement (phase 4: maintenance and evolution). Initial design of the virtual COPD CARE training occurred in 2017 and has since been revised using REP.20
This formative evaluation assesses the implementation package to facilitate uptake of COPD CARE at two VAMCs and five community-based outpatient clinics (CBOCs) using REP to guide training implementation. Clinician perceptions of training impact and fidelity check to evaluate training impact on effectiveness of implementing COPD CARE into clinical care were considered. Four online training modules were developed emphasising (1) Collaborative Care, (2) Patient Action Plan Review, (3) COPD Disease State Management and (4) Service Logistics. Each asynchronous training module was 60 min, and included videos and quick-guide references (eg, COPD clinical checklists, note templates and clinical decision support tools) to apply training during patient care delivery.
This evaluation was a prospective mixed-methods quality improvement project. Two PDCA cycles occurred over a 24-month period where the training methods were: (1) refined (2) implemented and (3) evaluated for impact on incorporation of best practices into primary care. This work began in October 2018 and ended in October 2020, during which 8 months were allocated to training refinement (phase 1), 4 months to training implementation (phase 2) and 12 months to evaluate service impact (phase 3). The first PDCA cycle began with the implementation of the COPD best practice training modules. The second PDCA cycle began after training completion and involved tracking adoption of COPD best practices and assessment of a clinician audit and feedback session.
An executive team was developed to initiate the project with the goal of refining training and strategizing an implementation plan. Team members included a project lead that designed the COPD CARE service, an implementation lead from a VAMCs interested in expanding the service to their facility, a training director engaged in clinical training development and support from a project manager and dissemination scientist. The evaluation was supported through the Department of Veterans Affairs Diffusion of Excellence Program, which strives to identify, diffuse and sustain best practices across the VA enterprise, prompting the opportunity to refine, implement and evaluate the current training programme at 2 VAMCs.15 The team held a 2-day kickoff and planning meeting in Orlando, Florida in October 2018, including training focused on application of D&I science strategies, project management and team-building activities.
Measurement
Primary objective
The primary aim was to evaluate the impact that training had on incorporation of COPD best practices into primary care delivery. Using data obtained from the VA electronic health record (EHR), best practices were grouped into critical training aspects and included COPD (1) patient referrals, specifically emphasising patient immunisations, tobacco cessation and pulmonary rehabilitation, (2) medication optimisation reflecting best practices in the Global Initiative for Chronic Obstructive Lung Disease 2020 guidelines, (3) patient empowerment, emphasising education provided during provision of a COPD Action Plan and (4) longitudinal follow-up care.
Secondary objective
The secondary aim was to explore clinician perceptions of the COPD implementation package through issuance of a Likert Scale-based questionnaire as well as review of qualitative content obtained during audit and feedback sessions with clinicians. Questionnaire scales emphasised clinician self-efficacy, attitudes, intent, among other aspects of the theory of planned behaviour as it relates to COPD management.27–30 Items were either modified from previously validated questionnaires to reflect COPD management or were adapted from Bandura’s ‘Guide for constructing self-efficacy scales’.31 The COPD CARE Skills Confidence questionnaire was designed by the evaluation team, while the theory of planned behaviour,29 Generalised Self-Efficacy Scale30 and Patient Engagement scale32 were modified from previously validated instruments to reflect COPD clinician training. A separate validated scale incorporating telehealth management, Telehealth Theory of Planned Behavior, was also issued as a measure of discriminant validity. Telehealth is unrelated to the current training and was anticipated not to change over time. Questionnaires were issued at different time points to quantify impact of the training package on clinician self-efficacy, attitudes and beliefs towards applying training content. PDCA cycle 1 was designed to allow for timely review of the implementation package, while PDCA cycle 2 was intended to assess longitudinal sustainability of the intervention.
Questions within each scale corresponded to a number, and scales were aggregated by summation per response. Scores were described by mean (SD) and median (IQR) at each timepoint and were compared using Mann-Whitney-Wilcoxon tests. The primary analysis was to determine if scores changed over time. Despite the multiple tests, no p value corrections were made to account for inflated type 1 error. Analysis was conducted using R V.4.0.2.33
Audit and feedback sessions were held with clinicians at the final stage of the second PDCA cycle. Data regarding programme outcomes were shared with clinicians including baseline characteristics of patients, COPD CARE service interventions and service impact on patient readmissions. Then clinician feedback was obtained on how training impacted clinician workflow, patient outcomes and overall clinician satisfaction with the training. We used the Braun and Clark34 thematic analysis approach to identify themes from these sessions.
Strategy
Phase I: preconditions, preimplementation (REP framework)
As part of the first step of the REP framework, the training to promote COPD management was refined to incorporate greater use of video examples illustrating in-clinic interventions as well as design of a workbook with quick-guide resources to be used while in clinic.22 These modifications were adopted to make the training more dynamic, which clinicians identified in our prior evaluation as an important learning tool for web-based training programmes.20
Phase II: training implementation (REP framework)
The training package was made available to clinicians in May 2018 and included asynchronous web-based training and synchronous virtual debrief sessions. Web-based training consisted of four modules each of which lasted 1 hour in length. Training was housed on a web-based platform, where all training content was uploaded to promote clinician understanding of service core elements. The blended learning approach that encompassed both web-based online modules and synchronic virtual debrief sessions was intended to promote application of content to the clinicians’ local practice setting.35
Two debrief sessions were held with three clinicians from each VAMC on completion of web-based training. This was the only aspect of implementation that differed across implementation sites. The VAMCs in the South Central region of the US preferred to have an on-site visit to debrief, which allowed for trainees to complete a simulated patient encounter led by six clinicians with experience in COPD management. The VAMC and associated clinics in the Midwest region of the USA elected to complete a debrief telephone call providing the opportunity for final questions to be addressed. The debrief call not only allowed for more rapid implementation of the training to practice but also required clinicians to enact training content without having a hands-on practice session. The final PDCA cycle, therefore, provided an opportunity to evaluate the importance of onsite simulations, and whether this aspect is critical to promoting clinician self-efficacy as compared with a telephone debrief discussion.
Phase III: maintenance and evolution (REP framework)
A mixed-methods approach was taken to evaluate training impact and opportunities for future refinement. A questionnaire issued prior to training, immediately following training completion, and 1 year after training emphasised clinician-reported self-efficacy as well as aspects of the theory of planned behaviour. EHR data were reviewed to evaluate how training influenced COPD care delivery during the 6-month period immediately following training completion, with data compared with the same 6-month period 1 year prior to training initiation.
An audit and feedback session with clinicians at the end of the final PDCA cycle was conducted. Data regarding programme outcomes were shared with clinicians including baseline characteristics of patients, COPD CARE service interventions and service impact on patient readmissions and clinician feedback was obtained to better understand how training influenced incorporation of COPD best practices into clinical care. A review of facility-specific strengths and opportunities to further enhance training was also discussed to enhance future training iterations. Transcribed recordings of each feedback session were analysed by five project team members using an inductive approach to coding to identify themes.
Results
This evaluation was conducted at two VAMCs that include five outpatient clinics across the Midwest and South Central regions of the USA. A total of 73 clinicians completed the initial questionnaire including a total of 26 (35.6%) clinical pharmacists and 35 (47.9%) RNs. Online supplemental table 1 provides complete demographic information. Twelve (16.5%) clinicians from other disciplines including respiratory therapists, triage nurses and clinic managers also completed the baseline questionnaire and training. Fourty-four (60.3%) clinicians had five or fewer years in practice and the remaining clinicians had greater than 5 years in practice. Over half (57.5%) of the clinicians had some prior experience with COPD management. Aggregate pretraining scores within each scale demonstrated a wide baseline variation in clinician experience supporting patient management of COPD as reflected by a large SD in self-efficacy, attitudes and preconceptions about incorporating COPD best practices into the clinic.
Supplemental material
Forty-four clinicians responded to the post-training questionnaire. Longitudinal improvements were observed in responses to questionnaire items from baseline to immediately after training completion, with significant improvements observed for all scales evaluating the COPD implementation package at the end of the final PDCA cycle (see figure 1). Scales included a modified Generalised Self-Efficacy scale to emphasise COPD management (p<0.001 immediate, p<0.001 1 year), self-efficacy of incorporating COPD best practices (p<0.001 immediate, p<0.001 1 year), theory of planned behaviour metrics (p=0.024 immediate, p=0.011 1 year) and confidence engaging patients in COPD management (p=0.069 immediate, p=0.003 1 year). Twenty-one clinicians completed the 12-month follow-up questionnaire. Improvements observed after training completion were sustained or further improved over the year following training completion. No changes were found in the Telehealth Theory of Planned Behaviour questionnaire (p>0.05), which was anticipated that given this scale was a measure of discriminate validity that was unrelated to training content.
Clinician questionnaire responses Clinician questionnaire responses collected after the first (pre-training to immediately post-training) and second (pre-training to 1 year post training) PDCA cycle. COPD, chronic obstructive pulmonary disease; PDCA, Plan Do Check Act.
Comparisons were made between VAMC at each time point for each of the five scales. No statistically significant differences were observed between implementation sites in the South Central region compared with those in the Midwest region of the USA with the exception of baseline average values for the Generalised Self-Efficacy scale (p=0.013) and Telehealth Theory of Planned Behavior (p<0.001). Key themes identified during audit and feedback sessions are included in table 1, with primary themes emphasising the interprofessional approach to training as well as confidence translating training content to practice. As one pharmacist stated, ‘PCPs are happy to have someone help with disease management… It’s helping to take off some of the stress of disease management’. Clinicians also emphasised the benefits of training resources, such as videos and quick-guides and how these materials helped them to deliver patient-centred care. One clinician commented, ‘I really like the (training) videos that were online. They were like a real appointment in a real setting… The in-person training just reinforced that’.
Themes identified from audit and feedback session
EHR data demonstrated significant improvements in the count of COPD best practices incorporated into routine clinical care after training completion (p<0.001, see figure 2). These improvements were observed across all critical training aspects, illustrating a widespread incorporation of COPD CARE practices into routine clinical care.
Enactment of clinical training to patient care. Data obtained from the electronic health record reflecting COPD clinician visits 6 months after training completion as compared with the same time period the year prior to training. Percentages reflect the occurrence of these events from 119 patient encounters post-training, and 130 patient encounters prior to training. COPD, chronic obstructive pulmonary disease.
Lessons and limitations
A limitation to this project was that it relied on retrospective EHR chart reviews to evaluate how training influenced care delivery. While it would have been beneficial to record clinic visits to better understand how changes in care delivery evolved, this also may have influenced how care was delivered as clinicians would be aware of the recording. There are several advantages to the approach we used, such as, use of the EHR allowed for a consistent approach to evaluate in-clinic implementation of best practices before and after training implementation. Also, the audit and feedback sessions provided another opportunity for clinicians to learn about programme outcomes and describe their perceptions of how training influenced clinical practice. The combination of qualitative input, along with quantitative data from the questionnaire and chart reviews, allowed for an opportunity to triangulate data and better understand why the training was associated with improved incorporation of COPD best practices.
An additional challenge was the asynchronous format of web-based training. While this approach allowed for clinician flexibility to view materials at their own pace, it also resulted in some clinicians requiring additional time for training completion. This resulted in a slight extension of the training period. As training is further revised and modified for expansion to additional VAMCs, it may be beneficial to dedicate a half-day of clinician time away from clinical responsibilities to view modules. This would also reduce the time needed to complete training and allow for a more streamlined approach to incorporating best practices. The web-based training, however, does allow for sites to elect what approach works best for their facilities and eliminates the geographical barriers to the executive team delivering training in-person.
Conclusion
Training successfully increased clinician self-efficacy as well as aspects of the theory of planned behaviour, with sustainment of improvements 1 year following training completion and less variability in clinician self-reported responses over time. Improvements in self-efficacy support use of the implementation package as an effective method to positively influence clinician perceptions toward COPD management.
Increases in clinician self-efficacy also translated to greater incorporation of COPD best practices in routine clinical care 6 months after training completion as compared with baseline. This significant increase across all fidelity metrics provides real-world context to how training programmes can contribute to greater incorporation of best practices. Qualitative data obtained from audit and feedback sessions 1 year after training completion further illustrate the positive perceptions of the training.
The training in this evaluation was designed to promote implementation of a care bundle, titled COPD CARE, into routine clinical practice by addressing deficits in COPD clinical knowledge and enhancing clinician self-efficacy for managing COPD. Similar barriers to implementing COPD care bundles into practice are described in the literature including challenges coordinating service implementation and training across service lines.36 The INSPIRED COPD Outreach Program, a COPD care transitions programme in Canada, describes lack of clinician training and experience as barriers to implement the programme across Canadian provinces.37 Furthermore, evaluators describe stakeholder relationships as a barrier when communication challenges existed. An article by Lennox and colleagues further explores barriers to implementation of COPD care bundles, with the five most significant challenges including: staff too busy, staff shortages, lack of staff engagement, added workload and patient coding challenges.38
This evaluation considers an implmentation package with an approach of offering external facilitation to build capacity and overcome barriers to service implementation using a coordinated, standardised, virtual approach to training an interprofessional team of clinicians. The blended learning approach taken to train with synchronous debriefs is designed to enhance trainee engagement through application of training content to the local setting where the COPD CARE bundle will be implemented. This approach of closely collaborating and problem-solving with trainees is termed Implementation Facilitation39 and is a D&I science strategy that has been used to promote adoption of trainings into routine clinical care. The use of Implementation Facilitation in this evaluation through collaborative dialogue with trainees after each training phase likely also promoted training uptake, as barriers and facilitators to training implementation were discussed and resolved during the training process.
Having the training package online helped overcome the barrier of staff being too busy to complete training content, as staff were able to complete training at their own pace. While extension of the service implementation date was necessary to accommodate clinician schedules, this allowed for clinicians to complete the training and participate in debrief sessions without compromising their patient care responsibilities.
There were no statistically significant differences across the implementation sites 1 year after training completion, indicating that the in-person 1:1 simulation may not have impacted long-term clinician perceptions of COPD management. Opportunities exist to further explore the short-term impact of these in-person simulations, as this was not considered as part of the current quality improvement work. It may be that the evaluation sites in different regions reported similar self-efficacy outcomes during the final PDCA cycle because both sites had applied training content to patient care over a 1-year period, thereby minimising the impact of a one-time simulation. While onsite simulations were viewed as beneficial by trainees in South Central region of the USA during the audit and feedback session, these in-person simulations may not be critical to long-term sustainment of applying training to clinical practice.
The intentional incorporation of a D&I scientist within the executive team served as a strength of this project. Incorporation of REP as a framework to inform training design reinforced the importance of developing training materials and approaches for training delivery that fit the unique workflow, culture and values of the local setting. The team strived to consider in the initial phases of training design how to develop a curriculum that allows for remote delivery, timely virtual discussions and opportunities to consider how adaptations can be made to enhance adoption of COPD best practices. The REP framework provided a step-by-step approach to consider these local contextual factors throughout this process.
Future directions include expansion of the COPD implementation package to incorporate resources for clinical managers leading the process of implementing the COPD CARE bundle. Through this evaluation, it was learnt that there are logistical barriers to implementing the COPD CARE service, including coordinating a patient referral process, integrating note templates and capturing patient outcomes. These logistical components are critical to service implementation and require additional resources to promote efficient service adoption. A similar approach to design this training content and delivery, leveraging REP as a guiding framework and incorporating a supportive implementation strategy such as facilitation39 are opportunities for future expansion of the service. The next iteration of COPD CARE implementation package will, therefore, expand beyond clinician-focused training to provide a more holistic approach to expansion of the COPD bundle.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. Not Applicable.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants but University of Wisconsin-Madison Institutional Review Board exempted this study. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to acknowledge the Department of Veterans Affairs Diffusion of Excellence Program for providing such exceptional mentorship and programming to front-line clinicians engaged in promising quality improvement initiatives. We also would like to acknowledge the University of Wisconsin-Madison Institute for Clinical and Translational Research (ICTR) for their support with statistical analysis during this evaluation.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors EP served as the guarator and accepts full responsibility for the overall work. EP supervised the project, data collection, and analysis, as well as drafted and revised the paper. EP, ML and TH implemented the implementation package at each site and guided the evaluation. MM reviewed and revised the paper. JK, SB and NV collected qualitative data and conducted qualitative thematic analysis, as well as contributed to the writing and revisions of the paper. JB conducted the quantitative statistical analysis and drafted portions of the paper. MAC attended regular meetings with EP to provide guidance and mentorship regarding the direction of the project and revised the paper. DG provided dissemination and implementation science expertise to the evaluation and revised the paper. MM and BH provided guidance throughout the implementation and evaluation and revised the paper.
Funding This study was supported by the US Department of Veterans Affairs Office of Rural Health and the Clinical Pharmacy Practice Office of the Pharmacy Benefits Management Service, though the Office of Rural Health’s Enterprise-Wide Initiative, PROG-0000104 and the University of Wisconsin Institute for Clinical and Translational Research, which is supported by the Clinical and Translational Science Award (CTSA) program, the National Center for Advancing Translational Sciences (NCATS), grant UL1TR002373 - KL2TR002374.
Competing interests There are no direct competing interests as it relates to this manuscript. Dr. EP has served as a consultant for AstraZeneca.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.