Article Text
PDF
Abstract
In 2019, the Indianapolis VA developed a Wellness Clinic in partnership with the Young Men’s Christian Associations (YMCA) to comprehensively address Veterans’ chronic pain. Our specific aims were twofold: (1) to evaluate the implementation of the Veterans Health Indiana (VHI) Wellness Clinic on patient utilisation and (2) to evaluate patient functioning.
We conducted a mixed-methods evaluation, which included the extraction of VA administrative data to identify a patient cohort; the conduct of chart review to extract clinic utilisation, clinical outcomes collected during pain-related healthcare services and comorbidities; and semistructured interviews with Veteran patients who used the VHI Wellness Clinic in different patterns to identify challenges and facilitators to clinic utilisation. We applied configurational analysis to a Veteran sample who had their first visit to the VHI Wellness Clinic in March/April 2019 to pinpoint difference-making factors linked to Veterans’ successful participation.
The cohort included 312 Veterans (83% male), mean age of 55.4 years. The configurational model included six factors: participation in physical therapy, pain psychology or pain education sessions (22%); presence of any ‘no-shows’ (57% had 0); history of depression (39%) and clinic referral source (51% self-referred from primary care). The model consisted of four different pathways to successful participation, explaining 60% of cases in the higher-participation group with 86% consistency. Patient outcomes after clinic utilisation demonstrated a significant reduction in self-reported pain and pain catastrophising across time. Moreover, patients reported distance to clinic as both a facilitator and challenge.
This mixed-methods analysis identified specific biopsychosocial factors and clinical services directly linked to higher Veteran participation in a new VA-YMCA Wellness Clinic. The VHI Wellness Clinic embedded within a YMCA facility is a feasible and efficacious healthcare delivery model for primary care patients experiencing chronic pain. Additional marketing to clinical providers for referrals and to patients to extend its reach is needed.
- Pain
- Implementation science
- Quality improvement
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Chronic pain remains a prevalent common condition in the USA, affecting 50.2 million adults with Veterans disproportionally affected. Overall, 1 in 3 Veterans has a diagnosis with chronic pain or similar condition; 1 in 5 describes persistent pain and 1 in 10 reports severe persistent pain.
WHAT THIS STUDY ADDS
An innovative and comprehensive VA Wellness Clinic was developed and embedded inside a Young Men’s Christian Associations fitness facility in the community to extend reach to Veterans living with chronic pain. Patient factors that contributed to successful participation in the clinic were identified by configurational analysis and supported by patient interviews.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Direct-to-consumer implementation strategies for interorganisational clinics located outside of traditional healthcare facilities can help target patient groups successfully participate in these non-traditional programmes.
Background
Chronic pain remains a prevalent common condition in the USA, affecting 50.2 million adults.1 Veterans are disproportionally affected,2 making chronic pain especially salient and a top priority for the Veterans Health Administration (VHA), the country’s largest integrated healthcare system: 1 in 3 Veterans has a diagnosis with chronic pain or similar condition; 1 in 5 describes persistent pain and 1 in 10 reports severe persistent pain.
In response to this challenge, the national VHA has expanded its traditional set of treatment options to combat chronic pain among Veterans to include new biopsychosocial approaches as part of the innovative VHA Whole Health Initiative.3 One of the main goals of the VHA Whole Health Initiative is to encourage patients to engage with their own healthcare and the inclusion of various complementary and integrative health programmes such as acupuncture, meditation and massage therapy. Recently, VHA has integrated a Whole Health Programme representative within the pain teams established in primary care to address Veterans chronic pain.4 In step with this larger national movement within VHA, the Roudebush VA Medical Center (VAMC) entered a local interorganisational partnership with the Young Men’s Christian Associations (YMCA). This partnership began in early 2019 with the opening of a brand new, Veterans Health Indiana (VHI) Wellness Clinic within the OrthoIndy YMCA, a newly built YMCA facility in Indianapolis, Indiana.
The YMCA of the USA (Y-USA) consists of independent and autonomous member associations often referred to as local YMCAs. Each local YMCA is dedicated to the overarching Y-USA goal of strengthening their community through youth development, healthy living and social responsibility. In partnering with the Roudebush VAMC, both entities supported the shared goal of improving the health and well-being of local Veterans and their families by offering programming, services, resources and space provided by the OrthoIndy YMCA.
The VHI Wellness Clinic is the result of the partnership between Y-USA and the Roudebush VAMC with the aim of empowering Veterans with alternative ways to manage chronic pain effectively. The clinic services include the provision of holistic treatment for chronic pain. For example, some services offered at the VHI Wellness Clinic were pain psychology, physical therapy, mindfulness for insomnia, chair yoga and Tai Chi, similar to the programmes laid out by the Whole Health Initiative. The specific aim of this project was to evaluate the implementation of the innovative VHI Wellness Clinic on patient utilisation of clinic services beginning in March and April 2019 and their prescores and postscores for functioning scales extracted from patient medical records (see table 1 for a full description of these scales). More specifically, we evaluated which conditions were linked to higher levels of engagement by Veteran patients with chronic pain within the VHI Wellness Clinic. The project included data for the primary outcome of patient utilisation of clinic services and data for the secondary outcome of patient results of functional scales collected from patient’s electronic records for evaluation purposes using assessments that are part of routine care. Key informant and stakeholder interviews provided insight about the implementation of the VHI Wellness Clinic.
Changes in patient outcomes—patient pain-related functional outcomes at baseline and postclinic visits
Methods
Patient and public involvement
This project was deemed to be quality improvement by the clinical director of the Roudebush VAMC Pain Clinic (see online supplemental appendix 1 for non-research declaration). This manuscript adheres to the Revised Standards for Quality Improvement Reporting Excellence reporting guidelines.
Supplemental material
The newly built YMCA was opened in December 2018 and the VHI Wellness Clinic began its services to Veteran patients in January 2019. To evaluate the implementation of the VHI Wellness Clinic services, we employed a mixed-methods evaluation of data including: (1) administrative healthcare data, (2) patient medical chart review and (3) qualitative interviews with patients. To allow time for the clinic to establish its practices, we evaluated data for patients who began attending services at the VHI Wellness Clinic in March and April 2019.
Administrative data
First, we obtained administrative data to identify a cohort of unique patients (with or without a prior PCP visit) who completed a visit during the timeframe between March and April 2019 to either the:
Roudebush VAMC chronic pain clinic (required if consumption >90 mg of morphine or equivalent).
VHI Wellness Clinic.
For each patient with at least one visit as noted above, we extracted the following from the VA Corporate Data Warehouse (CDW) administrative data: age, race, gender, zip code and diagnoses on the medical problem list.5 We used ICD-10 codes to calculate a Charlson Comorbidity Index score.6 We extracted all Pain and VHI Wellness Clinic visits, specifying the visit type, location and date, as well as provider name, listed name of clinic services in the patient charting system, and origin of the initial clinic consultation. From the VA CDW pharmacy files, we extracted the list of pain medications and doses within 100 days of the first clinic visit and at the time of the last clinic visit.
Medical chart review abstraction
We also conducted a VA medical chart review on the identified cohort to collect data on which clinic services each patient used as well as their prescores and postscores for the selected functional scales and collected as part of routine clinical care. We conducted spot validation on the administrative data to ensure clinic stop codes and patient chart documentation reflected the underlying service provided. The data collected from patients’ charts included those listed from each administrative data source, as well as: geographical information system—urban, rural or highly rural designations for each patient’s home address,7 date of the first consult request, service accessed on first visit, most recent appointment date and which service was accessed, total number of visits to the clinic (not including no-shows or phone calls), number of VHI Wellness Clinic notes in chart, number of no-shows for appointments, number of times patient has been reviewed by the VHI Wellness Clinic team conference (identified by the number of team conference notes in chart), number of telephone notes (from Chronic Pain Clinic or VHI Wellness Clinic) in chart, services patients accessed (Chronic Pain Clinic at VAMC; Pain Psychology; VHI Wellness Clinic social worker; Physical Therapy; Chiropractor; Yoga; Tai Chi; MOVE! Weight Management; Be Active and Move (BAM); iRest Meditation for Sleep; Acupuncture; Pain Education including sleep hygiene education and opioid education), patient data from various pain and function tools, opioid use and level of usage during first and most recent visit to VHI Wellness Clinic, patient history of depression, patient history of anxiety, patient history of substance abuse and patient history of post-traumatic stress disorder (PTSD) as collected in the medical chart as part of routine medical care.
The pain and function tools extracted from patients’ charts were as follows: Pain Self-Efficacy Questionnaire8; Pain Catastrophising Scale (PCS)9; Functional Assessment Interview10; Generalised Anxiety Disorder Assessment-711; Patient Health Questionnaire-912 and the Pain, Enjoyment of Life and General Activity Scale (PEG-3).13 These tools were collected across time as a ‘baseline’ and again as ‘postclinic’. The baseline data were patients’ results from the start of 2018. Postclinic data were patients’ most recent results at the time of data collection. Patients’ data included number of times each tool had been administered between 2018 and 2019, which service at the VAMC first administered tool in 2018, numeric results of first tool, date of first tool administration, numeric result of latest tool, date of latest tool administration and which service at the VAMC first administered tool most recently.
Qualitative data
Finally, to inform the future implementation of the VHI Wellness Clinic, we conducted interviews with Veterans who attended appointments at the VHI Wellness Clinic. Veterans were categorised as the following:
Type 1: Veterans who attended five or more VHI Wellness Clinic appointments or successfully completed a programme.
Type 2: Veterans who attended multiple VHI Wellness Clinic appointments but were discharged due to no-shows.
Type 3: Veterans who attended only an initial Wellness Clinic appointment before unenrolling or no-showing.
Type 4: Veterans who had a scheduled VHI Wellness Clinic appointment but never attended.
Type 5: Veterans who did not attend their initial appointment but later attended a VHI Wellness Clinic appointment in May–October 2019.
We worked with clinic leadership to develop an interview guide for Veterans to gain an understanding of how they chose to use services at the VHI Wellness Clinic and barriers that interfered with clinic utilisation.
We identified eight Veterans to conduct semistructured interviews with including four type 1 and four type 2 Veterans to understand patient barriers and facilitators to engage in VHI Wellness Clinic services. Veterans were selected for interview by visit type and gender. Male Veterans were selected at random by every 10th patient within each visit type category, while a purposeful sample of the female Veterans was selected to include more female Veterans for interviewing. We purposefully chose types 1 and 2 participants to understand participation in the clinic from those who attended all sessions and those who at least attended some visits to have a reference to discuss why they chose to stop attending the clinic. We conducted rapid qualitative analyses using the interview notes.14
Configurational analysis
We used configurational analysis to look across all individual cases and identify the crucial conditions that distinguished Veterans who had ‘higher’ clinic participation from those who did not. Configurational analysis searches for necessary and sufficient conditions for an outcome to appear and can detect causal complexity (when several conditions must appear together for an outcome to occur) as well as equifinality (when multiple pathways lead to the same outcome). For this analysis, we specifically used coincidence analysis (CNA), a relatively new member of the larger family of configurational comparative methods that has been gaining traction in health services research and implementation science, with over 50 articles published in the peer-reviewed literature since 2020.15 16
Overall, there were 85 different factors in the original dataset. The primary outcome for the configurational analysis was ‘higher’ clinic participation, defined as Veterans who (a) attended at least five appointments or (b) successfully completed a programme. To aid with factor selection, we began by substantively reviewing the complete set of factors and identifying 43 factors with a theoretically plausible connection to the ‘high clinic participation’ outcome. See online supplemental table 1 for a full list of these 43 factors.
Supplemental material
Next, using this dataset of 43 factors, we conducted an exploratory data analysis and applied the ‘minimally sufficient conditions’ (‘msc’) function within the R package ‘cna’ (39) to analyse the dataset inductively and identify configurations of conditions with particularly strong connections to the outcome (ie, high clinic participation). This configurational approach to data reduction has been described in detail earlier.17 18 To summarise, we considered all one-condition, two-condition and three-condition configurations that met predesignated thresholds for consistency and coverage. Consistency indicates how reliably a model yields an outcome and is calculated as the number of cases covered by the model that also have the outcome present divided by the total number of cases covered by the model. Coverage relates to explanatory power and is calculated as the number of cases covered by the configuration that also have the outcome present divided by the total number of cases that have the outcome. We began with a consistency threshold of 100% and a coverage threshold of at least 15%. If no configurations met this dual threshold, we iteratively lowered the specified consistency level by 5 points (eg, from 100% to 95%) and repeated the process. We continued lowering the consistency threshold until there were at least two potential configurations that met the specified consistency and coverage thresholds. We then assessed all configurations that satisfied those thresholds.
Using this approach, we identified a smaller subset of factors to use in the model-developing phase of the analysis. We developed models by iteratively using model-building functions within the ‘cna’ software package in R (V.39). We assessed models based on their overall consistency and coverage, as well as potential model ambiguity (when competing models explain the outcome equally well based on consistency and coverage scores). After a preliminary model was identified, we optimised coverage by reviewing the condition table to consider additional configurations that met consistency and coverage thresholds for facilities with higher impact that were not explained by models developed thus far. Our final model met an overall consistency threshold of ≥75%, a coverage threshold of ≥60% and had no model ambiguity. The CNA package (‘cna’) in R (V.39), R (V.3.5.0), R Studio (V.1.1.383) and Microsoft Excel were used in analyses.
Results
Patient characteristics
Data were collected on 314 Veterans of which 260 were male and 54 were female. The mean age at first appointment was 55.4 years old (SD=15.0 years) and the age ranged between 25 and 88. A number of patients were currently prescribed opioids as they initially engaged services at the VHI Wellness Clinic. At their first visit to the VHI Wellness Clinic, 38 patients were on prescribed opioids. At the most recent visit, 35 patients were on prescribed opioids. Among the 35 patients on prescribed opioids at their most recent visit, about one-third (n=13) were on a ‘maintenance’ level of opioid dosage. The patients also had diagnosed mental health comorbidities of which 122 patients had a history of depression, 73 had a history of anxiety and 57 had a history of PTSD. Table 2 displays the patient demographics, source of referral to the VHI Wellness Clinic and describes patient characteristics.
Patient demographics of 314 Veterans
Veterans used a variety of services at the VHI Wellness Clinic. Figure 1 shows the number and percentage of Veterans participating in each service within the VHI Wellness Clinic. Veterans could participate in more than one service, so the cumulative total exceeds the total number of patients. The cohort completed 448 classes including the following: 148 attended physical therapy; 69 attended pain education; 46 attended MOVE Weight Loss; 38 attended pain psych; 31 attended chiropractor appointments; 29 attended Tai Chi; 21 attended social work appointments; 18 attended yoga; 16 attended acupuncture; 15 attended recreational therapy; 9 attended iRest meditation sessions; 7 attended marriage/family therapy and 1 attended BAM.
Number of Veterans participating in each service within the VHI Wellness Clinic.
Table 3 shows the distribution of patient visit types to the VHI Wellness Clinic. Over half of our sample attended 5 or more sessions or completed the programme in full: 161 Veteran patients met criteria for visit type 1. Some patients attended the clinic sessions and then no-showed: 54 were of visit type 2, whereas 57 were of visit type 3. Some never showed: 33 were of visit type 4. A few never showed initially but attended the clinic in later months: 5 were of visit type 5. There were four patients who had missing data in their chart for visit types.
Patient Veteran Health Indiana (VHI) visit types
Patient outcomes
Changes in patient outcomes collected during usual clinical care are shown in table 1. A post hoc comparison of means shows that patients reported significant reductions in their degree of pain catastrophising and level of self-reported pain between baseline and postclinic clinical assessments PCS (p=0.002) and PEG-3 (p=0.049). Pain self-efficacy, functional status, anxiety and depressive symptoms remained steady over time.
Configurational analysis
A total of 286 Veterans had sufficiently complete data to be included in the configurational analysis. Of these, there was a roughly even split with respect to the outcome of interest: 146 Veterans had met the criteria for high clinic participation, whereas 140 did not.
The MSC routine identified a subset of seven candidate factors to include in the modelling phase: physical therapy, pain education, pain psychology, number of no-shows, history of depression, referral source: self-referral or referral source: primary care. The modelling phase identified a configurational solution for high clinic participation that met all prespecified requirements related to consistency, coverage and model ambiguity (see figure 2). The model had four distinct solution pathways:
Physical therapy and pain education.
Physical therapy and pain psychology
Physical therapy, zero no-shows and no history of depression.
Self-referral or from primary care, zero no-shows and no history of depression.
Configurational Model of Patient Utilization of the VHI Wellness Clinic
Physical therapy was the service that appeared most often across the pathways, but it was not sufficient on its own to explain ‘high clinic participation’; in each instance, it had to appear jointly with one or two additional discrete conditions. Likewise, the combination of zero no-shows and no history of depression on the part of the Veteran was a solution component that appeared in two of four pathways but was not sufficient by itself, as it had to conjoin with either physical therapy or either self-referral or primary care referral.
The combination of participating in the physical therapy and pain psychology services was the most consistent pathway: the 22 Veterans with these 2 conditions jointly present in the dataset also had the outcome of high clinic participation with perfect consistency (22/22 or 100%). This was followed closely by the combination of physical therapy and pain education, where consistency was also very high (25/27 or 93%). The remaining two pathways also featured high consistency scores, with the overall model featuring a consistency score of 86%. The overall model accounted for 88 of the 146 Veterans with the higher clinic participation outcome, or 60%.
Qualitative interviews
Building from our configurational models, we further expanded on Veterans’ challenges as well as facilitators to utilisation of the VHI Wellness Clinic services through patient interviews with a sample of the cohort. Veterans expressed (1) clinic location (facilitator), (2) provider communication (facilitator), (3) experience of pain or pain relief (barrier/facilitator), (4) activity difficulty (barrier) and (5) number of clinic sessions (barrier). Patients who found the clinic location convenient commented:
‘The [VHI Wellness] clinic was closer to my home than the VA [Medical Center].’ (ID 1096).
Patients’ comments on provider communication reported on attentiveness and delving deeper into patients’ conditions:
‘I knew the [chiropractic adjustments] were short-term, but they made me feel good there for a while and like I said, the location was a positive, also. … I knew I had some injuries and he finally defined what was going on and I got a better understanding on why my neck was hurting and why my back was hurting. … We had an extensive conversation about the problems I was having with my neck and back.’ (ID 1052); and
‘[The physical therapist] gave me areas of where my muscle tension was coming from and my condition. She showed me where to work on my pressure points and ease the tension.’ (ID 1186)
Patients perceived that the clinicians were good at communicating what was being done during appointments and giving resources for managing patients’ conditions at home:
‘The physical therapist was pretty good and attentive. We made it through the exercises really well. … [She was good at] explaining what she would do and exercises we were going to do. … She would print out what I needed and explain what she printed out.’ (ID 1167); and
‘Just everything [went well] from what we were going to be doing in the session to what [the physical therapist] was doing at the time and down to the science of why she was doing it. That to me was very beneficial and shows that she was very knowledgeable.’ (ID 1267)
‘[The physical therapist] helped my back quite a bit. … He gave me things I could do to stretch my back at home too.’ (ID 1052)
Some patients reported experiencing challenges to their utilisation of the VHI Wellness Clinic. One Veteran reported experiencing pain during his physical therapy session and that made participation challenging and others reported some of the physical activities were challenging (eg, yoga). Another patient reported being disappointed by the limited number of treatment sessions available as the Veteran wished to extend participation and another discussed that the distance to the YMCA facility which housed the clinic was a challenge as there were other YMCA locations within the Indianapolis area closer to home.
Nonetheless, Veteran patients from both types agreed that the VHI Wellness Clinic offered good programming that they were interested in, and these services were provided by competent, friendly clinical staff who provided excellent patient communication.
Discussion
The innovative VHI Wellness Clinic for Veteran patients with pain embedded into a local YMCA facility is a feasible and efficacious programme for primary care patients with chronic pain. For the patients in the VAMC catchment area who lived nearby this clinic, they enjoyed the convenience of the location in addition to the programming for which they wanted to continue as part of VA healthcare services. While the distance was a challenge to utilisation for those living away from the clinic, patients expressed that they would have liked the convenience of attending similar services at a YMCA location closer to home. Based on patient interview data which discussed both the convenience of living close to the YMCA clinic and the barrier from living further away, it appears that these pain programmes may see similar utilisation if they were housed in a VAMC. However, then the convenience would shift to patients who live close by the VAMC which is often in an urban location compared with a suburban location as the Roudebush VA Wellness Clinic in the YMCA.
In a similar hospital system/YMCA partnership, a successful Diabetes Prevention Programme integrated into the YMCAs in the Greater Cleveland area of Northeast Ohio was shown to be effective and then scaled up to a state-wide partnership.19 Based on the parallels between our current study and previous work, the successful implementation of the VHI Wellness Clinic within a YMCA facility demonstrates the potential to scale up and spread across nationwide YMCA programmes. Moreover, VHA has recently integrated a VA Whole Health representative within primary care pain management teams nationwide, increasing referrals to wellness-type programming similar to the wellness programmes at the VHI YMCA Wellness clinic for Veteran patients with chronic pain. Providing access to wellness programmes is important as Veterans with chronic pain and PTSDs have repeatedly expressed interest in such wellness programmes.4 20
In this configurational analysis, there was no ‘magic bullet’ condition or ‘one-size-fits-all’ model to explain why certain Veterans had high levels of clinic participation. Rather, the outcome of high clinic participation was directly linked to combinations of specific conditions, and there were four distinct pathways to the outcome.
A history of depression may be implicated in lower clinic engagement, suggesting that different populations of Veterans may benefit from tailored outreach and support. In particular, the combination of any ‘no-show’ visit with a history of depression appeared to serve as an early warning sign of Veterans being at higher risk of programme non-completion.
Veteran participation in physical therapy may provide a successful ‘entry pathway’ to engage more fully in the clinic.
Veteran engagement appeared to be a self-fulfilling prophecy: self-referred patients (with no history of depression) appeared in one of the four pathways.
Referrals from sources other than self-referral and primary care did not appear in the configurational model, indicating an improvement opportunity to increase Veteran engagement when referrals come from other specialties like neurology, sleep medicine or emergency medicine.
In previous research, patients who self-refer to care also report greater recurrent pain but report less pain disability. Patients who self-refer to pain-related healthcare programmes may differ than those who are referred to a pain-related treatment programme by a clinician. The self-referred patients may be more motivated to engage in treatment to relieve symptoms compared with patients who are referred by staff.21 In our study, the patient self-referral was present in one of the four pathways. These results suggest that implementing direct-to-patient consumer strategies may be fruitful for engaging patients in pain-related treatment programmes.
In terms of limitations, while the configurational model covered 60% of the Veterans with high clinic participation, the remaining 40% of Veterans with the outcome remained unexplained, suggesting a role for other factors not included in the dataset. This study was conducted at a single site and within the VHS system and may not generalise to other patient populations and other healthcare systems.
More work is needed on understanding patient pathways to and through the VHI Wellness Clinic. Based on the current results of this evaluation, recommendations on first offering patients services such as physical therapy for initial participation in the clinic and then informing them of other services may be an effective way to encouraging high participation. Furthermore, our in-depth qualitative interviews with Veterans who used the VHI Wellness Clinic suggested location of the clinic, friendly and knowledgeable staff with excellent communication skills and experiencing pain relief after treatment sessions facilitated Wellness Clinic utilisation. Distance from home to the clinic, pain experienced during treatment sessions and difficulty with treatment activities were challenges experienced by Veterans during clinic utilisation. Pain and depression symptoms are often reported as a barrier for patients to overcome to participate in external activities across clinical conditions.22–24
Next steps include additional planning and development of referral networks among the clinical staff and direct to consumer marketing communications to reach a catchment of Veterans with chronic pain and those with a history of pain and depression. Finally, the Indianapolis metropolitan area is composed of a series of YMCA facilities which provides a network to expand this feasible pain treatment delivery model.
Conclusions
The VHI Wellness Clinic embedded within a YMCA facility is a feasible and efficacious healthcare delivery model for primary care patients experiencing chronic pain. Configurational analysis provides an analytical approach for exploring and understanding complex phenomena related to implementation outcomes. Six factors consistently distinguished Veterans with high participation and completion rates. This mixed-methods approach generated new findings and insights into who kept ‘coming back for more’ in a YMCA-based Wellness Clinic.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Acknowledgments
We appreciate the assistance of Samantha Riley, MA in assisting in conducting qualitative interviews for this data collection
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors AKP analysed administrative data, conducted qualitative interviews, analysed qualitative data and was a contributor in writing the manuscript. CED reviewed administrative data, conducted qualitative interviews and was a contributor in writing the manuscript. EJM conducted configurational analysis, interpreted results and was a major contributor in writing results of the data analysis. LJM pulled administrative data for patient data. LSW and TMD designed the evaluation, advised on data collection and analysis, and were contributors in writing the manuscript. TMD accepts full responsibility for the work and/or the conduct of the quality improvement project, had access to the data, and controlled the decision to publish. All authors read and approved the final manuscript.
Funding This evaluation was supported by VA Health Services Research and Development Quality Enhancement Research Initiative, EXTEND QUERI (Expanding eXpertise Through E-health Network Development; QUE 20-010) and in part, a VA HSR&D Career Research Scientist Award to TMD (RCS 19-002).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.