Article Text

Download PDFPDF

Longitudinal, multidisciplinary, resident-driven intervention to increase immunisation rates for Medicaid, low-income and uninsured patients
  1. Cynthia Cantu1,
  2. Kristopher Koch1,
  3. Ramon S Cancino2
  1. 1Department of Medicine, University of Texas Health San Antonio, San Antonio, Texas, United States
  2. 2Department of Family and Community Medicine, University of Texas Health San Antonio, San Antonio, Texas, United States
  1. Correspondence to Dr Cynthia Cantu; CLCANTU5{at}GMAIL.COM


Introduction More payers are closely linking reimbursement to high-value care outcomes such as immunisation rates. Despite this, there remain high rates of pneumonia and influenza-related hospitalisations generating hospital expenditures as high as $11 000 per hospitalisation. Vaccinating the public is an integral part of preventing poor health and utilisation outcomes and is particularly relevant to high-risk patients. As part of a multidisciplinary effort between family and internal medicine residency programmes, our goal was to improve vaccination rates to an average of 76% of eligible Medicaid, low-income and uninsured (MLIU) patients at an academic primary care practice.

Methods The quality improvement project was completed over 3 months by three primary care resident groups. The setting was a suburban academic primary care practice and eligible patients were 18 years of age or older. Our aim was to increase immunisation rates of pneumococcal, influenza, varicella, herpes zoster virus and tetanus and diphtheria vaccination. There were 1690 patients eligible for the vaccination composite metric. Data were derived from the electronic health record and administrative data.

Interventions Cohort 1 developed an initial intervention that consisted of a vaccine questionnaire for patients to complete while in the waiting room. Cohort 2 modified questionnaire after reviewing results from initial intervention. Cohort 3 recommended elimination of questionnaire and implementation of a bundled intervention approach.

Results There were minimal improvements in patient immunisation rates after using a patient-directed paper questionnaire. After implementation of multiple interventions via an improvement bundle, there were improvements in immunisation rates which were sustained and the result of special cause variation.

Conclusion A key to improving immunisation rates for MLIU patients in this clinic was developing relationships with faculty and staff stakeholders. We received feedback from all the medical staff and then applied it to the interventions and made an impact in the average of vaccinations.

  • ambulatory care
  • healthcare quality improvement
  • medical education
  • patient education
  • primary care

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


Nationally, public and commercial healthcare models are evolving towards payment models which incentivise performance on health-related quality and utilisation metrics in addition to productivity. Despite wide availability of vaccines, adult vaccination rates remain low. Medicaid, low-income and uninsured (MLIU) patients are at most risk for poor health-related outcomes. In a retrospective database analysis of 154 381 adult discharges between the ages 18 and 64 years that were hospitalised with either pneumonia, acute myocardial infarction and stroke, Medicaid patients hospitalised with pneumonia had increased mortality compared with privately insured.1 Additionally, there are very few published articles demonstrating vaccination effort outcomes for MLIU patients specifically. Medical providers play an important role in providing vaccine recommendations for patients.2 3

A retrospective observational study published in April 2018 in BMC Geriatrics reviewed 1 949 352 individuals with Medicare Advantage insurance plan hospitalised during 2014–2015 for community-acquired pneumonia, myocardial infarction, stroke and osteoporotic fractures in adults aged 65 and older.2 This study found higher hospitalisation expenditures for pneumonia among the four diseases and it had the lowest preventive efforts which include pneumococcal and influenza vaccination as recommended by the Infectious Diseases Society of America and the American Thoracic Society.2 Only 7.6% had received pneumococcal vaccination in 2014.2 According to the Centers for Disease Control and Prevention (CDC), pneumococcal vaccination rates in the USA in 2017 was 69% for adults age 65 or older.3 Rates of vaccination are also higher among white adults compared with adults of other racial/ethnic groups.3

There are other vaccine preventable diseases such as herpes zoster (HZ), tetanus and influenza that also contribute to the healthcare economic hospitalisation burden. Poor influenza vaccination status among economically disadvantaged populations is a preventable factor contributing to the US economic burden.4 Lack of access to healthcare and language barriers are among some of the reasons that impedes many of these patients to receive influenza vaccination.4 A retrospective cohort study compared economic costs between HZ complications related hospitalisations to uncomplicated in a 5-year period among immunocompetent patients.5 Overall, resource utilisation and costs were higher for those with non-postherpetic neuralgia (PHN)-related complications (cutaneous, neurological and ophthalmic).5 HZ-associated complications including chronic pain, neurologic and ophthalmic-related hospitalisations impact the economy and cause undue distress in a patient’s quality of life,6 thus reinforcing the importance of vaccination. Other common obstacles faced at improving vaccination rates include provider’s knowledge of CDC vaccination recommendations, associated costs and lack of available vaccine registries.

Given these findings and known effectiveness of vaccinations, internal and family medicine residents sought to create and develop a patient-centred, staff friendly intervention to increase clinic vaccination rates. Additionally, as primary care delivery models transform, primary care residents need exposure to and involvement in transforming care delivery models.

Specific aim

We sought to increase the vaccination rates of pneumococcal, influenza, varicella, HZ virus and tetanus and diphtheria vaccination to an average of 76% by October 2019 at an academic primary care practice.


Study population

The patient population included patients seen at one primary care practice affiliated with the University of Texas Health San Antonio. Eligible patients were 18 years of age or older with Medicaid insurance or who were low income or uninsured. Patients were not included if they met vaccination requirements based on age and CDC recommendations in 2018 as follows: one tetanus and diphtheria toxoids (Td) vaccine in the last 10 years, two doses of varicella or history of disease up to year 1995, one influenza within last year, one pneumococcal polysaccharide vaccine (PPSV23) for patients 65 and older and one HZ vaccination for patients 60 years and older.7 Patients with specific medical conditions (alcoholism, chronic heart disease, chronic liver disease, chronic lung disease, cigarette smoking and diabetes mellitus) were also considered eligible to receive PPSV23 before age 65, as recommended by the CDC.7 Self-report regarding their vaccination history was counted.

Clinical site

The intervention site for the QI project was one academic primary care location. Key stakeholders at this location included five physicians, one nurse practitioner, five medical assistants, one licensed vocational nurse and one practice manager.

Context: Family medicine and internal medicine resident quality improvement (QI) project

This institution has ACGME accredited family medicine and internal medicine residency training programmes. In an effort to increase resident education on the topics of value-based care and physician leadership, a 1-month rotation was created where combined cohorts of family and internal medicine residents meet with primary care leaders, attend lectures, journal clubs and complete a component of a longitudinal QI project. Lean management tools were used in the QI project.8 The Revised Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines were used to format the manuscript.9 Over 3 months, three resident cohorts worked longitudinally to improve immunisation rates. The cohorts determined the scope and approach of the QI project, met regularly to review data, and were given feedback by faculty.


Cohort 1 (2/7/2018–29/7/2018): current state assessment and literature review

Cohort 1 assessed a list of performance metrics for the primary care practice. The group identified that the location performed less well on certain immunisation metrics when compared with other primary care locations. The residents selected improving immunisation completion rates as their QI project. They first completed a literature review on the impact of poor immunisation status on hospitalisations and patient outcomes in the USA. Residents shadowed key stakeholders to observe workflows, including the current immunisation review and administration processes. The residents obtained feedback from stakeholders about the challenges encountered during these processes. The residents met three times a week with faculty to discuss their findings. They learnt various factors affected processes including staff unfamiliarity with vaccination schedule, lack of standing orders, confusion on tasks, and difficulty accessing the online state vaccination registry. The residents created a current state process map. The process map was reviewed by all stakeholders, and, after gathering input from all, an ideal state process map was created (figure 1). Stakeholder input focused on the limited time staff had to update a patient’s vaccination history during the intake process. Therefore, Cohort 1’s intervention was the development of a self-administered questionnaire that patients complete while in the waiting room. The questionnaire asked patients to enter their previous immunisation history as well as their age and sex. Sex was included to determine whether there was gender-based differences in obtaining vaccinations and perform an additional intervention if needed. The cohort hypothesised that this new process would improve efficiency and data collection.

Figure 1

Process map of clinic flow for administration of vaccine prior to interventions. Shaded blue square represents where interventions #1 and #2 were implemented. EMR, electronic medical record.

Cohort 2 (30/7/2018–26/8/2018)

Cohort 2 implemented the questionnaire process. After 2 weeks, the group found that many questionnaires were returned incomplete. Stakeholders, including medical assistants, reported the questionnaire made the process more efficient and felt that the process should be continued. The questionnaire was updated by improving the readability of the document by better highlighting important questions. Despite these changes, there was no improvement in the percentage of questionnaires fully completed.

Cohort 3 (27/8/2018–23/9/2018)

Cohort 3 reviewed the process with the clinical staff and faculty and determined that there was not enough time for patients to complete the questionnaires, and the questionnaire relied heavily on the patients to remember their immunisation history, unfamiliarity with electronic medical record (EMR) documentation workflow and lack of standardised process for medical assistants to vaccinate patients. Cohort 3 decided to eliminate the questionnaire. In its place, Cohort 3 implemented a bundle of interventions (table 1) that included: standardising the medical assistant workflow (figure 2), increasing staff access to the Texas state vaccination registry, increasing staff access to the EMR and patient education. Additionally, EMR workflows were posted in examination rooms. Patient education slideshows were played in the waiting room and informational handouts were available for patients in the patient rooms. Last, clinical staff were trained on the proper vaccination timeline.

Table 1

Cohort 3 vaccination improvement bundle

Figure 2

Process map of clinic flow with implementation of intervention #3. Shaded blue squares represent interventions. EMR, electronic medical record.

Data collection

Immunisation data were collected prospectively from the EMR from 10/2018 until 10/2019. We used a composite immunisation metric which included patients eligible for pneumococcal, influenza, varicella, HZ virus and tetanus and diphtheria vaccination. The metric denominator was all adult MLIU patients seen at the primary care location and the numerator included those patients who received the immunisation in the measurement period. We also assessed the number of patient questionnaires that were filled out and completed in full. Data were audited by reconciling patient questionnaires with the EMR appointment list. Circle of work audits were completed by the practice manager.


The primary process measure was the proportion of eligible patients who received pneumococcal, influenza, varicella, tetanus and diphtheria toxoid and HZ immunisations.



There were a total 1683 vaccine gaps, including 662 patients eligible for pneumonia vaccination, 151 patients eligible for HPV vaccination and 1690 patients eligible for the vaccination composite metric. The eligible patient population for each measure had minimal variation throughout the year and this was affected mainly by insurance changes and new patients coming into the system. Table 2 depicts immunisation rates at baseline, of which 38.9% of eligible patients were up to date on pneumococcal vaccination, 55.5% had an influenza vaccine in the last year and 13.3% of eligible patients were up to date.

Table 2

Vaccination rates at baseline and after intervention of MLIU patients

Cohort 1


Patient-Centered Questionnaire (Version 1).


There were 139 collected forms. Of these, age and gender were left blank most often, 13.6% and 41.7%. respectively.

Cohort 2


Patient-Centered Questionnaire (Version 2).


There were 61 collected forms. Of these, age and gender were left blank most often, 24.5% and 37.7%, respectively.

Cohort 3


Improvement Bundle.


Average patients vaccinated with pneumococcal, influenza, varicella, HZ virus and tetanus and diphtheria increased from 10% to >30%.


By the end of the measurement period, pneumonia vaccination rates were 88.2%, influenza vaccination rates were 90.7%, and composite vaccination rates increased to 45.3%. All vaccination rate improvement demonstrated special variation (figure 3).

Figure 3

Control charts of pneumonia, influenza and composite measure vaccination rates over time.


This primary care resident QI project resulted in an overall increase in documented immunisation rates in the adult MLIU patient population at an academic primary care practice. The results are important for two reasons. First, a longitudinal multidepartment resident QI project can be successfully implemented with little financial or time investment. Second, the improvement bundle intervention to improve vaccination rates in the high-risk MLIU patient population practice was successful. The increase in vaccination rates was both clinically and statistically significant.

During this month-long rotation, family medicine and internal residents learnt about the lifecycle of a QI project. They learnt about the importance of stakeholder engagement and the differences and difficulties of data collection. Importantly, the residents learnt that QI projects may take one form that fails and other forms that succeed. This was demonstrated in that fact that the patient questionnaire did little to improve immunisation rates while the improvement bundle quickly led to immunisation rate improvement. This rotation allowed residents the time to research and understand their clinical problem, implement a process improvement and reflect on the results. Although some of the initial ideas did not work such as the questionnaires, it provided an opportunity for residents to reflect and internalise the process of quality improvement. By being engaged in every process, each cohort was able to make an improvement each month and ultimately reach our goal to increase vaccination rates. In addition, this project helped residents understand that persistently problem solving, investigation and collaboration with key stakeholders can impact the success of a project. Further, this project’s intervention was sustained past the residency academic year and continues to be implemented. Family and internal medicine residents continue to work in teams on longitudinal projects such as this.

Last, primary care residents were educated on the importance and impact of QI projects in a clinical care setting in the context of a value-based care delivery strategy. Like their other residency experiences, these value-based care principles are now a component of their foundational knowledge. Feedback from these residents was regularly positive and many residents indicate the knowledge learnt helped inform their job search.


An important limitation to consider is that this project was one of many immunisation initiatives undertaken by this organisation during the same time period. Therefore, the improvement in vaccination rates was likely positively impacted by these other initiatives. Additionally, the effect of intervention on influenza vaccination rates may have been affected by seasonal vaccinations as the QI project coincided with the beginning of the influenza season. There was also limited time for each cohort (4 weeks) to review data, obtain feedback and implement an intervention. Therefore, this may have impacted results of cohorts 1 and 2 since not much time was given to effectively see a change. Each cohort consisted of a different group of medical residents, resulting in the need for transition of the project between cohorts.


Increasing vaccinations rates have been shown to result in decreased infections and hospitalisations. Clinics should take a proactive role in improving preventive health. In addition to improving policies and procedures, stakeholder engagement in the development of QI projects and the education of both stakeholders and patients are key to immunisation rate improvements. As more value-based contracts and quality incentives are implemented in primary care, QI activities linked to quality metrics tied to financial incentives and educational activities linked to these QI activities will be important at academic medical centres. Future research should be done to assess resident education in value-based care and the relevance of quality improvement principles to future resident success.


We would like to acknowledge the following because of their important contributions to this project and to the education of our future physician leaders: Dr K Lee, Dr M Nadeau, Dr M Wiemers, Dr A Ramachandran, Dr P Wathen, Dr V Rodriguez-Perez, M Trevino FNP, Jeanie Paradise and all current third year internal medicine and family medicine residents.



  • Presented at Prior presentations: 24 May 2019 Southwest Texas American College of Physicians. 14 May 2019 22nd Annual Department of Medicine Research Day at the University of Texas Health San Antonio. 2nd April 2019 Long School of Medicine Research Day at the University of Texas Health San Antonio.

  • Contributors CC drafted the article. RSC and KK participated in data analysis and interpretation. All authors contributed to revisions of the article and final approval of the version to be published was by CC.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Patient consent for publication Not required.

  • Ethics approval This quality improvement project was approved by the University of Texas Health San Antonio Institutional Review Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article.