Problem

The COVID-19 pandemic continues in many regions of the world because of low vaccination rates. As a consequence, new variants continue to be generated that can partially escape the vaccine and have prevented countries from achieving and maintaining herd immunity. The highly contagious delta variant originated in India at a time when the percentage of the population vaccinated was only 10% and more recently the even more contagious omicron variant emerged in South Africa at a time when less than 30% of the population was vaccinated. Can Lean manufacturing principles be applied to improve the efficiency and effectiveness of mass-vaccination sites to more rapidly achieve vaccine-induced worldwide herd immunity and to reduce the emergence of SARS-CoV-2 variants?

Background

The speed of COVID-19 vaccination is a critical parameter for achieving herd immunity1 and mass-vaccinations sites were recently highlighted as effective approach for achieving this goal.2 One small country with a unified system of healthcare delivery has achieved high vaccination rates3; however, detailed descriptions of their vaccination procedures have not been published. Government websites also emphasised the importance of mass-vaccination sites and have provided suggestions for their structure.4 For example, the United States Centers for Disease Control and Prevention (CDC) recommends conducting ‘frequent time-motion studies and staff utilisation reviews to maximise staff roles’5; however, further details are not provided. A UK government website recommends drawing on the logistical expertise of the armed forces, and quotes a Lieutenant Lambert, a Logistics Sea Trainer: ‘As advisor to the vaccination operation, (I have engaged) various stakeholders to enable information flow, understanding and early identification of emerging issues. This has included helping design an operational view of the programme by identifying key performance indicators to aid us in spotting potential problems and providing solutions in a timely fashion’.6 While some are able to do so, many local vaccination sites are unlikely to be able to recruit military logistics or manufacturing experts to continually maximise efficiency and effectiveness.

Given the limited resources and healthcare personnel in many regions of the world, we strongly recommend that Lean principles7 and continual learning be applied to create high throughput vaccination sites. These approaches can be conveniently learnt through massive open online courses (MOOCs) that describe how Lean principles can be applied to healthcare and are available through Coursera and the English National Health Service.8 9 We describe in detail a real-world example of how we applied Lean principles and tools to create a mass-vaccination site in the hopes that others will emulate our approach as mass vaccinations are conducted around the world.

Measurement

The time of arrival at precheck-in and the time that vaccine injection was completed were recorded for all recipients and the lead time calculated for each vaccinee (the interval between arrival at precheck-in and vaccination completion) and representative values from each station were periodically entered into a cloud-based spreadsheet. The number of vaccinations for each 10-hour session was tabulated.

Design: creation of a mass-vaccination site

On 5 April 2021, the State of Florida released COVID-19 vaccines to all residents 16 years and older. The sudden eligibility of over 52 000 University of Florida students represented a major operational challenge. To create a mass-vaccination site, we identified a 32 000 square foot space in our football stadium, the Evans Champions Club, designed a vaccinee flow pattern with 8 vaccination stations (online supplemental figure S1) and created a standardised work sheet for each individual work station (figure 1) that defined the movement of each vaccinee and the 5 individual steps or work cycles in the vaccination process: (1) precheck-in, (2) online check-in, (3) fill out the consent form, (4) verify the consent form and (5) vaccinate. To further clarify our procedures, a process map was created that illustrated how these steps were combined to achieve timely vaccinations (online supplemental figure S2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Standardised work sheet showing the individual vaccine station lay out with the five work areas, and vaccinee movement. The diamond symbols mark the locations where a quality check was performed and the cross symbols mark a safety check. Each step or work cycle is numbered.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Value stream map: a standard workflow diagram that describes the work being done in each cycle, the expected time to complete each work cycle and the percentage of the work that is of value. Consent forms filled out before arrival enhance the value of step 3.

Strategy

We next conducted a series of plan–do–study–adjust (PDSA) cycles to maximise individual productivity, reduce unnecessary motion and eliminate time delays. These improvement cycles were performed on days 1–6 and the resulting process improvements were implemented on day 8. To save time, PDSA cycles were conducted simultaneously by those conducting each work cycle in the vaccination process. Online supplemental table S1 lists the individual PDSA cycles conducted during this period. For example, initially, we had two precheck-in people for each station; however, based on feedback following our early sessions, we learnt that one person was idle half the time. Therefore, we eliminated one precheck-in person per station and found that there were no time delays for the vaccinees. On creation of our value stream map (see above), we realised that the critical step for assuring efficient flow was the vaccination step. Our frontline observations revealed that some of our vaccinators were carrying on prolonged conversations (3–5 min) with their patients creating unnecessary bottlenecks. For our intervention, we created a 60-second script for all vaccinators that included a question about allergies to past vaccinations, the arm preferred for their injection and a summary of the potential short-term side effects of the vaccine (local pain, influenza-like syndrome with possible muscle aches, fatigue, headache, chills and fever) followed by advice to seek medical attention if these side effects persisted for over 72 hours. The script reduced excessive conversations reducing vaccination times. We also conducted multiple PDSAs to arrange the locations of each work cycle within the vaccination station. We discovered that keeping distances to the minimum dictated by infection control practices (6 feet) minimised walk times reducing wasted motion. On day 8, we implemented an improvement bundle that included all of changes derived from our PDSA learning cycles (see the Results section).

Because of the short time frame for the development of our mass-vaccination site (1 week), involvement of patients in the development of our site was not possible. During registration and while being observed following vaccination, patient comments were shared with two of our authors (MNF and MPW) and patient suggestions for improvement were incorporated into our PDSA cycles.

Time observation sheets recorded the five individual work cycles six times for each workstation (online supplemental figure S3). An observer used a stopwatch to time six individual subjects going through a vaccine station. We measured walk times and the time spent to complete (1) precheck-in, (2) online check-in, (3) filling out the consent form, (4) verification of the form and (5) vaccination. All times were entered on an Excel spreadsheet and the mean time for each step or work cycle was calculated. Using these mean times, a value stream map was constructed documenting a realistic ideal state for the process (figure 2). The time required to fill out the consent form was shortened by 2–3 min for those who had filled out the online form before arrival (approximately one-fourth of vaccinees). Because the 5 areas were 6 feet apart, walk times were only 2–3 s. Reconstituting the vaccine was performed by a pharmacist team as previously described following the manufacturer’s instructions10 and represented a separate process that was conducted in a site adjacent to the vaccine stations (online supplemental figure S1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Run chart showing the median for the daily mean vaccination lead time (the mean time from precheck-in to vaccination) before and after implementation of bundle of improvements (arrow).

Vaccinations began on 5 April and occurred each Monday, Wednesday, Friday and Saturday for 10 hours. For the first week, all vaccinations were prescheduled by online appointments, allowing an accurate estimate of the vaccine demand for each day. As the appointments began to decrease, we also accepted walk-ins, 150–200/day. Our highest vaccine demand occurred during the 1st week and consisted of 5000 appointments/day (online supplemental table S2). To meet the demand of 5000 vaccinations over 10 hours, we calculated that the maximum time for each vaccination needed to be 7.2 s (36 000 s/5000=7.2 s). Our time observation sheet (online supplemental figure S3) revealed the time required for one vaccinator to complete one injection varied from 73 s to 138 s (mean: 97 s). To meet the 5000 vaccines in 10-hour demand, we calculated that we would need between 10 vaccinators and 19 vaccinators (73/7.2=10.1; 138/7.2=19.1). To assure we had an adequate number of vaccinators, we chose to recruit 20 vaccinators per session and set as their goal to inject one person every 144 s or 25 vaccinees/hour. To assure smooth work flow, all other steps needed to take no longer than 144 s and as shown on the time observation sheet (online supplemental figure S3) and value stream map (figure 2): this condition was met.

To encourage all volunteers to follow a standardised work protocol, each job was completely described during morning orientation. Online supplemental table S3 provides complete descriptions of the four key jobs (precheck-in, online check-in, consent verifier and vaccinator). The precheck-in consisted of filling out a laminated card that served as a visual control to monitor each vaccinee’s progress (online supplemental figure S4). In addition to the person’s last name and birthdate, the time they arrived at the precheck-in table and time of vaccination were transcribed on the laminated sheet, as was the type of vaccine P for Pfizer, M for Moderna, 1 for first shot and 2 for second shot. A box was checked when online registration was completed and once the consent form was verified. The verifier then directed the individual to one of the two vaccinators who reviewed the allergy history, described possible early side effects from the vaccine, prepared the designated right or left deltoid with an alcohol wipe, injected 0.3 ccs of vaccine and placed an adhesive bandage over the injection site. Individuals with a history of past allergic reactions were then observed for 30 min and all others observed for 15 min as recommended by the US CDC. After completing the injection, the vaccinator entered the time the vaccine was administered, and the time the observation period would end on the laminated card, and finally, signed the consent form.

The laminated cards were collected by an individual designated as the ‘runner’ who entered the time of precheck-in and the time of vaccination in a cloud-based spreadsheet and then returned the cards back to the precheck-in desk where the card was erased and reused. The cloud-based shared spreadsheet was monitored real time by an administrator. If one station was consistently demonstrating delays, the administrator went to the site, identified the sources of delay and solved the problem with the team.

The runner was also responsible for transporting the signed consent forms to the end-of-the-line inspection station (online supplemental figure S1). At this site, forms were inspected to assure that all parts of the form were completed before delivery to the county health department. Finally, the runners were responsible for continually monitoring vaccine supply and transporting prefilled syringes from the pharmacy, replenishing the stock when the number of syringes dropped below 5. One runner was assigned to each station.

Other support personnel included ushers who directed the vaccinees, that included one person in a golf cart who transported those who had difficulty walking from the adjacent parking lot to the vaccine site. The total personnel required for each session was 78: 20 vaccinators, 20 check-in personnel, 10 consent form verifiers, 8 runners, 6 ushers, 2 end-of-the line inspectors, 8 pharmacists and 4 administrators. With the exception of the administrators, all others were volunteers from the health centre and/or the community.

Lessons and limitations

Multiple mass COVID-19 vaccination sites have been described in the medical literature4; however, few peer-reviewed reports have included detailed logistics or delivery performance measures. One modest-sized site used a large conference room and 5 vaccinators to perform 250–300 vaccinations/day. Vaccinations were administered for 14 hours each day, therefore, each vaccinator performed 4–5 injections/hour or less than one quarter the output per vaccinator of our system.10 A mass-vaccination site in San Diego took place in a 280 000 square foot parking lot and used 300 as compared with our 78 personnel to deliver 5000 vaccines/day,11 our approach achieving 3.4-fold higher productivity. Another parking lot vaccination programme in Florida applied PDSA cycles, but did not use value stream mapping to achieve an average output of 8.6 vaccinations/vaccinator/hour.12 In Italy, vaccine stations were established in a large open parking lot and achieved a similar output of 8 vaccinations/vaccinator/hour.13 A college in Montana created a vaccination site for students and faculty and conducted two 1-day sessions that vaccinated 807 and 776 individuals. This site achieved a maximum productivity of 8.2 vaccinations/vaccinator/hour.14 These last three vaccination programmes achieved comparable productivity levels that were one-third of the productivity of our vaccine programme.

We used the work sheets downloaded at no cost from the Coursera MOOC entitled ‘Fixing Healthcare Delivery 2.0 Advanced Lean’ to design and improve our mass-vaccination site.8 The stardardised work sheet was used to design the physical layout of our vaccination station (figure 1), the time observation sheet to record the times to complete the individual work cycles and walk times (online supplemental figure S3), and a value stream map to assess the value of each process and identify opportunities for improvement8 15 (figure 2). For example, our analysis revealed that filling out the consent form prior to arrival at the vaccination site had the potential to shorten the lead time by 2–3 min, which would increase the number of vaccinations per hour by 20%–30%.

To assure that registration, consent forms and allergy histories were accurate, error proofing was designed into our system: precheck-in was verified by the online check-in person, the vaccinees consent form was reviewed by the verifier, allergy history confirmed by the vaccinator and finally the end-of-the-line inspection for all consent forms was conducted at the inspection desk prior to these forms being sent to the health department.

Mass-vaccination sites are particularly suitable for the application of Lean principles and the US Naval Academy applied Lean principles to improve the efficiency of annual vaccinations for 1200 new naval cadets reducing vaccination lead times (time of check-in to receipt of the vaccine) from 55 min to 11 min.16 Very recently, a team from the NHS applied the lessons taught in their Lean MOOC to improve vaccination lead times in a small beta test involving 16 individuals.9 More recently, this same team applied Lean principles to create a mass-vaccination site that enabled 8 vaccinators to inject 1500 individuals/12-hour shift achieving an output of 15.6 vaccinations/vaccinator/hour.17 We have now applied this same approach to deliver up to 5046 vaccines over 10 hours using 20 vaccinators achieving an individual output of 25 injections/vaccinator/hour. Like the NHS team, we have provided detailed descriptions of each improvement step so that others inexperienced in Lean principles can emulate our example.

A limitation of our project was the application of our Lean improvement tools to a single vaccination site rather than multiple sites. Second, we depended on volunteers that often worked for only three–four sessions, a condition that could compromise the consistent application of standardised work. However, the first 30 min of each session consisted of a detailed orientation that included breakout sessions for each individual job. The orientation instructors were a stable team of paid healthcentre employees who were present for all the vaccine sessions, a condition that maintained continuity and encouraged standardised work. The impact of a single bundle of improvements rather than multiple sequential improvement steps was documented by a formal run chart. Because the semester ended in mid-May, the time window was narrow for vaccinating a significant percentage of student body and this time limitation required us to accelerate change by combining multiple rapid PDSA cycles into a single bundle of improvement interventions. Finally, because times were entered by the runners, it is remotely possible that they favoured entering shorter times; however, this bias would be expected to have been present throughout the study and, therefore, relative differences between early, mid and late sessions should be equally impacted. One of our authors (MG) is a renowned Lean expert who did not participate in the mass-vaccination site design and served as an objective referee to curb positive bias.