Continuity and timely access are hallmarks of high-quality primary care and are important considerations for urgent concerns that present both during the day and after-hours. It can be especially difficult to ensure continuity of primary care after-hours in urban settings where walk-in clinics offer patients easy and convenient access. Patients of our large, multisite primary care practice in inner-city Toronto, Canada were reporting that they were not easily able to access after-hours care from their team without having to use outside services. In partnership with patients, we combined the Model for Improvement with Experience-Based Design methodology to address the issue of poor access to after-hours care. We did a root cause analysis to isolate the causes of the local problem, using a variety of capture tools designed to incorporate the patient voice. Then, patients and providers codesigned two Plan-Do-Study-Act (PDSA) cycles aimed to increase the ease of accessing after-hours care. Key actions included a redesign of our after-hours advertisement and communication of the material in multiple formats. Following these PDSA cycles, the team saw a 26%, 23% and 17% increase in awareness of weekday evening clinics, weekend clinics and after-hours phone services, respectively, and a 16% increase in the proportion of patients reporting that it was very or somewhat easy to get care during the evening, on the weekend or on a holiday from their care team. Measures continued to improve and improvements have been sustained 3 years later. Our success highlights the effectiveness of partnering with patients to improve access to primary care.
- continuity of patient care
- continuous quality improvement
- quality improvement methodologies
- quality measurement
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- continuity of patient care
- continuous quality improvement
- quality improvement methodologies
- quality measurement
The St Michael’s Hospital Academic Family Health Team is located in inner-city Toronto, Canada. An interdisciplinary team of over 70 staff physicians, 40 resident physicians and >60 other health professionals provide primary care to over 45 000 patients, at six practice sites. We serve a diverse patient population, including a higher than average proportion of patients living with HIV, addictions and mental health conditions.
As a Family Health Team, we are required to provide after-hours care to patients. For the purposes of this initiative, ‘after-hours’ was defined as during the evening, on the weekend or on a holiday. Evening and weekend clinics are operated out of different practice sites on different days of the week, with all physicians rotating to provide coverage. All six practice sites share one electronic medical record, meaning that although patients are unlikely to see their primary physician in the after-hours clinics, information is shared easily between all clinic staff.
Despite longstanding evening and weekend clinics, a patient experience survey we began in 2014 noted that only 56% of surveyed patients reported being able to easily access after-hours services at our team, without having to go to an outside walk-in clinic or emergency department, the last time they were sick after-hours. We began our initiative in the spring of 2015 and our aim was to improve the proportion of surveyed patients who could easily access after-hours care by 10% within 1 year of the intervention.
Access to primary care has demonstrable health benefits and has been linked to reduced health system costs.1 2 As such, primary care has been at the heart of health system reform in Ontario, Canada.3 In 2016, 94% of surveyed patients reported having access to a primary care provider for their regular care,4 and yet in Canada, an estimated 20% of emergency department (ED) visits not resulting in hospital admission were for conditions that would be best managed in primary care.5 Although the association between after-hours primary care and ED use has been mixed,6–10 access to timely primary care after-hours has been linked to reduced ED use in some circumstances, including for those with chronic conditions.11 12
Ontario’s primary care reforms have resulted in approximately 75% of patients being enrolled in ‘medical home’ models, which require organisations to provide after-hours care to their patients.3 Despite this shift, only 56% of patients reported being able to easily access after-hours care without having to go to an outside walk-in clinic or the ED, placing Ontario in the middle when compared with other high-income countries.13 Our Family Health Team’s performance at baseline was higher than the provincial average but still lower than other high-income countries and we were motivated to do better. This paper describes our efforts to improve access to after-hours services in our large, multisite primary care practice in Toronto, Canada.
Table 1 summarises all measures used for this initiative. We selected outcome and process measures that used data from our organisation’s established patient experience survey. All Family Health Team patients with an email address on file are emailed a link to the online experience survey during their birth month.14
We selected ease of access as our outcome measure, and awareness as a process measures, because we hypothesised that a change in awareness may precede a change in the proportion of patients reporting being able to easily access after-hours care. We used visits and phone call volumes as balance measures because we hypothesised that increasing awareness of after-hours clinics may increase utilisation. Although each measure was collected monthly, data were summarised bi-annually due to sample size considerations.
Baseline measurements were collected prior to the intervention. Between January and June, 2015, 63% reported that it was easy to access after-hours care and 62%, 58% and 35% reported awareness of the evening clinic, weekend clinic and telephone service, respectively. Between March and May 2015, prior to the start of our intervention, the evening clinics, Saturday clinic and telephone service saw average monthly volumes of 135, 118 and 90 visits per month, respectively.
To determine whether the change in each measure was associated with the interventions, each measure was charted on a run chart, with the intervention time annotated. Because the process and outcome measures were assessed using a survey, we assessed changes in these measures using 95% CIs.
We used experience-based design15 16 to integrate the patient voice at each step of the Model for Improvement. We hypothesised that involving patients in improving access to care would result in more marked and sustained improvement.
First, we completed a root cause analysis to determine why patients reported difficulty accessing after-hours care, despite the Family Health Team’s after-hours services. We hypothesised that the root cause was that we had not adequately informed patients of after-hours services, resulting in low awareness and that the structure of the clinics was confusing.
To determine whether the structure of the clinics was causing confusion, we used our electronic scheduling system to explore the volumes of visits at each evening clinic, and for each visit, compare the practice site in which the evening clinic was being held to the practice site for the patient’s regular physician. We found that patients were statistically significantly more likely to seek after-hours care at the site their physician normally holds clinic, despite the fact that they are highly unlikely to see their regular physician after-hours (table 2), suggesting a lack of clarity about the Family Health Team’s after-hours services.
Our surveys ask two open-ended questions: what patients like best about being a patient at the Family Health Team, and how we could improve. We used these open-ended questions to glean insights into after-hours care, determining that many patients were unaware of the after-hours clinics, but that generally speaking, those who attended the clinics were pleased with the services offered. When asked about why they attended the clinic, one patient said ‘It’s convenient, walk-in and family doctor connect and share info’. The open-ended questions also revealed that the survey itself acts as an awareness tool in and of itself.
Finally, we developed two different questionnaires for patients. The first was administered in after-hours clinics and was used to better understand patients’ experience accessing after-hours services. The second was administered in regular daytime clinics and was intended to gauge awareness of after-hours services and barriers to accessing such services. We interviewed 39 patients at this stage and found that the most common barrier to accessing after-hours services was a lack of awareness of the services altogether, rather than issues with the structure or timing of the clinics themselves. With a better idea of the root causes of patients having a difficult time accessing after-hours care, we addressed the next question in the Model For Improvement, ‘what changes can we make that will result in improvement’.
Our work did not undergo formal ethics review at our institution as the primary purpose was to improve our core operations in line with standards of care.
Following our patient-informed root cause analysis, we partnered with patients to test changes using the Plan-Do-Study-Act (PDSA) methodology.
PDSA 1: working with patients to determine how to increase awareness of after-hours services
We developed a patient questionnaire to understand patient views on what methods would work best to increase awareness. Because we already had posters and tear-off sheets throughout clinics to advertise after-hours services, we predicted that patients would not want more print materials and would prefer electronic means of contact.
We interviewed patients immediately after viewing our current poster, asking both about the effectiveness of our current advertising materials and how they could be improved. In total, 20 patients provided input at this step.
Our predictions were partially accurate; although patients wanted an email informing them of the after-hours services, they also advised that posters and tear-off sheets be redesigned to be brighter, larger and clearer. They also suggested that we create a pamphlet to educate them about a variety of services, including after-hours services.
Because there were no dedicated resources for this initiative, we addressed the least costly method first sending an email to the approximately 10 000 patients with an email address on file. We then partnered with patients to redesign the print materials.
PDSA 2: partnering with patients to codesign print materials advertising after-hours care
We printed existing print materials and approached patients in the regular clinic waiting rooms to codesign new materials together.
Because time with each patient would be limited, we started by drafting potential posters, tear-off sheets and pamphlet designs for patients to weigh in on and tweak. We then canvassed patients in waiting rooms at each of our six sites from a variety of age groups, ethnic backgrounds and spoken languages. We asked them about message clarity, literacy level and layout. The codesign process was iterative—as patients provided new ideas, the materials were edited before being presented to more patients. This process was repeated until suggestions were minimal or conflicting.
The resulting poster and tear-off sheet are shown in figure 1. Key changes included highlighting the phone line more prominently and using symbols to guide patients for whom English is not their first language; making the posters more colourful; and clarifying that patients can seek after-hours care at any clinic of their choosing, regardless of where their regular physician practices. The redesigned materials also explain why it is safer to access after-hours care from their care team, as some patients conveyed that after-hours care services seem focused on revenue-generation, rather than improved care. The pamphlet conveyed similar information, as well as other information such as lab hours and the members of the care team.
We took the newly codesigned materials to a professional designer to be finalised before being placed throughout the clinic, in locations advised by patients.
The awareness of evening, weekend and telephone services increased from 62% (95% CI: 58% to 66%), 58% (95% CI: 54% to 62%) and 35% (95% CI: 31% to 39%), from January to June 2015, to 87% (95% CI: 85% to 90%), 81% (95% CI: 78% to 84%) and 52% (95% CI: 49% to 56%), from January to June 2017, respectively (figure 2). Subsequent to the increase in awareness process measures, the per cent of patients reporting being able to easily access after-hours care from their care team the last time they were sick after-hours increased from 63% (95% CI: 56% to 69%), from January to June 2015, to 79% (95% CI: 74% to 84%), from January to June 2017 (figure 3). The most recent survey data show that from July to December 2018, patients reported the awareness of evening, weekend and after-hours phone line services was 83%, 75% and 48%, and 74% reported being able to easily access after-hours care.
The volume of visits and phone calls to after-hours clinics—used as balancing measures—did not increase uniformly. The volume of phone calls increased following the intervention before subsequently decreasing, whereas the number of visits to the evening and Saturday clinics did not show substantial changes (figure 4).
Lessons and limitations
We partnered with patients to understand and address barriers to accessing care after-hours in a large primary care organisation. Patient recommendations were relatively straightforward and implementation of these recommendations resulted in significant, sustained improvements in patient-reported ease of accessing care after-hours. Improvements in ease of access were associated with higher use of after-hours phone services but not higher visit volumes.
Our methodology had both strengths and limitations. We integrated the patient perspective in our root cause analysis and in the design of our intervention. We were able to engage a diverse group of patients with minimal effort by approaching those in the waiting room. We measured ease of accessing after-hours care and awareness of the clinic over a long time period allowing us to understand baseline performance and sustainability of improvements. However, our process and outcome measures were taken from a patient experience survey that is sent only to patients with an email address. Our previous work showed there were fairly minimal differences in ease of after-hours between patients surveyed via email vs the waiting room (Slater), but patients receiving our email survey may have developed higher awareness of clinics as the survey itself is an awareness tool. Despite this, we noted a minimal increase in ease of after-hours access in the baseline period that suggests the improvements were related to our intervention.
Our improvement initiative also coincided with other changes in our Family Health Team including the opening of our sixth site in July 2015 and the introduction of an afternoon Sunday clinic at that site in September 2015. Although these changes likely had little impact on awareness of after-hours services, the opening of the Sunday clinic may have also contributed to the improvement in the proportion of patients reporting being able to easily access after-hours services. However, it is very unlikely to have alone resulted in the degree of improvement sustained at all six of our clinic sites.
At the outset, there were concerns that the process of partnering with patients would be too costly to be sustainable, and the interventions patients suggested may not be affordable; both concerns were unfounded. As with many primary care organisations, we have limited non-clinical resources to facilitate improvement efforts. In total, we partnered with over 60 patients at multiple stages in the Model for Improvement. Rather than extensive consultation with a small number of patients, we partnered with many patients for a shorter period of time and in an iterative manner. Many patient recommendations were pragmatic and fairly easy to implement such as revisions to the poster design.
As a result of our initiative, we noted a temporary increase in the volume of after-hours phone calls. Although we measured this and noted it early, it took us time to make needed operational changes. On-call resident physicians work with the staff physician in the evening and weekend clinics and traditionally responded directly to after-hours phone calls. However, after the intervention, residents began to be interrupted repeatedly with phone calls during the after-hours clinics. To minimise clinic interruptions and ease the workload, we redirected after-hours calls that came during clinic hours to the clerical staff at the clinic. Clerical would direct the call to the nursing staff and/or the resident physician as needed. Adding additional triage enabled other members of the team to address concerns, easing the workload of resident physicians.
We partnered with patients to improve after-hours access in a large multisite urban primary care organisation. We used codesign strategies at various stages of the project including root cause analysis and design of the intervention. Implementation of patient recommendations resulted in sustained improvements in self-reported ease of after-hours access and a corresponding increase in after-hours phone calls. Although it is unclear if our efforts have impacted other health system use, feedback from patients is positive. Overall, our efforts have supported our primary care team in providing timely, continuous care to patients regardless of the time of day.
The authors would like to thank the following individuals for their contribution to this work: MaryBeth DeRocher, Charlie Guiang, Jackie Chen, Paula Blackstien-Hirsch, Trish O’Brien, Amy McDougall, and the St Michael’s Hospital Academic Family Health Team Quality Steering Committee.
Contributors SD and TK conceived and designed the initiative together with members of their team. SD conducted the analysis. SD and TK interpreted the results. SD drafted the manuscript. Both authors critically reviewed the manuscript and gave final approval.
Funding TK is the Fidani Chair of Improvement and Innovation at the University of Toronto. She is supported as a Clinician Scientist by the Department of Family and Community Medicine at St. Michael’s Hospital and at the University of Toronto. She is also supported as an Embedded Clinician Researcher by the Canadian Institutes of Health Research and Ontario Health (Quality Business Unit).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon request.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.