Problem

The rheumatology clinic at Sunnybrook Health Sciences Center (SHSC), Toronto, Ontario, sees an average of 200 patients a week under the care of 5 rheumatologists, with 3 supporting administrative staff managing booking, phone calls, and registration. SHSC is a tertiary referral center, with a middle to high socioeconomic demographic in the immediate area, although referrals are seen from all sociodemographic groups. Missed appointments or ‘No-Shows’ (NS) present barriers to appropriate disease and medication monitoring and result in wasted time and resources. A lack of appointment reminders is a possible root cause of NS, however available resources do not permit telephone reminders, including automated 'teleminders', to address this problem. Our aim was to improve appointment attendance by 50% by April 2015.

Background

Patients with rheumatic diseases are at risk of progressive organ dysfunction, disability, deformity, and premature death.1 ,2 High quality care requires early diagnosis and treatment by a rheumatologist,3 and longitudinal follow-up to monitor disease activity, treatment side effects and toxicities.4–7

Among patients with chronic medical conditions, missed appointments or “No-Shows” (NS) are associated with increased emergency department visits,8 same-day clinic visits,9 hospital admissions,10 ,11 worse disease outcomes,9 and increased mortality.11 ,12 For providers, no-show appointments lead to inefficiencies in time and resources,13 ,14 resulting in missed opportunities for other patients waiting to be seen, and reduced educational opportunities for trainees.15

No-shows in rheumatology have a reported frequency of 7-29%.14 ,16 NS have been attributed to forgetfulness,14 and have been associated with longer wait times, distance from the clinic, younger age, and being an ethnic minority.16 Patients with rheumatic diseases report feeling too physically impaired to attend, or having too many other specialist appointments.17 Conversely, greater perceived disease severity was associated with appointment adherence in patients with systemic lupus erythematosus.18

Most interventions to reduce NS take the form of appointment reminders, and have led to absolute reductions in missed appointments of 3-27%.19–29 Telephone reminders are a common intervention,19 ,22 ,23 ,29 but can be time and resource consuming for the provider. Text and email reminders have been effective alternatives.24–27 Web-based portals that enable patients' access to personal health information are now widely used in healthcare settings.30 In one study, no-shows were significantly reduced among health portal users who signed up for appointment reminders to their personal email account.24

When rheumatology patients were surveyed, they expressed a strong interest in appointment reminders, with younger patients preferring text messages and older patients preferring a phone call.31 ,32 A study found that hand arthritis was not a significant barrier to using email or text messaging, but computer use may present fewer physical difficulties compared to mobile phones.33

In summary, reducing missed appointments could improve efficiency, timely access to care, patient experience, and patient outcomes.Although interest and feasibility of electronic reminders among rheumatology patients has been assessed, there are no reports of the impact of such an intervention on appointment attendance.

Baseline measurement

Baseline measurement of the NS population was sought through a retrospective chart review and patient telephone interviews, for which institutional ethics approval was obtained. All NS to 4 participating rheumatologists' clinics over a 6-month period were tracked. One-hundred and ten No-Shows took place, with a NS rate ranging from 2.5-6.8% of clinic visits depending on provider. The majority of NS (78.2%) were follow-up appointments. See Table 1 in supplementary materials for baseline characteristics of new and follow-up patient NS.

Patient characteristics were compared between non-attenders and a random sample of attenders from the same period. Among follow-up patients, after adjustment for confounders, older age (OR 1.023, 1.003-1.043) and MyChart enrollment (OR 2.58, 1.32-5.06) were associated with attendance (Table 1a). Non-attenders and attenders were similar among new patient NS (Table 1b)⇓.

No-Show patients' experiences with the clinic appointment process and reasons for non-attendance were sought through personal telephone interviews. Patients were mailed a letter of information and were called within 2 weeks of their missed appointment (Figure 1, Supplemental Materials). If we were unable to reach them within 2 weeks of their missed appointment, these patients were not interviewed. Thirty-three patients (approximately one third of NS) were ultimately reached and agreed to participate. Of these, 15 subjects (45.5%) forgot about the appointment or were unaware of the appointment in the first place, and 7 (21.2%) had thought the appointment was on a different date. Seventeen (51.5%) subjects believed an appointment reminder would be helpful, and of these, thirteen subjects (76.5%) were interested in email or text message reminders (Figure 1). Reasons for forgetting the appointment varied, such a failed reminder system, long intervals between appointments, having multiple specialist appointments to juggle, or having a language barrier (Figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Pareto Chart - Interview Responses of Reasons for No-Show (n=33)

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Interview responses- other interview themes

Informal focus group discussions were conducted with the 3 administrative staff regarding their perspectives on barriers to appointment attendance. They found it challenging to communicate with patients regarding appointments due to a high burden of incoming calls to the clinic. A 1-week audit of incoming calls revealed that of 296 calls, 85 (29%) were to confirm or discuss existing appointments. The remaining calls were to schedule or cancel existing appointments (n=55, 19%), disease-related questions (n=15, 5%), or inquiring about medication, test result, referral, or other (n=141, 48%).

Design

The institution's health portal, “MyChart”, allows patients to access personal health and appointment information. An appointment reminder through MyChart was chosen as the main intervention, to be piloted in one rheumatologist's clinic, for the following reasons:

1. Reminders are the most commonly cited successful intervention in the literature, and the clinic did not offer appointment reminders for patients at baseline.

2. The majority of missed appointments (85%) were the result of forgetting, being unaware of the appointment, date mix-ups, or other miscommunication, all of which would be addressed by a reminder.

3. Patients expressed interest in electronic appointment reminders.

4. The electronic reminder would create minimal work for administrative staff and would be low cost, making it sustainable.

5. Twenty-nine percent of incoming calls were to confirm or discuss appointments, and a reminder could potentially reduce calls to the clinic to clarify appointment details.

The electronic appointment reminder would be possible through the linkage of clinic schedules with the MyChart application and patients' associated email accounts. Newly referred patients would not receive a reminder unless they already had a portal account through prior clinical encounters at the hospital.

The following measures were used throughout the intervention:

1. Process Measures: In order to know how many patients were exposed to the intervention, MyChart (health portal) enrollment and email reminder receipt were measured. An anonymous waiting room survey was distributed asking whether they were enrolled and whether they received the email (Figure 8, supplementary materials). Responses were tracked on a monthly basis across 6 months in the form of a run chart. In addition, health records department data was obtained on the number of new MyChart accounts created per month by rheumatology patients. This was also tracked using a run chart.

2. Outcome Measures:

a. No-Shows were tracked, along with the number of patients who attended in the interval between two NS. Data collection occurred approximately once per month. Frequency of NS, expressed as ‘number of patients between no-shows’, was tracked using a G-chart. This method was chosen as it is useful for examining changes in the frequency of relatively rare events, and to control for variability in patient volumes from month to month.

b. Perceived effectiveness of reminders and patient satisfaction was evaluated through anonymous waiting room surveys (Figure 8, supplementary materials) across 6 months. Patients were asked if they found the email reminder helpful, and were allowed to write any additional comments on the survey. Responses were tracked on a monthly basis.

1. Balancing measures: Balancing measures included tracking any adverse impact that the emails had on patients or on administrative staff workload.

a. At all meetings with administrative staff we emphasized the need to relay any complaints about MyChart or the email reminders. In addition, waiting room surveys (described above) allowed patients to relay any feedback about the intervention anonymously. The health records department was asked to identify any patients who had requested to be unsubscribed from the email reminder service.

b. We directly inquired to the three administrative staff about the impact of the interventions to their workload. We tabulated the number of calls to the providers' administrator during two randomly selected one-week periods, before and after the intervention, to ensure that call volumes were not increasing.

To engage administrative and physician staff, regular meetings were conducted to discuss the project, progress, and concerns.

Strategy

PDSA 1-3 relate to the process of increasing MyChart (health portal) enrollment, and PDSA 4-6 relate to the process of improving appointment attendance. PDSAs 1-3 and PDSAs 4-6 were two related improvement processes occurring in parallel with each other with overlapping timespans.

PDSA 1:

Plan: Our aim was to increase enrolment in MyChart above 80% in preparation for the future email reminder intervention that would require patients to have an account.

Do: Based on interviews, NS patients were not aware of MyChart or did not know how to obtain an account, but were interested in signing up once they learned about the application. We hypothesized that if we provided information to patients on what the portal was and how to sign up, that enrollment would increase. Starting December 2014, signs were displayed in the clinic waiting room, reception, and exam rooms to instruct patients how to sign up for MyChart. Take-home flyers were also provided at every receptionist window and clinicians were reminded to refer patients to join. A follow-up card was also introduced in January 2015.

Study: Between December 2014 and February 2015, the enrolment rate increased, but then decreased again below the median (Figure 5, supplementary materials). We learned that some patients had difficulty going to the Health Records department following their appointment to initiate the sign-up process, or would forget to do so.

Act: Further strategies to increase enrolment should make it easier for patients to sign up.

PDSA 2:

Plan: We hypothesized that being able to sign up from home would increase enrolment due to increased convenience.

Do: Due to institutional procedures, patients could initially only register for MyChart by visiting the Health Records department. Through collaboration with Health Records, we provided an email address through which patients could request a Personal Identification Number and register from home, with family assistance if needed.

Study: From February 2015 to June 2015, reported MyChart enrollment (Figure 4, supplementary materials) as well as the rate of MyChart enrollment (Figure 5, Supplemental Materials) appeared to increase. However, the rate of enrollment again dropped, and we had not yet reached our goal of 80% enrollment.

Act: Further intervention was required to improve enrolment. We discovered that the cancer center at SHSC was able to achieve high portal enrollment, and identified that demonstrations/ presentations in the waiting room and direct physician endorsement of MyChart were possible factors.

PDSA 3:

Plan: We hypothesized that interactive waiting room demonstrations might be informative for patients and increase interest.

Do: We collaborated with the Health Records department again to carry out two pilot sessions of waiting room demonstrations which were held in September 2015.

Study: Overall the demonstrations were well-received, however, there were a limited number of waiting room patients at any given time, making demonstrations less efficient, and many patients reported already having accounts. Reviewing the data, enrolment had increased from a baseline of 47% among attenders to 70% at the latest survey.

Act: We did not pursue further specific interventions to increase MyChart enrolment.

PDSA 4:

Plan: Our aim was to improve appointment attendance by 50% by May 2015. We hypothesized that an integrated follow-up card, listing the next steps in the plan of care (lab tests, x-rays, medication changes) as well as the next appointment date could improve upon the current reminder system, which was a business-sized card that listed the appointment date and time only.

Do: Starting in January 2015, patients were given a standardized follow-up card (5.5 × 8.5”) that the clinician was able to complete prior to the end of the visit, and to which the administrative assistant added the appointment information.

Study: This integrated reminder slip did not increase appointment attendance above baseline (Figure 4). Based on the results of telephone interviews, factors such as long intervals between appointments may result in misplaced appointment cards.

Act: A reminder closer to the time of the next appointment was needed.

PDSA 5:

Plan: We hypothesized that an appointment reminder closer to the appointment date would target one of the main root causes for NS, which was forgetfulness. An email reminder for patients enrolled in MyChart was a low-cost strategy that did not require extra work for administrative staff.

Do: A semi-automated reminder intervention was piloted in one rheumatologists' clinic starting February 2015. The administrative assistant uploaded the list of the next weeks' clinic patients onto the secure hospital server. A program was engineered in collaboration with Information Technology and Health Records: patients who were enrolled in MyChart would have an automated standardized email sent to the address associated with the account.

Study: From February - March 2015, no improvement in appointment attendance was seen. Email reminder receipt was measured through waiting room surveys, and it was discovered that on several days the email reminders sent the previous week had not been received (17th February and March 2nd) (Figure 3). This was due to a technical issue despite the patient having a MyChart account.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Email reminder receipt among MyChart users

Act: We needed to increase email reminder receipt in order for the intervention to be effective.

PDSA 6:

Plan: We needed to resolve technical difficulties with email reminder send-outs that were resulting in missed send-outs on some weeks (Figure 3).

Do: The Information Technology division of Health Records determined the reason for send-out failures, and fixed the underlying problem. In addition, a weekly monitoring system was established to ensure send-outs were occurring every week.

Study: The email reminder was associated with improved attendance in May 2015, when the number of patients between NS rose over 4-fold (Figure 4).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Number of patients between No-Shows

Results

A trend toward increased portal enrolment was observed among surveyed waiting room patients. On the last and third last weeks sampled, over 70% of patients reported having MyChart accounts (Figure 4, Supplemental Materials), compared to 47% of attenders at baseline (Table 1B). The rate of new account creation remained constant over time (Figure 5, Supplemental Materials). Of 281 consecutive waiting room patients surveyed, 79 (28%) received the e-mail and 76 (97%) found the reminder helpful and wanted future email reminders. Among those who did not get the reminder, 39/47 (83%) MyChart users and 66/155 (42.6%) non-users were interested in future e-mail reminders.

A G-chart was used to measure the impact of the intervention on NS (Figure 4). Early during the intervention period, the number of patients between NS was consistently below the upper control limit (UCL). In April 2015, there were less NS and thus the number of patients seen between NS significantly increased, as seen in the G-chart as the point above the UCL. This improvement was not sustained in May and June. Overall, twenty-two patients (8%) indicated from the survey that they only learned of their appointment from the email reminder, suggesting that these 22 patients were averted NS.

Pre-intervention there were 79 incoming calls received by one administrative assistant in a one-week audit, and post-intervention there were 75 calls, indicating no change in overall call volumes. Staff did not receive any complaints about the reminder emails and no one indicated they wished to unsubscribe from the service. We found no perceived difference in the self-reported administrative staff workload.

Lessons and limitations

The strengths of this project include a practical, low-cost intervention in a clinic with resource constraints that required minimal extra time and effort by the administrative staff, and led to high patient interest and satisfaction with improved attendance in April 2015. There was evidence that the intervention resulted in averted NS, as 22 patients reported learning of their appointment from the email reminder alone. No negative effects of the intervention were identified.

The project also has several limitations. Pre-intervention, NS patients' reasons for non-attendance were determined from interviews. There may be factors leading to NS that patients were not willing to discuss in an interview, such as purposeful avoidance of healthcare encounters or fearing bad news.34 During and after the intervention, patient satisfaction and interest was obtained from waiting room (attending) patients only. This may have biased the results and introduced confounding, as it represented the opinions of the patients who did attend their appointment. We may have obtained useful information by continuing to interview No-Shows after the start of the intervention, or conducting focus groups with non-attending patients.

Another limitation is that the improvement was not sustained after April 2015, and we hypothesize several reasons for this. Firstly, the number of patients exposed to the intervention was small: between 33-71% of patients sampled were enrolled in MyChart during the intervention. Signing up required multiple steps, despite attempts to improve this process (See PDSA cycles 1-3), and the email reminder intervention was initiated before maximal MyChart enrollment was achieved. This, and the technical difficulties with the reminder send-outs in February and March 2015, may have resulted in the time lag between the interventions and the improvement. In addition, attempting two simultaneous improvement processes made it more difficult to measure the true effect of the email reminders.

After technical issues were corrected, only 72% of MyChart users on average received the reminder. Spam filters, incorrect email addresses, or patients not checking their email may have been factors. A ‘read receipt’ feature used in some clinical communication tools35 would have been helpful in tracking this process. The astronomical data point seen in April coincided with 100% of waiting room patients receiving the reminder, suggesting that the intervention was effective when it did reach recipients.

In addition, the intervention was not targeted towards new patients, most of whom were not enrolled in MyChart. We started to see more new patients in clinic in May 2015, which could have further attenuated the effect of the intervention. We had difficulty contacting new patient NS for interviews, and further study into this population would help define improvement interventions for them.

One of the challenges in executing the intervention was achieving initial buy-in from administrative staff, who did not perceive NS to be an important problem in the clinic. We designed the intervention to require minimal extra work from staff and held meetings to discuss progress, present data, and obtain feedback, which appeared to increase buy-in and enthusiasm. A lesson learned would be the importance of involving staff earlier on in the planning process.

Finally, electronic reminders are applicable to other chronic disease outpatient settings,28 ,35–39 but may be less generalizable for clinics not requiring chronic disease follow-up (e.g. surgical care), for clinics without access to an electronic health portal or Information Technology support, or for clinics treating populations with less access to the internet.

Conclusion

This is the first reported electronic appointment reminder intervention to be tested in rheumatology outpatients. From our baseline measurement, we learned that forgetfulness and date mix-ups were the most common reasons for missed appointments. Our baseline NS rate was 6.8%, which is lower than reported in the literature,14 although we did not count patients who cancelled at the last minute. Despite this, the cost of NS was approximately $20,000 per year, in addition to the cost rebooking patients and added medical costs from urgent healthcare visits, making it a significant problem worth addressing.

Our aim was to decrease non-attendance by 50% by April 30 2015. A secondary aim was to increase health portal enrolment and improve patient satisfaction. Our interventions included a follow-up card, a number of small interventions to increase enrollment in the hospital's online patient portal, and finally e-mail reminders. We measured number of patients between NS, which is appropriate in the case of rare events, and conducted waiting room surveys. Our results show that the email reminder intervention was associated with improved attendance in April 2015, although this improvement was not sustained, which likely related to too few patients being exposed to the intervention.

Spread was achieved by sharing the results of the intervention with the rheumatology division, which led the email reminder strategy to be adopted by 3 other rheumatologists' clinics. The piloted project was also shared with other hospital divisions through a research day symposium and at a hospital-wide ‘patient experience exposition’. The email reminders have been continued with minimal effort from administrative staff. New strategies will be needed to increase portal enrolment and effective email delivery.

Although previous studies showed improvement in attendance with phone reminders,19 ,22 ,23 ,29 this was not feasible for us as there was little buy-in from stakeholders, citing cost and lack of resources. There are two examples of successful email reminder interventions in the literature.24 ,26 One center had phone and mail reminders already in place and they monitored the intervention for a longer period of time.24 Another group sent the reminder 3 days before the appointment rather than the week prior.26 This has scheduling limitations if a patient calls to cancel at short notice, but may be more helpful for forgetful patients, making it a possible avenue to explore in the future.