Article Text
PDF
Abstract
Improving the quality of healthcare delivery is increasingly a global health priority. However, quality improvement training opportunities that provide theoretical foundations and basic skills for patient safety and other quality initiatives have been limited or historically out of reach, especially in low-and-middle income countries (LMICs). To address this gap, the Harvard Initiative on Global Health Quality (HIGHQ) created and launched a massive open online course (MOOC) in 2014 focused on patient safety and quality of care using the edX platform. More than 30 000 students from across 195 countries registered for the online course. This paper summarises an innovative educational partnership between the course team and one of these countries, Rwanda, to develop a blended-learning model to bolster participation in this new course among Rwandan healthcare professionals. Although a small country, Rwanda was among the top performing countries for attracting course registrants and was the leading country for the proportion of enrollees who ultimately completed the course. Further, half (21 of 42) of Rwanda’s district hospitals opted to appoint a PH555x course facilitator at their site to help lead regular meetings and discussions about the course content at their facility. The majority of Rwandan enrollees were health professionals (63%) and 81% reported that PH555x was their first experience taking an online course. Among those participating in the ‘flipped’ component at hospital sites, 94% reported that the course helped them to think of specific ways to improve healthcare quality at their facility. In this paper, we describe this innovative public–private educational model, challenges to implementation and lessons learned that may be helpful for future MOOC developers who wish to augment learning opportunities among healthcare professionals in LMICs.
- health professions education
- patient safety
- healthcare quality improvement
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Improving healthcare quality is central to strengthening health systems and improving the health and well-being of patients worldwide.1 2 However, educational opportunities for health practitioners have been limited on this topic, especially in low-and-middle-income countries (LMICs). One increasingly used approach to address broader education access gaps in LMICs, including those for healthcare professionals, is massive open online courses (MOOCs).3–7 There is some controversy regarding how effective they have been in ensuring educational access to people who may benefit most.5 7 Yet in resource-limited settings, MOOCs have the potential to allow learners to engage with previously inaccessible content and can facilitate innovation in ‘technology-enabled pedagogy’ and other ‘blended learning’ approaches.3 5 7
To expand educational opportunities for healthcare providers, policymakers and patients on the fundamental principles of healthcare quality, Harvard University launched an edX course entitled ‘Improving Global Health: Focusing on Quality & Safety’ (PH555x) in 2014.8 This MOOC, hosted through HarvardX and created by faculty from the Harvard Initiative on Global Health Quality (HIGHQ), was the first of its kind to provide a freely accessible overview of quality of care for a global audience. In addition to the online course, the Rwandan Ministry of Health and HIGHQ partnered to augment the PH555x course modules with a blended learning educational programme for clinical practitioners based at district hospitals throughout Rwanda.
In this summary, we provide a detailed overview of the PH555x course along with overall enrolment and completion rates globally. We then describe this novel public–private academic partnership between PH555x and the country of Rwanda to both broaden awareness of the course nationally, while also developing a mechanism to host in-person course discussion groups among healthcare professionals from district hospitals. We then summarise lessons learned from implementing the PH555x Rwanda Programme that may be beneficial to other initiatives that wish to explore this type of model.
Course description
PH555x was an 8-week course organised into eight modules covering a variety of domains related to healthcare quality (table 1). Each module was released on a weekly basis and consisted of a set of 3–7 min core videos that feature a guided conversation between course faculty and guest experts, which addressed each module’s learning objectives. The videos were augmented by required and recommended readings, discussion boards, multiple choice questions and live Twitter sessions with course faculty. There were no prerequisites to enrol in the course and it was freely available through the edX platform. In terms of assessment, a student’s completion rate was based on homework (eg, multiple choice questions assessing knowledge from readings and video content integrated throughout each module) (60%), self-rated participation (eg, verifying contributions to discussion boards on a weekly basis) (20%), and a final writing assignment (20%). Students who achieved a score of at least 60% or above were eligible for a course completion certificate. Of note, there was no separate assessment scheme for the Rwanda PH555x facilitated sessions as described further below; course completion was based on meeting the aforementioned course requirements as captured on an individual edX account.
Weekly course structure, featured experts and module learning objectives
While basic principles of quality hold across contexts, distinct barriers to assessing and improving quality have been identified in LMICs.9 It was therefore imperative for the course to involve experts in the measurement and improvement of quality in such settings. In particular, Rwanda—a country of 12 million people in East Africa—was identified as a national case study for the PH555x course given the health sector’s well-documented health gains over the past two decades.10 11 Despite being devastated in 1994, Rwanda’s health sector emerged as an outlier in their achievement of health-related Millennium Development Goals, including those focused on maternal health, infant mortality and the burden of HIV/AIDS.12–14
Launching the Rwanda-Harvard PH555x collaboration
When making plans to feature Rwanda’s Minister of Health as a guest expert in the overall PH555x course, this spurred conversation regarding how to optimise the availability of the PH555x content to a key audience of interest: health professionals at Rwanda’s district hospitals. One of Rwanda’s rural district hospitals had prior experience in leveraging a ‘flipped classroom’ model15 through a separate edX course, which focused on the social determinants of health.16 The Ministry of Health (MOH) wished to build on this experience and engage health practitioners more broadly with the content proposed by the PH555x course as this was well aligned with the country’s health sector priority to build capacity in quality improvement among its healthcare professionals.
Similar to other blended learning approaches designed to promote capacity building, the Rwanda-Harvard PH555x collaboration sought to complement free online content with high value discussion groups at the hospital level, promoting context-specific learning.3 4 7 The aim was to foster opportunities for engagement with the online supplementary course material in a more learner-centred model that allowed for multidirectional discussion between different types of clinical practitioners working to improve the quality of care in Rwanda.
Supplemental material
To achieve this desired engagement, prior to the PH555x online course launch, the MOH sent an official invitation to all individuals working in Rwanda’s health sector to encourage them to enrol. Further, the MOH specifically asked the leaders of its 42 district hospitals to consider naming an active member of the hospital’s Quality of Care Committee as the site facilitator. The facilitator was asked to promote the course to hospital staff, coordinate optional, in-person discussion groups that would allow for members at that hospital to discuss the course content together, and arrange relevant local speakers or experts to expand on the content and discuss ongoing quality improvement programme or challenges at their site. To mitigate anticipated barriers to participation for clinical leaders, approval was obtained from Rwanda’s Medical and Dental Council so that all who received a course completion certificate were eligible to obtain Continuing Professional Development (CPD) credit.
To support the initiative, a PH555x Teaching Fellow with graduate level training in public health and health policy served as a liaison with the MOH and all district hospital PH555x facilitators. Funding was available through edX to support the Teaching Fellow’s time in supporting the overall course discussion boards (as is generally done across other edX courses). For the Rwanda programme in particular, the Teaching Fellow also supported the hospital course facilitators by distributing weekly sample session guides (online supplementary file 1), sending reminders regarding course milestones and requirements, sample discussion prompts and helping connect learners with any edX platform-related questions to the edX technical staff. All of these electronically available materials, which were provided both in email as well as through a shared drive folder to all named facilitators, were provided with the expectation that facilitators would customise the session to add value and context specific to their clinical setting and participants.
Overall, enrollees from Rwanda had two potential ways of participating in the PH555x course: (1) they could opt to participate in the freely available edX course on their own time, as was true for any other enrollee, and thus had no exposure to the discussion groups facilitated at the district hospitals, or (2) participate in hospital-based discussions as organised by the site facilitator throughout the 8-week course and complete whatever aspects of the programme that interested them.
Programme outcomes
For this article focused on an innovative public–private educational partnership, the key outcomes of interest include overall PH555x course enrolment and completion rates globally, which serve as a proxy for engagement with course content. In addition, for the Rwanda initiative specifically, we summarise district hospital participation levels within Rwanda as well as characteristics of participants who were in communication with the site-based facilitators.
Overall course enrolment and completion rates
A total of 30 150 students from 195 countries enrolled in PH555x for the course’s inaugural launch in 2014. The majority of these enrollees came from the United States (USA) (n=7635). Rwanda ranked 13th of all countries with the number of registrants (n=396) (table 2) and first in terms of ratio of registrants to overall population.
Top 20 list for course registration, completion and population size, by country
Among countries with the greatest number of course enrollees, Rwanda ranked first worldwide in terms of the proportion of enrollees who received course completion certificates based on the assessment guidelines noted above. A total of 45.2% (179 of 396) of Rwandan students completed the course (table 2). Since some enrollees may have never started the course, however, we examined completion rates among those who ‘viewed’ the course at least once. Rwanda’s completion rate on this metric (percent of students certified who viewed the course) was 61.7% (179 of 290). As a comparison, only 14.1% of those who registered and viewed the course at least once from the USA actually completed it. Rwanda’s completion rate on either metric is markedly higher than the typical completion rate for edX courses and for the majority of MOOCs overall (generally less than 10%).17 Of note, some countries not featured in table 2 had higher completion rates among students that viewed the course (ie, Armenia (n=15 students; 86% completion rate)); however, these countries had only a small number of enrollees in the course overall (<20 students).
District hospital participation and Rwanda programme enrollees
The Rwandan MOH invited all of its 42 district hospital leaders to promote widespread enrolment in the course throughout Rwanda, as well as consider naming a course facilitator at their site (figure 1). In total, half of Rwanda’s district hospitals (21 of 42) opted to name a PH555x course facilitator. In terms of activities, some facilitators hosted weekly gatherings with local speakers, food and other discussion-based learning experiences. Other sites met together only a few times over the duration of the course, but facilitators would regularly engage their colleagues by sending weekly course reminders through email and other text-based communications (eg, What’sApp, SMS).
Location of Rwandan registrants in the PH555x course.
A total of 140 individuals completed an anonymous pre-course survey that was distributed by Rwanda’s hospital-based facilitators to their contacts who were considering participating in the discussion groups at the hospital sites. The majority of respondents (63%) were healthcare providers, yet participants spanned a variety of disciplines, including hospital administrators (8%), researchers (6%) and data analysts (3%). The majority (81%) of respondents indicated that PH555x was their first exposure to an online course.
At the conclusion of the course, a total of 47 individuals completed the anonymous post-course survey, which hospital facilitators circulated to their contact lists. It was not possible to delineate if these individuals actually participated in the hospital-based discussions, but all had at least interacted with the course to some degree. Among these post-course survey respondents, 94% agreed (with 81% strongly agreeing) with the statement that the course helped them to think of ‘specific ways to improve the quality of care at my workplace’. In terms of next steps, qualitative comments from both individual participants and the facilitators suggested that the majority of participants hoped to continue their discussion groups at their hospital, engage with more trainings related to specific quality improvement projects at their healthcare facility and exchange in dialogue regarding quality improvement with other hospitals throughout the country.
Lessons learned
The successful enrolment and completion rates among Rwandan enrollees in PH555x—regardless of whether or not they participated in a discussion-based group at a district hospital—suggests the value of identifying innovative public–private educational partnerships to broaden access to the basic tenants of quality improvement. In table 3, we summarise key facilitators and barriers to optimising engagement among Rwandan healthcare professionals in PH555x.
Barriers and facilitators to optimising PH555x course participation in Rwanda
Regarding the educational partnership between HIGHQ and Rwanda’s MOH, the following underlying principles were helpful for ensuring the launch of this collaboration.
Alignment with national priorities and standards
First, the content of PH555x was relevant to Rwanda’s national policy priorities to improve quality and safety in the health sector. Concurrent to the course launch, district hospitals were actively engaged in Rwanda’s hospital accreditation process, which was a key tool for quality improvement discussed in the course.18 Although this was a motivating experience that had strong alignment with the course content, it would have been helpful to provide enrollees the option of participating in the course beyond the designated 8-week period as they faced this competing although complementary focus on accreditation. As such, future MOOC providers that wish to partner with the public sector to promote broadened access should identify ways that the course content is aligned with national objectives and can actively support existing policies and strategies, while also being sensitive to time demands on relevant stakeholders.
Leveraging online course content to promote local capacity building
Second, the objective of broadening the accessibility of knowledge—a foundational mission within MOOC learning and other blended learning models in LMICs7 15—has strong alignment with Rwanda’s commitment to building local capacity and promoting professional development within the health sector. Since each district hospital already had a quality committee at the time of the course roll-out, members from this committee were natural candidates to serve as facilitators and motivators for recruiting colleagues at their hospital to participate in the course. Further, physicians had the incentive to participate as PH555x was deemed eligible for CPD credit. The MOH wanted all participants to see the unique role that they—whether they be an administrator to physician—had in improving quality of care at their site. Fortunately, the post-course survey suggested that the course had this desired effect in helping Rwandan health practitioners complete the course feeling as though they had specific ideas on how to improve healthcare quality at their workplace. Although not within the scope of this article, future impact evaluation work would be helpful that demonstrates whether changes in hospital-based process and outcome quality measures occurred in those hospital settings that engaged with the course content versus those that did not.
Flexibility and pragmatism to reduce logistical barriers and to promote access to content
Third, as has been reported in other studies related to launching MOOCs in a resource-limited setting, it was important to be flexible to modify course content in real time in order to enhance access.7 16 A key logistical challenge was that of limited, regular internet connectivity at the district hospitals. Although Rwanda has a robust information and communications technology (ICT) infrastructure, a reliable, high-capacity internet connection was not universally available at all times and across all participating sites when the course launched in 2014. Among those hospitals that did participate, a number reported that internet connectivity was a barrier to accessing the course content as originally presented on the edX platform as the default system required more bandwidth than was commonly available. To mitigate this barrier, the HarvardX course team provided hospital facilitators with lower resolution videos, audio files, video transcripts, and—in the very few cases it was requested—flash drives containing all course content—so their colleagues could proceed through the course content over that 8-week timeline. Future MOOCs that wish to broaden access across settings that struggle with limited internet connectivity (not only in terms of availability but speed to download large files) should consider these sorts of creative alternatives to accessing content with their partners at the outset.
Encouragement of local experts to discuss their perspectives on a particular course topic
Fourth, this partnership encouraged greater buy in among enrollees by allowing for the local context to influence how the content was delivered. By encouraging district hospitals to host these discussion groups and customise the content related to that week’s course module, they were encouraged to coordinate meetings with key leaders at their hospital site to discuss current, locally relevant updates that directly related to the module of interest. For instance, for the session on electronic medical records, the facilitator at Rwinkwavu District Hospital invited their site’s information and technology experts to comment on this dimension of healthcare quality.
Limitations and next steps
Although we believe this PH555x Rwanda programme represents a novel public–private educational partnership to broaden desired healthcare quality content to Rwandan health practitioners, there are a number of limitations to consider. First, this programme was implemented in a single country that had prior experience (although less extensive) with an edX course,16 thus it is not possible to assume that this approach is generalisable across other settings. However, it is our hope that future MOOC developers with an interest in partnering with LMICs to expand health practitioner engagement in relevant course topics may find this model useful for their own experience and encourage them of the value of liaising with relevant governmental leaders when attempting a national roll-out. Second, we acknowledge that the outcomes of interest in this innovation education report, namely course enrolment and completion, cannot be assumed as markers of skill, knowledge or have an impact on improving the focus area of PH555x: healthcare quality. Although it does not capture the totality of all Rwandan participants and cannot fully be attributed to the hospital-based component of the course, it was nonetheless encouraging to see that the large majority of respondents felt that the course provided them with specific ideas on how to improve quality at their workplace. Yet, as previously mentioned, future research in this space should aim to measure actually differences in quality outcomes as a result of course participation. Third, in terms of evaluating the impact of the ‘flipped’ component of PH555x at the district hospitals versus non-exposed Rwandan participants, we were unable to analyse the differences between these potentially interesting comparison groups for a variety of reasons. This included the fact that the overall goal when creating this national programme was not to test learning models but rather to increase exposure to the course via any route possible. Further, the flexible and variable nature of participation at the hospital-based sites and lack of granular data among course completers precluded us from doing a post-hoc analysis of these subgroups. As such, future MOOCs that strive to engage LMICs in this type of blended learning model and wish to have a more precise understanding of the impact of their flipped model should aim to design the course in a way that captured the desired measures for this level of analysis.
In summary, a unique educational partnership between HarvardX and the Rwanda MOH helped to bolster health practitioner participation in an MOOC focused on patient safety and quality. This was accomplished by strategic promotion of the course throughout the health sector, as endorsed by the MOH, and efforts to coordinate a complementary hospital-based course discussion group programme that half of Rwanda’s district hospital sites opted to participate in over the 8-week course. Lessons learned from this partnership can be used to improve future initiatives that strive to bolster LMIC access to relevant course content. Further, this partnership demonstrates the potential for future MOOC courses to collaborate with the public sector to expand exposure to relevant knowledge and skills, including those focused on the goal of improving healthcare quality, to health professionals and beyond.
Acknowledgments
We wish to acknowledge Dr Ashish K. Jha, who was the faculty lead for PH555x and supported this partnership. In addition to Dr Jha, the authors would like to thank all PH555x course staff that helped make this course possible and also applaud facilitators who led these discussions within district hospitals throughout Rwanda. Lastly, we thank the students for their commitment to improving the quality of healthcare delivered in Rwanda and beyond. All figures and images were created or taken by authors and course facilitators and approved for publication.
Footnotes
Twitter @kirstinwscott
Contributors KWS and AB were involved with the initial drafting of the manuscript. KWS, LW and CA were involved with compiling summary data regarding the course. KWS, TD, VR and AB were involved with reviewing the Rwanda programme attributes. All authors were involved with the course implementation, manuscript revisions and overall review of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Map disclaimer The depiction of boundaries on the map(s) in this article does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. The map(s) are provided without any warranty of any kind, either express or implied.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval This is a descriptive study that does not meet the requirements of being human subjects research and thus was not required to undergo institutional review board approval. Survey data reported in this study come from optional, anonymous end of course evaluations that were part of general course quality improvement and are not considered human subjects research.
Provenance and peer review Not commissioned; externally peer reviewed.