Article Text

Download PDFPDF

Closing the gap: a transatlantic collaboration to foster quality improvement training in graduate entry medical students using applications of QI methodologies to medical education
  1. Allison Brown1,
  2. Seamus Sreenan2,
  3. Alice McGarvey2
  1. 1Community Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
  2. 2Graduate Entry Medicine, Royal College of Surgeons in Ireland, Dublin, Ireland
  1. Correspondence to Dr Allison Brown; allison.brown{at}ucalgary.ca

Abstract

The alarming prevalence of medical error and adverse events in the health system raises a call to action to ensure that doctors in training receive adequate training in quality improvement (QI). Training medical students in QI remains a challenge given time constraints, lack of clinical exposure, and already saturated curricula. In some instances, QI training may be delivered during clerkship through didactic, and in some instances, and experiential learning. Preclinical years of medical school remain focused on introducing students to scientific and clinical concepts, rarely do they learn about QI. The Program for Innovation in Scholarship and Medicine (PRISM) is a programme that introduces first-year medical students to the fundamentals of QI using their experience as a medical student as the context. PRISM is a condensed QI curriculum that is delivered through an international partnership, based on a previously piloted programme at a Canadian medical school. Following an introductory workshop, medical students work in teams to develop QI proposals (project charters) which detail how QI principles and tools can generate small-scale improvements within their educational programme. Project charters are assessed by a team of faculty and upper year students, who have previously participated. On completion of the programme, students demonstrated increased knowledge, skills, and attitudes towards QI. Programme participants were satisfied with the structure and expectations of PRISM and expressed a newfound interest in QI. Nearly all participants would recommend PRISM to another medical student. In conclusion, PRISM serves as a resourceful, efficient educational approach for preclerkship students that provides an introduction to the concepts of QI in order for early trainees to build on baseline knowledge and skills throughout their training.

  • medical education
  • quality improvement
  • curriculum evaluation

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Full Text

Statistics from Altmetric.com

Introduction

Studies replicated around the globe have described the alarming prevalence of adverse events and harm due to medical error. In 2016, the Irish National Adverse Events Study reported that 12.2% of patients experienced an adverse event, of which 70% were considered preventable.1 Concerningly, the economic burden of medical error in the Irish health system is estimated at over €194 million. In recognising that medical error and harm in healthcare is the result of a number of combined, complex factors, quality improvement (QI) is a well-established methodology that can reduce the prevalence of error and enhance the overall quality of care within health systems. Educators have acknowledged the importance of training current and future health professionals in QI and patient safety in order to influence organisational safety culture. In North America, the Canadian Medical Education Directives for Specialists (CanMEDS) and Accreditation Council for Graduate Medical Education (ACGME) have integrated QI into their competency frameworks.2 3 In the UK, the General Medical Council (GMC) is emphasising the need to incorporate QI into undergraduate medical education while it is also being incorporated into postgraduate frameworks throughout Europe.4 5

Educational strategies must be in place to ensure that all future physicians receive exposure to QI during medical school and postgraduate or residency training in order to acquire the foundational QI knowledge that permits an understanding of how to identify and implement changes for the advancement of patient care, patient safety, and overall performance of healthcare systems.6 7 This may be achieved through early and longitudinal integration of didactic and experiential QI training throughout undergraduate and postgraduate medical training. Historically, applied clinical projects have been successful in educating trainees about QI during their clinical years; however, these projects are infrequently available and resource intensive. For students who are in their preclinical stage of training, early engagement to QI is often difficult due to a lack of contextual understanding and exposure to the health system.7 Preclerkship learners spend their first year of medical school adjusting to the new environment and focusing on the acquisition of basic sciences and clinical content and therefore may not be at an appropriate stage to learn and apply principles of clinical QI. Despite their lack of exposure to medicine and health systems, junior medical students are perfectly situated within an already complex system in need of improvement: their own medical education.

PRISM Overview

The Programme for Innovation in Scholarship in Medicine (PRISM; formally known as the Programme for Improvement in Medical Education) aims to foster the early development of QI competencies among medical students by using education as the context for QI applications. Previous evaluations of the programme in a Canadian medical school revealed that first-year medical students were able to transfer their acquired knowledge from the context of education to clinical scenarios at levels comparable to postgraduate residents and that the early exposure to QI fostered later clinical engagement during clerkship.8–10

The Royal College of Surgeons in Ireland (RCSI) is an international degree-awarding health sciences institution, offering medical education programmes that range from 4 to 6 years, with the Graduate Entry Medicine (GEM) programme offering a 4-year medical degree (MB BAO BCh) to those who have already attained a degree. RCSI students come from a variety of backgrounds ranging from the arts to science, and the class size for the GEM programme is approximately 80 students per year. PRISM was first piloted in 2016 with students in the GEM Class of 2019. During the first cycle, PRISM was delivered over three consecutive days, two of which were on a weekend. However, in the two most recent cycles, the programme was condensed over two half-day workshops. Since 2016, four cohorts of GEM students have completed the programme. Despite being offered as an extracurricular programme delivered on the weekend, participation rates were very high over the first three cycles: 76%, 69% and 54%, respectively.

The PRISM workshops are delivered in-person by a Canadian scholar with content expertise in both medical education and QI methods. The first workshop focuses on introducing the fundamentals of QI, including the Institute of Medicine’s 6 dimensions of quality. The purpose of this introductory workshop is to encourage students to begin thinking about ‘quality’ using an established framework so that they can identify quality deficits in the education programme received to date. By the end of the first workshop, students are expected to work in teams to identify and define a ‘quality gap’ in their education. The second workshop focuses on the applications of QI methodologies, namely, the Model for Improvement. During this workshop, students focused on applying the Model for Improvement to their quality gap. Breakout sessions are held so students had time to work on various aspects of the Model for Improvement, including developing: an aim statement; a family of measures (ie, outcome, process and balancing measures); an appropriate intervention or change concept; and an implementation plan using Plan-Do-Study-Act cycles. During the breakout sessions, senior faculty members were available to provide contextual insight into the areas of the curriculum on which students were focusing their improvement efforts while the visiting scholar consults groups on QI concepts. An observed benefit of having the workshop delivered by an individual external to RCSI is that medical students appeared more willing to speak openly about strengths and weaknesses of the curriculum, even with lead faculty members present during the workshop. Further, each participant receives a programme handbook which serves as a resource both during and after the workshop.

Six to eight weeks following the PRISM workshops, the student groups are required to submit a project charter—a QI proposal—that outlines their educational quality gap and how they would use the Model for Improvement to test, implement, spread, and scale an intervention (online supplementary appendix A—PRISM Project Charter). Students are expected to apply QI concepts and terminology in the project charters to provide a detailed QI proposal, including an aim statement, measurement plan, and a detailed strategy for how they would use multiple Plan-Do-Study-Act cycles. Charters for each student group are marked by at least two faculty members and three to four upper year RCSI students who have previously completed the programme. Each charter is assessed by raters using a standardised scoring framework, the Project Charter Assessment Tool (PCAT), which generates mean scores that are then used to rank order the submissions and identify the top groups who are awarded a prize (online supplementary appendix B—PRISM PCAT). Each group receives formative feedback from both faculty members and upper year students that aims to reinforce QI principles and provide insight into the feasibility of their ideas.

During the inaugural offering, 100% (n=53) of participants completed the Beliefs, Attitudes, Skills and Confidence in Quality Improvement (BASIC-QI) instrument prior to and following PRISM.11 Cronbach’s alpha suggested high internal consistency, providing reliability evidence (pre=0.905, post=0.951). Overall, BASiC-QI scores increased 59.28 ± 23.5 following PRISM (p<0.001), suggesting that learner knowledge, skills and attitudes improved following completion of the programme. In addition, 66% (n=35) of students completed an exit survey following the completion of the programme to gather further feedback. 94% (n=33) of students agreed that the workshops were well organised and 94% (n=33) agreed that they obtained enough knowledge from the workshops to develop a project charter aimed at improving the quality of medical education at RCSI. 94% (n=33) agreed that the expectations of the programme were clear and manageable, while 89% (n=31) agreed that PRISM was a valuable learning experience. 91% (n=32) agreed they have a good understanding of what QI is and how to develop interventions that can improve a system. 88% (n=31) agreed they enjoyed learning about QI and have a new interest in QI and 94% (n=33) would recommend future students participate in the programme. Finally, 74% of students said they are more likely to engage in future QI opportunities if offered to them.

Discussion

Evaluations of PRISM at RCSI have suggested that the programme increases first-year medical students’ knowledge, attitudes, and skills towards QI by using education as a context for learning about QI. Students were satisfied with the programme’s expectations and delivery methods, denoting high satisfaction with the workshops. Students expressed confidence in their understanding of QI and a new interest in learning about QI. Nearly three-quarters of students expressed interest in participating in future QI opportunities.

The GEM programme at RCSI is comprised of approximately 60% North American students who traditionally have returned for residency training. The remaining 40% are European students, predominantly fromIreland . Following the first year of training, many students participate in extracurricular summer research programme and in projects that relate to health services research and QI for which they are now better prepared. Several students agreed to further their involvement in later years by serving as PRISM mentors. These upper year students provided mentorship to student groups prior to charter submission and then feedback to submissions regarding feasibility, alignment with the RCSI curriculum, as well as their adherence to QI principles. In this way, these upper year students began to develop their own abilities to mentor others in QI and in turn, reinforced their own developing QI expertise.

PRISM offers a platform to engage students in the evaluation and improvement of medical education. By providing students with a structured programme, students were able to identify system-level deficits in education from their own experience and use a systematic approach (ie, the Model for Improvement) to propose solutions that could improve educational quality. Thus, students have learned to use a constructive process to work with faculty members to develop ideas and suggest appropriate and efficient changes to improve quality. Although the PRISM charters are a simulation of the QI project the RCSI GEM programme has implemented a number of the students’ QI initiatives following each iteration. Examples of such charters are: implementation of a student well-being programme within the first year; review of how Radiology is taught alongside anatomy; introduction of a simple measure to enhance communication to students when a member of faculty is delayed or cancels teaching. The added bonus to such an initiative is the development of a partnership between faculty and students as they work together to improve the quality of education delivered using an approach which engages students as end users in the process.

PRISM offers medical students from various backgrounds a structured introduction to scholarly activities that aim to improve the quality and safety of healthcare, providing them with the foundational knowledge to collaborate in QI initiatives early and throughout their training. Furthermore, ensuring that these graduates have received training in QI prepares them for residency programme and careers that emphasise QI during professional practice.

We have used QI principles in our efforts to improve the programme each cohort and have made changes in response to student feedback, including condensing the content from 3 days to 2 and integrating the programme into the curriculum in order to avoid running the course on the weekend as an extracurricular. The PCAT has also been modified to provide more structured feedback to groups. In 2019, PRISM has become integrated into the GEM programme’s core curriculum within its Evidence-Based Health module. In previous years, some students were unable to participate due to weekend conflicts. By integrating PRISM into the curriculum as a credit-bearing module, we expect to enhance the overall accessibility of the programme and provide all students with protected curriculum time for QI activities.

Limitations

The present study is limited in that it is a single site study of a homogenous student population and there is no comparison or control group given that the vast majority of GEM students elect to participate in this programming. This study is also limited in its ability to draw inferences about how PRISM influences medical student behaviours towards QI and patient safety during the remainder of their training, such as clerkship or residency. Furthermore, in recognising that many approaches for QI exist (eg, Model for Improvement, Lean, Six Sigma), PRISM only teaches one method. However, the Model for Improvement has remained one of the most popular approaches to health system improvement and is the most commonly described in the literature for QI in health professions education.

Conclusion

Since the initial pilot in 2016, PRISM has educated over 200 students in QI using their own education as the context for learning meaning that the majority of RCSI GEM graduates are entering postgraduate training with QI exposure. The collaboration between Irish and Canadian medical schools has allowed for innovative ideas and approaches to spread internationally. We anticipate that other schools who wish to emphasise QI or curricular improvements would benefit similarly from this approach, providing an early solution to QI training in medical education.

Acknowledgments

The authors wish to thank the first-year students at RCSI who participated in the early offerings of PRISM, eagerly participating in the course over the weekends and providing rich feedback at the end of the programme to improve this educational offering for future students.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
View Abstract

Footnotes

  • Contributors AB, SS and AM planned and executed the evaluation of the work described in the article. AB conducted the data analysis. All authors assisted with the writing of the manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.