Problem

The two Lead Providers of Postgraduate Medical Education in North West London, Imperial College Healthcare NHS Trust (ICHT) and Central and North West London Mental Health Foundation Trust (CNWL), have assumed responsibility for ensuring that medical trainers are competently trained. Assessment of the training of trainers showed that:

1. Too few trainers were being trained to the required standards.

2. There was widespread local dissatisfaction with existing Train the Trainers provision.

3. There was evidence to indicate poor Trainer performance.

Background

Professional development of medical trainers has been recommended by the London Deanery and is now a requirement of GMC revalidation. The London Deanery Professional Development Framework (PDF) incorporates 7 domains of competence and is compatible with the GMC requirements (London Deanery, 2012: see Figure 1). The aim is to ensure all medical trainers are appropriately skilled to enhance the quality of postgraduate training and, in turn, attract high calibre trainees who become well trained. Effective training will improve patient safety and quality of service, and avoid sanctions for poorly performing organisations such as the removal of training status.

Figure 1: 7 Domains of the London Deanery Professional Development Framework (London Deanery, 2012)

London Deanery PDF Framework

1. Ensuring safe and effective patient care through training

2. Establishing and maintaining an environment for learning

3. Teaching and facilitating learning

4. Enhancing learning through assessment

5. Supporting and monitoring educational progress

6. Guiding personal and professional development

7. Continuing professional development as an educator

Local education provider Trusts in London had been encouraged and supported by the London Deanery to build a database of clinical and educational supervisors and their training in order to comply with the GMC's Standards for Trainers (GMC, 2010). Training was accessed from a number of sources. Some supervisors had accessed training provided by the Deanery and other providers, some local education providers had brought in training packages for their staff, and some had developed in-house programmes.

Baseline Measurement

The assessment by the Lead Providers demonstrated that

1. Too few trainers were being trained to the required standards. The 2011 and 2012 London Deanery census of training showed that coverage fell short of the target of 100% attendance at training in all 7 domains of the Professional Development Framework. Most aspects of required training achieved coverage of 30-80%, training of Educational Supervisors being higher than for Clinical Supervisors. In outlier cases, training attendance in some domains was 0% and in some 100%.

2. There was widespread local dissatisfaction with existing Train the Trainers provision. This was evident from feedback in the sector medical educational network where it was reported that uptake of training had been patchy, there was variable engagement of medical staff in the drive for educational excellence, and there were barriers to staff attending training away from their work place. Some consultants' experience of previous training was perceived to be too theoretical and not sufficiently based in the real-life clinical environment

3. There was evidence to indicate poor Trainer performance, with, for example, Red Flags on National Trainee Surveys. This was considered to be consistent with patchy training of trainers as above.

See supplementary file: ds1709.docx - “BMJ article v2 figures”

Design

The T4 programme target is for 100% of trainers to have attended training in the 7 domains of the PDF (London Deanery, 2012) and GMC requirements for approval of trainers (GMC, 2012) as a baseline, with evidence of ongoing professional development thereafter.

Stakeholder engagement has been crucial (see Figure 2 for summary of stakeholders). In 2011, the Lead Providers held focus groups across the sector to develop a way forward and secure buy-in. Two strong themes emerged. A course that met GMC standards, and a new model of delivery, was required; it also had to be locally responsive, clinically focused, integrative, sustainable and offer value for money.

Figure 2: T4 stakeholder map

Responding to this local feedback, T4 builds on 3 essential requirements for a future shape of training:

1. Clinical Leadership and a Collaborative Approach

The T4 design builds on an approach of "by Trainers for Trainers". The programme has been led and designed by experienced Trainers (Lead and Specialty Tutors) to ensure it is based in real-life and relevant to clinical practice. The team emulates collaborative leadership with close working between clinicians, the two Lead Provider organisations, educationalists from the London Deanery, and the sector wide Medical Education leadership network. It recognises the needs of the "expert learner" (Dreyfus & Dreyfus, 1986), utilising discursive and reflective teaching styles. It is designed as a 3 level course (see Figure 3). The content of Levels 1 & 2 is designed to address all 7 areas of the Professional Development Framework. The Level 3 programme, designed for experienced and aspiring local leaders in medical education, is flexible and will address needs responsively. The first workshop, for example, focuses on appraisal and revalidation as it pertains to medical education.

Figure 3: T4 Module Programme Design

1. Cross-Specialty Design and Participation

T4 has from the outset combined both Teachers and Learners drawn from Medicine, Surgery and Psychiatry to promote integration of the learning pathway. This brings many benefits - for example, networking, sharing experience and learning from different settings and specialties. The modules above can be taken in any order and attended at any of the local education provider sites across the sector. This further encourages cross specialty training, with consultants and higher trainees from all specialties attending any of the advertised sessions.

1. Local Delivery and Governance Networks

T4 has been delivered by the two local Lead Providers at all 11 Provider sites across the sector. To ensure coverage of training across this large geographical area, and at the same time maintain stakeholder engagement, a cascade model of training was adopted. This model, used favourably by the East of England Deanery and commended by the GMC (GMC, 2011), is embedded in the design to maximise sustainability. Factors known to influence a positive outcome of cascade training include: ensuring the training of cascade trainers is contextually appropriate; creating a milieu that supports post-training implementation (Weddell, 2005); providing leadership and accountability for the success of delivery and quality assurance (Draper et al 2009); the training of cascade trainers being experiential and reflective, with materials that are open to reinterpretation without expectation of rigid adherence to prescribed ways of working, diffusion of expertise throughout the system, and engagement of a cross-section of stakeholders in the preparation of training materials (Hayes 2000).

To date, 14 consultants and higher trainees have volunteered as Local Faculty Development Trainers. Transfer of skill has been cascaded through workshops, observation of Specialty Tutors, micro-teaching sessions, individual coaching, ready access to Specialty Tutors and educational bursaries. The Local Trainers exercise expertise in their service and are local champions for excellence in training. They provide a communication flow between the programme and local providers, and trouble shoot local barriers to engagement. Quality assurance will be provided through direct observation of sessions and ongoing participant feedback. Workshops will be held quarterly to develop the network, gain feedback on the experience of delivering training and to iteratively develop the training materials.

Strategy

Penetration of training coverage is continually monitored by the Lead Providers through attendance data provided to the employee provider Trust. There is regular 3 monthly reporting to a standing item for Faculty Development at the Lead Provider Committee including representation from all Providers and both Lead Providers. A plan of sessions and delivery sites is developed every 3 months. Each plan takes into account recorded demand for each module, waiting lists for attendance, popularity of sites, and feedback from the provider Trusts about training needs. Attendance by specialty is also reviewed, the Specialty Tutors trouble shooting wherever attendance is low in a particular group. This has, for example, led to training modules being delivered as part of specialty meetings in order to facilitate attendance. The modules are delivered as half-day sessions in order to facilitate clinical work for the other half day. However, in response to feedback, some whole day sessions are now provided.

In addition, attendance by grade is reviewed every 3 months. Sixty seven percent of attendees have been consultants, the remainder higher trainees, thus creating the educational workforce of the future. The high consultant workforce attendance was achieved by engaging with our Training Programme Director (TPD), Medical Education Managers (MEM) and Director of Medical Education (DME) network across the sector, with recruitment assisted by their postgraduate departments.

Content of the modules and delivery methods have also evolved through an iterative process to ensure the training is meeting the needs of the trainer workforce. This is achieved through 3 monthly reviews of participant feedback in addition to observations from the independent coaches, a survey evaluation conducted in July 2012, and an external evaluation of Level 1 undertaken in August-October 2012. This has led to changes in content with more information available on-line, more emphasis in-session on interactive delivery and the modelling of a dynamic learning environment.

Results

Outcomes of the programme have been evaluated by attendance numbers, session feedback, a survey of attendees at Level 1 modules, and external evaluation of Level 1.

Evaluation of penetration has been extremely positive with 834 attendances in 11 months, exceeding the 12 month project target of 350 attendances. Design and delivery commenced with Level 1, modules 1 & 2 (see Figure 4).

Figure 4: T4 Attendance May 2012 - March 2013 Figure 5: T4 Attendance by Specialty Cluster

Attendance has also been achieved from across all specialities as summarised in Figure 5.

Evaluation of content May 2012 to March 2013 via participant feedback demonstrates Level 1 (modules 1 & 2) have 87% & 93% rating as "good\excellent" respectively, and Level 2 (modules 3 & 4) content rating 96% and 98% "good\excellent". Quality of delivery of the two levels overall has rated 89-98% "good\excellent".

The above findings demonstrate high engagement and satisfaction with the training, achieving the aim of improving uptake and focus on medical education excellence across the sector. Further evidence of engagement can be drawn from the volunteering of 14 consultants and higher trainees to join the programme as Local Faculty Development Trainers.

Impact on quality of training has been assessed by changes the trainer participants have stated they have made. This has mostly related to learning styles and the learning environment. Indicative comments include: "Think more about training as a process that should be flexible/adaptable but dogmatically done"; "More structured plan for learning"; "Ensure I make time for trainees"; "Find ways to engage the trainees with the rest of the team in innovative ways"; "Many things around induction"; "Adapt and learn from trainees and their styles". In the on-line survey of attendees (July 2012), responses from trainers evidenced improved confidence, improved knowledge of educational roles, improved clinical supervision skills, and increased skill in recognition of the trainee in difficulty, careers advise resources, use of feedback and completion of workplace based assessments.

An external evaluation of Level 1 modules using in-depth interviews showed that new trainers were excited by the programme and emerged with many ideas about how different trainees learn. The changes in behaviour they reported included enhanced supervision skills, use of reflection as a learning method, and the value of goal setting. The training was also reported to have provided a valuable space for moderation of practice.

See supplementary file: ds1710.docx - “BMJ article V2 Figure 4 & 5”

Lessons and Limitations

Regular review of module content, delivery and attendance is needed to ensure the programme remains responsive to need. This has taken place as above with input from participant feedback, live-observation of sessions, consultation with the Deanery Faculty Development team and experience of the Specialty Tutors. The modules will continue to be developed iteratively with increasing input from the Local Faculty Development Trainers as they gather experience of session delivery.

Establishment of and communication with an educational network is required to shape and drive the uptake of training. As stated in a recent review of networks in healthcare (Randall, 2013), networks are useful for delivering services in ways that make the most of network members' capability and resources, in this case defining and delivering training standards. Ongoing communication and awareness of the programme in the network will be essential to its sustainability. It is anticipated that developing the voice of trainees within the Local Trainer group will assist by providing focus on the purpose and importance of Faculty Development.

The cascade of skills and support has proved invaluable resulting in considerably improved teaching skills, coaching skills and support across a network of trainers. Repeat census data is not yet available to provide comparison from the baseline. The London Deanery 2013 Census of Faculty Development training is awaited. The T4 attendance data would, however, indicate there will have been improvement in uptake of training.

Whilst the programme has developed with considerable energy and enthusiasm in the first 12 months, maintaining consistent quality of delivery and the commitment and drive of the Faculty team will require attention. This is particularly relevant as we move into a new phase of delivery with extended membership and a more diffuse group of trainers. Focus will be needed on the structure, leadership and management of the Faculty network (Randall, 2013).

The ultimate test of improvement is the experience of trainees. Whilst this is difficult to capture in a meaningful way, the GMC Trainee Survey data will provide an indicator of change and will be reviewed year on year. Further development of quality metrics are needed to identify the impact and complexity of Faculty Development interventions. The use of Kirkpatrick's model for evaluating educational outcomes might provide a way forward (Kirkpatrick, 1994).

Conclusion

T4 has demonstrated that Lead Provider partnership initiatives can improve quality of training through creation of sector-wide educational networks. One outcome that we underestimated was the enthusiasm for and usefulness of the network. We intend building on this. Making training relevant through design and delivery by clinicians, and easily accessible by delivery at the work place, enhances the engagement of medical trainers.

References

Draper T, Roots S and Carter H (2009). Safeguarding Adults: Perspectives from Primary Care Trusts in Kent and Medway. Journal of Adult Protection, 11, (3), 6-11.

Dreyfus H and Dreyfus S 1986 Mind Over Machine: The Power of Human Intuition and Expertise in the Era of the Computer. New York: Free Press

General Medical Council (2010), "Standards for Trainers", available at: http://www.gmc-uk.org/Generic_standards_for_training.pdf_31300576.pdf

General Medical Council (2011), "Quality Assurance of the Foundation Programme: Report on East of England Multi-Professional Deanery", available at: http://www.gmc- uk.org/East_of_England_Multi_professional_Deanery.pdf_41279220.pdf

General Medical Council (2012), "Recognition and Approval of Trainers", available at: www.gmc-uk.org/education/10264.asp

Hayes, D (2000). Cascade Training and Teachers‘ Professional Development. English Language Teaching Journal, 54, (2), 135-145

Kirkpatrick, D L (1994). Evaluating Training Programs: The Four Levels. San Francisco, CA, Berrett-Koehler Publishers

London Deanery (2012), "Professional Development Framework for Supervisors", available at: http://www.faculty.londondeanery.ac.uk/professional-development-framework-for-supervisors/Professional%20Development%20Framework%202012

Randall S (2013). Learning report: Leading networks in healthcare. The Health Foundation, London. Available at: http://www.health.org.uk/public/cms/75/76/313/4003/Leading%20networks%20in%20healthcare.pdf?realName=UwkTta.pdf

Weddell M (2005). Cascading Training Down into the Classroom: The Need for Parallel Planning. International Journal of Educational Development, 25, 637-651

Declaration of interests

Nothing to declare

Acknowledgements

The T4 programme was initially sponsored by the London Deanery. The Faculty Development team at the London Deanery have been of huge support in developing this programme. The input of Dr Catherine O'Keefe and Dr John Launer, in particular, has been greatly valued.