Article Text

Download PDFPDF

Questionnaire-based service evaluation of the efficacy and usefulness of SEREN: a structured education programme for children and young people diagnosed with type 1 diabetes mellitus
  1. Rashmi Sarah D'Souza1,
  2. Matthew Ryan2,
  3. Esther Hawkes3,
  4. Claire Baker4,
  5. Yvonne Davies5,
  6. Judith R-Screen6,
  7. Jonathan Price7,
  8. Rebekah Pryce8,
  9. Nirupa A D'Souza9
  1. 1Institute for Women's and Children's Health, King's College London, London, UK
  2. 2Department of Paediatrics, Swansea Bay University Health Board, Port Talbot, UK
  3. 3School of Medicine, University of Southampton, Southampton, UK
  4. 4Department of Nutrition and Dietetics, Aneurin Bevan University Health Board, Headquarters, St Cadoc's Hospital, Newport, UK
  5. 5Department of Paediatrics, Hywel Dda University Health Board, Ystwyth Building, Hafan Derwen, Carmarthen, UK
  6. 6Department of Paediatrics, Betsi Cadwaladr University Health Board, Bangor, UK
  7. 7Improvement Cymru, Public Health Wales, Cardiff, UK
  8. 8Consultant Endocrinologist, Department of Paediatrics, Noah’s Ark Children’s Hospital, Heath Park, Cardiff, UK
  9. 9Department of Paediatrics, Princess of Wales Hospital, Bridgend, UK
  1. Correspondence to Dr Nirupa A D'Souza; nirupa.dsouza{at}wales.nhs.uk

Abstract

Objectives To evaluate the usefulness and effectiveness of a new structured education module for children with type 1 diabetes: Structured Education Reassuring Empowering Nurturing (SEREN) ‘Diabetes at Diagnosis’.

Design Retrospective questionnaire-based service evaluation.

Setting 12/14 paediatric diabetes centres across Wales took part.

Participants Children diagnosed with type 1 diabetes 1 year before (pre-SEREN group) and 1 year after the introduction of SEREN (post-SEREN group) were selected using a national diabetes register.

Resource ‘Diabetes at Diagnosis’ delivers structured education to empower children and families with self-management of type 1 diabetes.

Evaluation Primary outcomes were patient-reported effectiveness and user-friendliness of the educational resources and quality of life (PedsQL). Age-appropriate child and parent questionnaires were provided. Clinical outcomes included glycated haemoglobin (HbA1c) at 6 and 12 months, service engagement and diabetes-related hospital admissions in the first year.

Results 89/106 responded pre-SEREN and 108/115 post-SEREN, with no demographic differences at diagnosis. Parent scores for educational package evaluation significantly improved post-SEREN, with a non-significant trend towards improved results in children. PedsQL scores were similar. There was no change in HbA1c overall. Subgroup analyses at 12 months showed a trend towards a lower HbA1c in key stage 1–2 (62 vs 58 mmol/mol, p=0.06) and increased HbA1c in key stage 3–4 (56 vs 66 mmol/mol, p=0.009). There were no differences in hospital admissions or missed clinic appointments.

Conclusions This is an evaluation of the only standardised type 1 diabetes structured education programme in use for children throughout Wales. This module improved parent-reported outcomes and showed a non-significant trend towards improved usefulness in children, without a difference in a PedsQL scores overall. Ongoing evaluation of the cohort who received subsequent SEREN modules may show the long-term benefit of the programme.

  • paediatrics
  • patient education
  • diabetes mellitus
  • evaluation methodology
  • patient reported outcome measures

Data availability statement

No data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

The incidence of type 1 diabetes mellitus (T1DM) has been relatively stable between 2013 and 2019, at around 24.5 per 100 000, highest in the 10–14 years age group, with an overall prevalence of 196 per 100 000 in England and Wales.1

In 2018–2019, under 11% of children and young people (CYP) with T1DM achieved a glycated haemoglobin (HbA1c) <48 mmol/mol (<6.5%)—the nationally defined target for ‘excellent’ control, and 6.9% of CYP with known T1DM were admitted to hospital at least once for a diabetes-related reason.1 Also, the rate of decrease in HbA1c appears slower in the UK, compared with other European countries where structured education is integrated into care at diagnosis.2 3

Health education and awareness are key to improving management of long-term conditions, with established benefits of structured education programmes for diabetes in adults.4 5 National and international guidelines support diabetes education as an integral part of therapy and it is considered a cost-effective intervention which reduces complications.6 7 While up to 72% of CYP with T1DM in the UK receive structured education, there is significant geographical variation—an area highlighted for improvement to reduce national variation in outcomes.8

Structured Education Reassuring Empowering Nurturing (SEREN)9 is a structured education programme for CYP diagnosed with T1DM. It was rolled out nationally in Wales in 2016–2017 and is designed to: address the core principles in structured education for T1DM according to The International Society for Paediatric and Adolescent Diabetes (ISPAD) and National Institute for Health and Care Excellence (NICE) guidelines3 6; be practically accessible in terms of content, delivery and resources for children across Wales; and be delivered in a consistent manner incorporating educator training and quality assurance.

Prior to the introduction of the SEREN programme, diabetes education across Wales was locally led. Resources and delivery, including content and duration, varied widely across units and was not structured. Therefore, it was challenging for previous education systems to be easily transferable, replicated or evaluated.

The SEREN structured education programme

A working group was created, comprising paediatricians, dietitians, diabetes specialist nurses and a psychologist, with an interest in T1DM and working in paediatric diabetes services across Wales. Parent representatives were involved at all stages of module development. The group set out to achieve the aims above with the philosophy of ‘empowering the CYP and family to manage diabetes from diagnosis right through and including transition to adult services’. A curriculum was developed in line with these core components, forming lesson plans and learning resources aligned to the national educational Key Stage (KS) targets (figure 1). The first SEREN module ‘Diabetes at Diagnosis’, delivered as face-to-face sessions, includes understanding the pathophysiology of T1DM, carbohydrate counting, insulin dose adjustment, managing hypoglycaemia, sick-day rules, management of diabetic ketoacidosis, complications and impact of exercise. Delivery aims to be flexible, matched to educational ability and at a pace guided by the family within 6 weeks of diagnosis. A quality assurance strategy was embedded into the education programme design: each educator is observed delivering SEREN during their first year, with ongoing reviews on a 3 yearly basis.

Figure 1

SEREN learning resources for the Diabetes at Diagnosis module.9 SEREN, Structured Education Reassuring Empowering Nurturing.

The SEREN programme is endorsed and supported by the CYP Wales Diabetes Network and recognised with a Quality in Diabetes Care award (2017). ‘Diabetes at Diagnosis’ has been accredited by the Quality Institute of Self-Management for Education and Training in August 2020.

Aims

We aimed to evaluate the impact of the first module, ‘Diabetes at Diagnosis’, on CYP and families.

In line with ISPAD guidance, two specific areas were evaluated:

  1. Patient-selected diabetes-care goals including psychosocial adaptation, CYP and family confidence and self-efficacy.

  2. Disease outcomes such as glycaemic control, engagement with healthcare and need for hospitalisation in the first year after diagnosis.3

Methods

This was a retrospective national questionnaire-based evaluation of SEREN ‘Diabetes at Diagnosis’. It assessed the impact of the first module of a standardised structured education package for CYP newly diagnosed with T1DM in Wales.

SEREN ‘diabetes at diagnosis’ module

The SEREN working group, including parent representatives, designed the programme and learning resources that were aligned to educational KS. In the UK, educational KS 1 refers to CYP aged <7 years, KS 2 to 7–11 years, KS 3–4 to >11 years. Each KS group resource pack included a curriculum, workbooks and educator records for CYP and teaching resources for the facilitator. Workbooks contained bite-size lessons to achieve educational goals with objectives and outcomes, key points and exercises. Module resources were available in English and Welsh.

‘Diabetes at Diagnosis’ was rolled out to CYP newly diagnosed with T1DM across 14 units in all 6 Welsh health boards in two stages: KS 3–4 package from 4 January 2016 and KS 1–2 package from 6 January 2017.

The module was delivered by paediatric diabetes specialist nurses and dietitians via face-to-face sessions of 1–2 hours each, over a 6-week period. Facilitators were trained in the use of this module with follow-up training provided. Each session was delivered on a one-to-one basis with the CYP and family.

Participants

CYP were identified by date of diagnosis from the Brecon Group register. This Welsh population-based national register containing demographic and clinical details has captured 98% of CYP with a new diagnosis of Diabetes since 1995.

The control group, defined as the ‘pre-SEREN’ group, were identified as CYP diagnosed in the year prior to SEREN’s introduction, and the intervention group, defined as the ‘post-SEREN’ group, consisted of CYP diagnosed in the year after its introduction. A wash-out period of 2 months either side of SEREN introduction dates was incorporated to allow for variation between units.

For CYP in KS 1–2, the module was introduced on 6 January 2017, and evaluation groups selected as follows:

  • The pre-SEREN KS 1–2 group were diagnosed between 4 January 2016 and 4 January 2017.

  • The post-SEREN KS 1–2 group were diagnosed between 8 January 2017 and 8 January 2018.

For CYP in KS 3–4, the module was introduced on 4 January 2016, and evaluation groups selected as follows:

  • The pre-SEREN KS 3–4 group were diagnosed between 2 January 2015 and 2 January 2016.

  • The post-SEREN KS 3–4 group were diagnosed between 6 January 2016 and 6 January 2017.

In order to validate the pre-SEREN and post-SEREN classification, a spot check was performed in one of the larger health boards. This involved reviewing the patient list for this unit and cross-checking with local patient records to establish whether each CYP had received SEREN education or not. This check showed that the CYP had all been correctly categorised.

CYP with non-T1DM were excluded, as were those who had reached their 17th birthday before the beginning of data collection, due to the likelihood of having moved to adult services.

Evaluation

The evaluation phase was conducted from December 2018 to September 2019 across Wales.

The evaluation pack included age-appropriate information leaflets and questionnaires for CYP and families, designed by the SEREN working group. Evaluation packs were distributed to families by local healthcare practitioners independent of SEREN and local diabetes services, in order to reduce reporting bias. The information leaflets provided an overview of the SEREN programme, and included the purpose of the evaluation and its relevance to CYP and parents.

Each pack included separate information leaflets for the CYP and the parent, two KS-appropriate questionnaires and two parent questionnaires; one each for the CYP and parent, designed to gather information about the usefulness of education received at diagnosis and one validated diabetes quality of life questionnaire each for the CYP and parent (PedsQL V.3.2).10 If the child was under 4 years of age at the time of evaluation, only parent questionnaires were administered.

Outcomes

ISPAD recommends that evaluation of structured education programmes should include measurement of outcomes directly related to diabetes education such as the patient’s achievement of self-selected diabetes-care goals, improved psychosocial adaptation and enhanced self-efficacy and glycaemic control.3 Therefore, the focus of outcome measures was evaluation of educational impact, specifically the empowerment of CYP and families, improving confidence and ability to cope with T1DM self-management.

Primary outcomes were impact on diabetes-related quality of life of CYP and families and the usefulness of the ‘Diabetes at Diagnosis’ module.

Secondary outcomes included HbA1c at 6 and 12 months, engagement with services (number of missed hospital appointments) and diabetes-related hospitalisation beyond the diagnosis period within the first year.

Data handling and statistics

Questionnaire responses were anonymised and identified by a unique study number so they could be matched to demographic details, HbA1c, clinic attendance and hospital admission data.

Statistical analysis was conducted by a medical statistician using χ2 tests for discrete data and Student’s t-tests for continuous data.

Results

Participants

All 14 centres across the 6 Welsh health boards were invited to take part. Twelve centres participated: one tertiary centre and two large, five medium and four small district general hospitals.

A total of 172 CYP (and family) in the pre-SEREN group and 162 CYP (and family) in the post-SEREN group were identified from the Brecon register at these 12 centres. A total of 113 CYP had non-T1DM or had their 17th birthday before the evaluation was conducted and were excluded. A total of 221 CYP (and family), comprising 106 in the pre-SEREN group and 115 in the post-SEREN group were invited to participate. Eighty-nine (84%) CYP and family pack responses were returned from the pre-SEREN group and 108 (94%) from the post-SEREN group. The CYP and their families were invited to complete separate questionnaires, which were analysed independently. A total of 106 completed questionnaires were returned in the pre-SEREN group and 156 in the post-SEREN group, with between 56% and 74% of CYP and family packs containing fully filled questionnaire responses (figure 2). The response rates for individual domains varied as not all CYP and families completed all the question domains. Within each questionnaire, completed domains were included in the analysis and missing domains were excluded. There was no difference in demographic variables between groups (table 1).

Table 1

Demographic characteristics of CYP in the pre-SEREN and post-SEREN groups

Figure 2

Flow diagram of the SEREN programme introduction, patient selection, exclusion and evaluation phases. CYP, children and young people; KS, key stage; SEREN, Structured Education Reassuring Empowering Nurturing.

Outcomes

There was a significant increase in positive feedback scores from the parents across the following domains: usefulness of resources; ability to manage; coping with diagnosis and understanding of diabetes (p≤0.01) with a trend towards increasing positive results in CYP in the post-SEREN group (table 2, figure 3).

Table 2

Evaluation of resources used for diabetes education at diagnosis for CYP and parents before and after the introduction of SEREN ‘diabetes at diagnosis’ module

Figure 3

Evaluation survey of children before and after the introduction of SEREN diabetes at diagnosis. cyp, children and young people; SEREN, Structured Education Reassuring Empowering Nurturing.

Overall, there was no difference in CYP PedsQL scores between the groups; there was a trend towards improved scores from the parents in the post-SEREN group (p=0.05) (table 3).

Table 3

Quality of life scores, as measured by PedsQL, for CYP and parents with a new diagnosis before and after the introduction of SEREN ‘diabetes at diagnosis’ module

There was no change in HbA1c overall between groups at 6 or 12 months (p>0.05, (table 4)). However, subgroup analysis at 12 months for KS 1–2 shows a HbA1c of 62 (59–64) mmol/mol pre-SEREN vs 58 (55–60) mmol/mol post-SEREN (p=0.06) and for KS 3–4 shows 56 (52–61) mmol/mol pre-SEREN vs 66 (60–72) mmol/mol post-SEREN (p=0.009).

Table 4

HbA1c at diagnosis, 6 months and 12 months for CYP before and after the introduction of SEREN ‘diabetes at diagnosis’ module

Eighty-six per cent pre-SEREN vs 75% post-SEREN attended multidisciplinary clinic appointments (p=0.06), 11% vs 19% missed one appointment (p=0.17) and 2% vs 6% missed more than one appointment (p=0.22) respectively in the pre-SEREN and post-SEREN groups. There was no significant difference in hospital admissions between the groups—20% pre-SEREN vs 23% post-SEREN required admission in the first year post-diagnosis (p=0.62).

Discussion

This national evaluation of education at diagnosis showed that parent satisfaction significantly improved across all resource evaluation domains after the introduction of SEREN ‘Diabetes at Diagnosis’. The striking impact of the module from the parents’ evaluation as compared with that of CYP may reflect the fact that initially only parents were involved from the programme inception and in resource development. CYP are now involved to ensure the user friendliness of further modules and updates. There was no change in diabetes-specific quality of life scores for CYP and a trend towards an improvement for parents, particularly in the communication domain.

There was no change in overall HbA1c, clinic attendances or hospital admissions between the two groups. A limitation here is the use of HbA1c at 6 or 12 months, which may not reflect the impact of an intensive programme delivered in the first few weeks after diagnosis. We need to consider the possibility of waning of knowledge and motivation a year on from diagnosis. HbA1c may also be affected by the biopsychosocial effects of puberty, changes in schools and the confounding effect of the honeymoon period.11 Addressing this gap with ongoing education may improve HbA1c and therefore, the cost-effectiveness of the intervention.

The positive impact of structured education programmes on psychosocial and quality of life outcomes is widely reported,12–15 yet there appears to be no consistent effect on HbA1c.12 14–16 A systematic review in 2017 of 10 randomised trials of diabetes psychoeducational and educational interventions for CYP found a non-significant reduction in HbA1c.17 However, the holistic impact of an educational intervention may be more relevant than glycaemic control alone as an outcome measure. Health responsiveness encompasses user-confidence, dignity and empowerment. When evaluating an intervention, health responsiveness needs to be considered in parallel with health outcomes. However, as a relatively new concept, it requires further operationalisation for it to be integrated into evaluation strategies.18 19

In contrast to the UK, the reported success of European structured education prototypes20 could be due to their pattern of intervention: CYP included in previous UK trials had already been living with T1DM for at least 1 year,12–16 compared with European programmes which aim to target CYP earlier.21 The ‘Diabetes at Diagnosis’ module is provided to CYP newly diagnosed with T1DM and uses an intensive face-to-face delivery by trained facilitators, emulating the European philosophy that CYP and families are most receptive soon after diagnosis.

Generating resources which engage both CYP and parents is likely to be challenging due to varying CYP age demographics and evolving lifestyle and learning needs.14 Resource-specific feedback for ‘Diabetes at Diagnosis’ was very positive; possibly reflecting multidisciplinary input including parent representation, in curriculum and resource development. End-user involvement from SEREN’s inception has been key in creating resources which were useful at the coalface.22 The established learning language from school curricula that children are already familiar with, such as ‘WALT’ (We are learning to…) and ‘WILF’ (What I’m looking for), was used in the resources.

Continuing education is integral to creating positive results13 23 24; studies of a standalone intervention have not shown a clear change in HbA1c, implying it may take longer to consolidate the learning process. Two randomised trials have also reported worsened outcomes at 6–24 months in the structured education intervention arm compared with current practice controls.13 14

Since this evaluation, continued investment by the Welsh Government has enabled ongoing SEREN education, which aims to address this gap. A library of group modules has subsequently been introduced including ‘SEREN Active’, ‘SEREN for pumps’, ‘Moving to year 7’ and ‘Connect’.9 With these additional modules in place, there may be a more sustained and cumulative effect on both patient-reported outcomes and long-term diabetes control. The sustainability of SEREN is supported by its adoption by the National Health Service (NHS) Wales and being embedded in the national diabetes delivery plan. All health boards across Wales are required to deliver SEREN to CYP with T1DM, starting from diagnosis. In addition, commercialisation of the programme is expected to generate income that will further support SEREN’s sustainability.

The strengths of this evaluation include the multicentre design, with age-appropriate questionnaires and relatively high response rates for paper-based questionnaires, increasing its validity. Bias was reduced by the inclusive all-Wales design and the use of blinding before statistical analyses.

Limitations of the evaluation process include the retrospective nature and the varied timing of evaluation after diagnosis. The family questionnaire packs were administered some months after the diagnosis-specific education. This allowed validity for quality of life measures, however, it introduced recall bias in the education package assessment. Furthermore, we were not able to assess differences in knowledge gained because we could not capture pre-education baselines. Finally, an intensive programme delivered at diagnosis is unlikely to affect HbA1c measured after a year. A limitation of ‘Diabetes at Diagnosis’ at this evaluation point was that it was a standalone module without ongoing education or a digital component.

The next national evaluation iteration, which will capture CYP who received continuing SEREN structured education, will provide more information on long-term impact. Indeed, this may also inform the benefit of developing similar programmes in other chronic conditions in childhood such as asthma, epilepsy and inflammatory bowel disease, empowering the CYP and family in self-management.

Conclusions

This short-term evaluation report of the SEREN structured education module ‘Diabetes at Diagnosis’ shows improved CYP and family reported outcomes with positive resource-specific feedback but no positive change in PedsQL scores overall or HbA1c at 1 year. Evaluation of the longer-term SEREN programme in CYP who receive subsequent modules will provide a better understanding of its true impact.

Data availability statement

No data are available.

Ethics statements

Patient consent for publication

Ethics approval

This service evaluation uses an anonymised dataset with study numbers only, and without patient-identifiable data. It was agreed by the Research and Development Committee at the local health board and the Swansea University Medical School Research Ethics Sub-Committee that ethical approval was not required.

Acknowledgments

The authors would like to acknowledge the following for their help in producing this report of the first stage of service evaluation: members of the SEREN working group including Paula Jarett, parent representative, for assistance in preparing and delivering packs to centres across South Wales; Jon Matthias, CYPDWN network manager, for assistance with sourcing graphic design for the questionnaires and facilitating statistical support; Dr Rebecca Robinson, Dr Manoj Potdar and Debbie Lewis for help with administering questionnaires and collecting data; Heather O’Connell and Tracy Wiggham for support with coordinating and collating data; Emma Loyns, parent representative, who was instrumental in providing the lesson plan structure for the work books; Jill Rodgers, who helped to set up and coordinate the SEREN project and played a key role in structuring the first module with integration of evaluation and quality assurance aspects; staff from local units across Wales, for help in administering the questionnaires; and Professor John W Gregory for peer reviewing the paper.

References

Footnotes

  • Contributors RSD'S: interpreted results and wrote the primary and subsequent drafts of the manuscript. MR: oversaw identification of the selected cohorts, design and distribution of questionnaire packs, performed data collection and edited the final manuscript. EH: performed data collection and edited the final manuscript. CB: performed data collection, coordinator for the SEREN programme including dietetic lead, training and as an educator, reviewed programme resources and quality assurance and reviewed the manuscript. YD: performed data collection, nursing lead for the SEREN programme and an educator, reviewed programme resources and the manuscript. JR-S: oversaw selection of psychological evaluation tool. JP: performed statistical analysis and interpreted results. RP and NAD'S: medical coleads for the SEREN programme, designed and oversaw the evaluation strategy, interpreted results and provided critical review of primary and subsequent manuscripts and are joint lead authors. CB, YD, JR-S, RP and NAD'S: are also founder members involved in the conception and development ofthe SEREN programme.

  • Funding The SEREN project was launched in 2013 through an educational grant from Sanofi and Roche and has subsequently been supported by a grant from the Welsh Government channelled through the All Wales Diabetes Implementation Group as annual funding, over the last 5 years.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.