Problem

Between the Flags was implemented across New South Wales to act as a system to detect clinical deterioration in patients early. The standardised paediatric observation charts (SPOC) have fixed criteria for when a child's physiological observations trigger a call (calling criteria). There are times where it is appropriate to alter these calling criteria, because the physiological parameters reflect the underlying illness. For example, if a patient with bronchiolitis is being admitted to the ward from the emergency department (ED) then it may be appropriate to alter their respiratory rate calling criteria (as they will have ongoing tachypnoea and the cause is known).

Altering the calling criteria introduces risk if not done safely, as patients can potentially deteriorate on the wards without this being flagged. In Sydney Children's Hospital, patients can be admitted to inpatient wards from ED with altered calling criteria. However, there is confusion among the junior medical staff around the process of making alterations, and what the consequences of the alterations are. Although guidelines exist, junior medical staff may not be aware of them.

Background

Many hospitals now use early warning systems to track physiological markers such as heart rate, respiratory rate, and temperature.[1] These systems have been implemented in response to root cause analysis of critical incidents.[2] Use of these systems is controversial and their helpfulness is debatable.[3]

NSW Health has developed Between The Flags, which is their own version of an early warning system. The purpose is to ensure early detection of serious illness before clinical deterioration occurs and to avoid the pitfall of inadequate data assessment. A certain physiological measurement can trigger a call for medical review. These triggers are known as the calling criteria.

Altering the calling criteria is a practice that happens across many hospitals, where a patient is admitted to the ward, and it is clinically appropriate to make changes to the formal Between the Flags calling criteria. There has been little research carried out regarding the impact or frequency of making alterations to these calling criteria.

Baseline measurement

Our baseline measurement was to establish current emergency department junior medical officers' knowledge of guidelines on making alterations to the calling criteria. We conducted a survey of all junior medical officers (JMOs) in the emergency department - 20 out of 24 (83%) responded. The questionnaire focused on JMO's awareness of the process of making alterations, specifically areas such as: length alterations remained valid, staff who need to be informed when an alteration is made, and indications for making alterations.

Forty-two percent of junior medical staff surveyed were aware of the guidelines on making alterations to calling criteria. Forty-seven percent were either "very confident" or "pretty confident", while 16% were "not confident" in making alterations. Seventy-four percent reported having received no education on how to make alterations to calling criteria, and only 5% knew that their alterations to calling criteria only lasted for one hour after the patient arrived on the ward. Fifty percent knew that the patient should be reviewed by the ward JMO within one hour of arriving on the ward.

Design

Our intervention is an educational initiative for junior medical staff around the process of altering calling criteria for all patients who are being transferred to the ward from ED. The aim of the intervention is to ensure that junior staff are aware of the existing guidelines, consequently ensuring the safe transfer of their patients and follow-up for their alterations.

Junior doctors receive education on guidelines from many sources, and it is difficult for all the information to be retained. We designed educational tools that were user friendly, short, and easy to understand. We felt that targeting via email, and effective use of technology, would help junior medical staff to remember the teaching content.

Two methods were used. The first was a short video (90 seconds) which was uploaded and shared via YouTube (https://www.youtube.com/watch?v=27VAs3d7EbA). A link to this video was emailed to all junior medical staff. YouTube video statistics allow us to monitor how many times the video is watched.

The second method, was a poster. The poster was designed to be clear, colourful, and concise. It was emailed to all junior medical staff as a png file. Both of these are sustainable as the resources exist and only need to be shared with each new cohort of trainees electronically. It does not require resource-intensive teaching time or manpower.

Strategy

PDSA cycle 1: A short video was designed to train junior medical staff on making alterations to calling criteria. This was emailed to staff but many staff members did not open the link in the email.

PDSA cycle 2: Word of mouth had spread about the innovative nature of the video and more junior staff members were aware of the existence of this video. The link was the emailed again to all junior medical staff and there was a higher uptake in this cycle.

PDSA cycle 3: In addition to the video, a poster was designed to summarise the key teaching points. This was shared via email alongside the video link and was also shared with senior medical staff in the emergency department.

Post-measurement

By three months post introduction of the education and training, 88% of junior medical staff were aware of the guidelines on making alterations to calling criteria. Eighty-one percent felt "very confident" or "pretty confident" in making these alterations themselves.

Sixty-three percent noted that they had received training on making alterations to calling criteria and 91% found it to be "good", "effective", or "excellent".

Fifty percent of junior doctors were aware that their alterations would last for one hour after the patient arrived on the ward, and 81% knew that the patient should be reviewed within one hour of arriving on the ward.

See supplementary file: ds4774.png - “Training poster”

Lessons and limitations

We learnt a number of lessons from carrying out this project:

1. Junior doctors receive education on best practice and guidelines from many different sources. In order for them to retain, or enjoy reading and learning about a resource, the education package has to be accessible and engaging

2. Videos and posters may not be seen as formal training by junior doctors. Although all doctors had seen the poster and most had watched the video, on the post-intervention questionnaire, many junior doctors noted that they had received no training. Perhaps teaching using alternate methods needs to be clearly flagged as being "training"

3. Senior medical staff need to support the intervention in order for it to be continued once the junior medical staff rotate to a new term.

Conclusion

Our educational intervention was effective in improving junior doctor awareness of the guidelines on making alterations to calling criteria (in the short-term). Prior to our training, only 42% were aware of such guidelines, but afterwards 88% were aware of them.

Most importantly, more junior doctors understood the process of making alterations to calling criteria and knew how long their alterations would last (50% post-training versus 5% pre-training). This training needs to be sustained after junior doctors rotate to a new term, and the new medical staff need to be encouraged to access the resources.

References

1. Roland D, Oliver A, Edwards ED, Mason BW, Powell CVE. Use of paediatric early warning systems in Great Britain: has there been a change of practice in the last 7 years? Arch Dis Child 2014;99(1):26-9.

2. Hooper AJ, Tibballs J. Comparison of a Trigger Tool and voluntary reporting to identify adverse events in a paediatric intensive care unit. Anaesth Intens Care 2014;42(2):199-206.

3. Roland D. Paediatric Early Warning Scores: Holy Grail and Achilles' heel. Arch Dis Child Educ Pract Ed 2012;97(6):208-15.

Declaration of interests

Nothing to declare.

Acknowledgements

Thank you to Arjun Rao (Sydney Children's Hospital) for his support during the latter stages of the project and for reviewing the final draft.

Thank you to the other project team members at Sydney Children's Hospital - Lauren Abdy, Paul Hunstead, Lisa Sealey, and Camille Wu.

Thank you to Cathy Vinters, Rachel Primrose, and Sarah Dalton from the Clinical Excellence Commission.