Article Text
PDF
Abstract
Background Healthcare worker (HCW) anxiety and depression worsened during the pandemic, prompting the expansion of digital mental health platforms as potential solutions offering online assessments, access to resources and counselling. The use of these digital engagement tools may reflect tendencies and trends for the mental health needs of HCWs.
Objectives This retrospective, cross-sectional study investigated the association between the use of an online mental health platform within a large academic health system and measures of that system’s COVID-19 burden during the first 3 years of the pandemic.
Methods The study investigated the use of Cobalt, an online mental health platform, comprising deidentified mental health assessments and utilisation metrics. Cobalt, serves as an online mental health resource broadly available to health system employees, offering online evidence-based tools, coaching, therapy options and asynchronous content (podcasts, articles, videos and more). The analyses use validated mental health assessments (Generalised Anxiety Disorder-7 (GAD-7), Patient Health Questionnaire-9 (PHQ-9) and post-traumatic stress disorder (PTSD)) alongside publicly available COVID-19 data. Statistical analyses employed univariate linear regression with Stata SE Statistical Software.
Results Between March 2020 and March 2023, 43 308 independent user sessions were created on Cobalt, a majority being anonymous sessions (72%, n=31 151). Mental health assessments, including PHQ-4, PHQ-9, GAD-7 and primary care-PTSD, totalled 9462 over the time period. Risk for self-harm was noted in 17.1% of PHQ-9 assessments. Additionally, 4418 appointments were scheduled with mental health counsellors and clinicians. No significant associations were identified between COVID-19 case burden and Cobalt utilisation or assessment scores.
Conclusion Cobalt emerged as an important access point for assessing the collective mental health of the workforce, witnessing increased engagement over time. Notably, the study indicates the nuanced nature of HCW assessments of anxiety, depression and PTSD, with mental health scores reflecting moderate decreases in depression and anxiety but signalling potential increases in PTSD. Tailored resources are imperative, acknowledging varied mental health needs within the healthcare workforce. Ultimately, this investigation lays the groundwork for continued exploration of the impact and effectiveness of digital platforms in supporting HCW mental health.
- Mental health
- Teamwork
- Healthcare quality improvement
Data availability statement
Data are available on reasonable request. Data are available on request with appropriate protocols and permissions.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Mental health strain persists among healthcare workers including anxiety and depression, which have worsened during and beyond the pandemic.
WHAT THIS STUDY ADDS
A cross-sectional approach to understanding the associations between an external strain and how healthcare workers use a digital mental health platform and their assessment scores.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
In the evolving landscape where health systems are attempting to support and improve the mental health of the workforce, many have launched online platforms for resources. These platforms may provide insight into the scope of the problem and its trajectories over time.
Introduction
Rates of anxiety and depression among healthcare workers (HCWs) remain high.1–3 The importance of preserving HCW mental health has gained significant attention, including from the US surgeon general and the National Academy of Medicine.4 5 One approach to support the workforce at scale has been the rapid development and deployment of digital technologies such as online mental health platforms, applications and even chatbots.6–8 These platforms may provide a strategy and venue for individual HCWs to complete mental health assessments, access asynchronous related content (eg, podcasts, literature and videos) and connect to individualised expert mental healthcare to initiate mental health counselling or therapy.
Digital programmes offer an additional advantage in that they make it easier to measure use in real time. The objective of this study was to investigate the use of a web-based mental health platform (named Cobalt) across a large, academic health system within the context of COVID-19 case burden, hospitalisations and mortality. The hypothesis was that increasing COVID-19 case burden, hospitalisations and mortality would be associated with higher collective levels of HCW anxiety, depression and platform utilisation.
Methods
We conducted a retrospective cross-sectional study of HCW mental assessments and utilisation data of ‘Cobalt’, a mental health platform at the University of Pennsylvania Health System. This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines for observational studies.9
Cobalt
Created in the early stages of the COVID-19 pandemic, Cobalt is an online resource available to all health system employees and functions as a web-based platform that curates mental health content.8 Users navigating to Cobalt can complete mental health assessments related to anxiety, depression and post-traumatic stress disorder (PTSD) using validated and brief online tools. The platform then provides individual HCWs with links to evidence-based individual or group-based tools and resources and access to coaching and professional therapy using a stepped model of care. Effectively, Cobalt provides a centralised source for HCWs seeking mental health or well-being care to find and use resources for support on their own and also offers the opportunity for individuals to schedule time with mental health experts for one-on-one appointments, peer support or group sessions.
Individual Cobalt users can navigate to the web homepage and choose to proceed anonymously to navigate asynchronous content or identify themselves via name, phone number or email address to connect with psychiatrists, clinical psychologists and other expert mental healthcare. Users can remotely access online content related to mental health, schedule therapy, peer-to-peer support, coaching sessions or appointments with a trained mental health professional.
Cobalt mental health assessment data (eg, assessments of anxiety, depression and PTSD) were aggregated and accessed. HCW privacy remains paramount to Cobalt, and to this investigational study, particularly when related to mental health data. Thus, this university IRB-approved study and approach did not identify individual user-level information.
Measures and outcomes
The Cobalt measures included validated mental health assessments: depression (Patient Health Questionnaire-4 and 9 (PHQ-4 and PHQ-9)), anxiety (Generalised Anxiety Disorder-7 (GAD-7)) and post-traumatic stress disorder (primary care PSTD screen (PC-PTSD)).1 10–12 We also assessed publicly available COVID-19 data including documented case counts, mortality and hospitalisations using the Centers for Disease Control and Prevention (CDC) COVID Data Tracker.13 The primary outcomes were aggregated mental health assessment scores and platform utilisation (eg, number of user sessions, appointments made and completed). Anonymous sessions from the same user were possible and counted separately. Cobalt functionality used PHQ-4 as an initial depression and anxiety screen which then triages those with moderate or high scores to complete the PHQ-9 or GAD-7 assessment.
Data analysis and statistical methods
Linear regression analyses were conducted to analyse the association between COVID-19 case counts and the number of user-initiated sessions, assessment scoring, and appointments made and completed. Analyses were performed using Stata SE Statistical Software: Release V.18.0 (StataCorp).
Results
Cobalt utilisation and mental health assessments
Between March 2020 and March 2023, there were 43 308 distinct user sessions accessing the Cobalt platform with an average of 1170 (SD 550.5) monthly user sessions. Of these, 31 151 (71.9% were anonymous user sessions) with an average of 865 anonymous user sessions per month and 113 non-anonymous user sessions per month. In total, the users completed 9462 mental health assessments including the PHQ-4 (n=2762, 29.2%), PHQ-9 (n=2398, 25.3%), GAD-7 (n=2327, 24.6%) and PC-PTSD (n=1975, 20.9%). Over the study analysis period, there were 409 assessments endorsing the risk for self-harm of those individuals completing the PHQ-9, question 9 (17.1%) and in total, there were 4418 appointments made with a mental health professional. Figure 1 displays depression, anxiety and PTSD assessments and regional COVID-19 case burden over time, from May 2020, when the first county-level data were available, to June 2022.
Regional COVID-19 case burden and mental health assessments. Figure 1 displays monthly variation in COVID-19 case count across five counties within the health system’s geography and median scores of depression (PHQ-4, PHQ-9), anxiety (GAD-7) and PTSD (PC-PTSD) assessments completed on Cobalt. Assessment volume is indicated by indicator size. GAD-7, Generalised Anxiety Disorder-7; PC, primary care; PHQ-4, Patient Health Questionnaire-4; PTSD, post-traumatic stress disorder.
Depression, anxiety and PTSD
The PHQ-4 was implemented and used during the first 29 months of Cobalt’s launch. During this period, the total average PHQ-4 score was categorised as mild with a mean score 5.4 (SD 3.3). PHQ-9 and GAD-7 were used throughout the entire period (March 2020–March 2023). The PHQ-9 had a total average in the moderate range of 10.7 (SD 5.7). Among those indicating risk for self-harm, the final question of the PHQ-9, the overall mean score was higher (15.9, SD 6.0). GAD-7, on average, was at a moderate level with a total mean of 10.2 (SD 5.2). PC-PTSD was used during the first 29 months and had a low total mean (mean 1.9, SD 1.4). Figure 1 displays variation assessments over time.
Mental health assessments and COVID-19 case burden
Using publicly available COVID-19 regional and state case burden including case counts, hospitalisations and ICU admissions, we investigated the association between COVID-19 data and Cobalt accounts, mental health assessment scores and instances of risk for self-harm (question 9 on the PHQ-9). We found no significant associations between account creation on Cobalt or the number of assessments completed with regional or state COVID-19 burden. There was a trend towards significant association indicating more regional and state hospitalisations with more assessments completed (p=0.07 and 0.05, respectively). Anxiety, depression and risk for self-harm scoring were not associated with COVID-19 burden. We did find a significant association between state hospitalisations and number of appointments made (p=0.012). Figure 2 displays the trends in PHQ-9 median scores and COVID-19 cases across time (March 2020–March 2023), with the presence or absence of suicidal ideation in PHQ-9 results depicted in separate figures. Among PHQ-9 respondents with the risk for self-harm, there was a general trend of increased median scores occurring 1–2 months after a hospital case peak.
Median depression (PHQ-9) Scores, risk for self-harm and COVID-19 case burden. Figure 2 displays median monthly depression scores (PHQ-9) among healthcare workers indicating risk for self-harm and those who were not against COVID-19 hospitalisations and ICU admissions. PHQ-4, Patient Health Questionnaire-4.
Discussion
HCW mental health plays a crucial role in healthcare delivery and patient care, with literature highlighting increased rates of burnout, workforce attrition and deteriorating mental health.14 15 As health systems implement mechanisms to broadly support their employees, understanding these strategies and the evolving needs of the workforce is imperative. This study focuses on the utilisation of a health system-wide web-based mental health platform, yielding four key findings.
First, Cobalt offers a potential strategy to assess, at a high level, the collective mental health of a segment of the workforce. The analysis reveals a growth in engagement, measured by the number of unique sessions, with variations in assessments related to depression, anxiety and risk for self-harm. Over the 3-year study period, approximately 1200 users per month engaged with Cobalt. A key finding is that a large majority (72%) were anonymous. To our knowledge, this is one of earlier investigations studying this intersection of public health burden, digital health platforms and the mental health of the workforce over time. Notably, these results suggest that these platforms may serve a dual purpose by providing a real-time, broad assessment of a system’s workforce mental health while preserving privacy and confidentiality. While lacking granularity at the role or division level, these findings offer insights into how health systems can navigate HCW mental health in the digital era. The impact and effectiveness of these platforms over time remain an area for future exploration.
Second, the use of Cobalt was not significantly linked to prominent markers of the pandemic case burden at regional, city or state levels. Although trends indicated increased completion of mental health assessments during periods of high case burden, these did not reach statistical significance. Limitations in the analysis include the influence of other stressors outside patient care, such as racial injustice and political unrest, as well as a host of personal factors at the individual level. Additionally, the nuanced emotional reactions to a case surge may manifest at varying times for individuals. Recognising these complexities is essential for a comprehensive understanding.
Third, using validated mental health assessments across this cohort, median HCW depression and anxiety remained in the moderate range over the study period. Additionally, as anxiety and depression scores slowly lowered, we identified signals of potentially increasing severity of PTSD using the PC-PTSD assessment. These data are aggregated and anonymous, so these findings cannot be interpreted as repeat assessments of individuals. Rather, as intended in the study design, to more broadly understand the group at large. A global improvement of anxiety and depression with a potential increase in individuals potentially experiencing PTSD aligns with mental health reactions to other catastrophic events. Within the context of the pandemic waves, the initial uncertainty and fears may have driven anxiety and depression, and as the public focus shifted, PTSD may rise in the face of the workforce emotionally processing their experiences. While the PC-PTSD assessment is not tailored for HCWs experiencing moral distress, no other current validated tool exists for this population and should be further investigated. The finding does identify early signals for health systems to help guide specific resources to supporting individuals with anxiety and depression versus those with PTSD in healthcare settings.
Fourth, Cobalt’s continuous engagement and outreach can inform a learning health system (LHS) approach towards maintaining a healthy workforce, lowering barriers to mental healthcare, protecting privacy and reducing stigma. Stein et al underscore the critical importance of LHS to harness data to advance, personalise and integrate in clinically meaningful ways.16 As many health systems grapple with continued workforce shortages and high rates of mental health challenges, the exploration of the adoption and utilisation of these platforms’ usage would provide valuable insights.
Limitations
This study has several important limitations. It reports aggregated data across roles within a large healthcare system and due to the anonymity built into the system, it does not provide demographic or individual-level data. Importantly, a single individual accessing Cobalt anonymously for separate sessions at different times would be counted multiple times in this analysis. It also reports cross-sectional information from a single health system, though over a significant period of time. The mental health assessments used within this platform are validated but do not provide diagnostic capacity for mental health disorders. Additionally, the assessments do not provide a full description of the complexities of mental health, symptoms of depression, anxiety or PTSD, but rather assist in triage and identifying individuals at risk. COVID-19 case data, even from a robust source like the CDC, does not fully reflect the burden as testing capacity and requirements fluctuated through the years of the study and do not capture home test results. Finally, the sample accessing and completing the assessments presents selection bias and may not be entirely generalisable to the entire healthcare workforce. The study also has strengths in using these validated mental health assessments over a broad sample of HCWs over the first 3 years of the pandemic.
Conclusion
In this retrospective cross-sectional study, we do not find significant associations between COVID-19 case burden, the use of a mental health platform or mental health assessment scores. We identify variations in depression and anxiety scores over time indicating moderate levels across the workforce and persistent levels of suicidal thoughts. Finally, the use of a web-based mental health platform serves as an important resource for HCWs and one that is consistently used.
Data availability statement
Data are available on reasonable request. Data are available on request with appropriate protocols and permissions.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by the University of Pennsylvania Institutional Review Board (#848844).
Acknowledgments
Dr Merchant is the PI of NIH NHLBI R01HL1-141844, NIH/DHHS R01 MH127686 and NIH K24 HL157621.
Footnotes
X @AgarwalEM
Contributors AA was responsible for design, execution and draft of the manuscript. AA is the corresponding and guarantor author. AP was responsible for data analysis. HJM was responsible for data analysis. REG was responsible for data collection and execution. DAA provided manuscript revisions and oversight. CL was responsible for Cobalt design and manuscript revision. LB was responsible for Cobalt design and manuscript revision. SB was responsible for Cobalt design and manuscript revision. RK and CL were responsible for Cobalt design and manuscript revision. LS was responsible for study design and execution. RMM was responsible for study design, execution and manuscript preparation.
Funding This study was funded by National Institute of Mental Health (R01MH127686).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.