Objective To assess whether engagement in a COVID-19 remote patient monitoring (RPM) programme or telemedicine programme improves patient outcomes.
Methods This is a retrospective cohort study analysing patient responsiveness to our RPM survey or telemedicine visits and outcomes during the COVID-19 pandemic. Daily text message surveys and telemedicine consultations were offered to all patients who tested positive for SARS-CoV-2 at our institutional screening centres. Survey respondents with alarm responses were contacted by a nurse. We assessed the relationship between virtual engagement (telemedicine or RPM survey response) and clinical outcomes using multivariable logistic regression.
Results Between 10 July 2020 and 2 January 2021, 6822 patients tested positive, with 1230 (18%) responding to at least one survey. Compared with non-responders, responders were younger (49 vs 53 years) and more likely to be white (40% vs 33%) and female (65% vs 55%) and had fewer comorbidities. After adjustment, individuals who engaged virtually were less likely to experience an emergency department visit, hospital admission or intensive care unit–level care.
Conclusion Telemedicine and RPM programme engagement (vs no engagement) were associated with better outcomes, but this was likely due to differences in groups at baseline rather than the efficacy of our intervention alone.
- decision support
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
The COVID-19 pandemic has placed a remarkable burden on healthcare systems internationally. As of July 2021, there have been over 33 million identified cases of infection with SARS-CoV-2 resulting in >600 000 deaths in the USA alone.1 The overwhelming majority of patients are managed in the outpatient setting and do not require inpatient or emergency management. However, there are unpredictable cases, in whom acute clinical worsening can occur with minimal warning. Due to the rapid spread of the virus and limited availability of healthcare professionals, the development of strategies to monitor and assess patients remotely has accelerated.2 Telemedicine has allowed for outreach to patients in more remote regions, rapid assessment of more patients per provider and safer monitoring of symptoms obviating the need for in-person examinations.3 However, given the rapid infectious rate of the virus, the staffing required for telemedicine visits cannot be sustained, and self-reported patient triage is becoming a popular method of remotely surveying symptom severity.4
Remote patient monitoring (RPM) programmes have been developed across hospitals nationally to track patient symptoms and health data and notify remote healthcare providers about alarm symptoms. Not only do RPM programmes allow for closer monitoring of patients, but also trends in patient reporting may reflect clinical courses of COVID-19 that may otherwise remain uncharacterised.5 Despite the benefits RPM programmes promise, challenges to widespread implementation exist. Prior studies on RPM programme implementation have shown that the use of technology is limited in some populations, particularly the elderly who may not be able to easily navigate smartphone applications.6 RPM programme responses may also rely on patient subjective responses, and so, patients with lower healthcare literacy may not be triaged appropriately or may have more difficulty navigating the RPM programme.7 Older adults as well as racial and ethnic minorities have been two groups most severely affected by COVID-19, highlighting the importance of effective remote monitoring in populations historically disadvantaged with RPM advances.8
Our centre sought to maximise outreach to ambulatory patients with COVID-19 through a combination of telemedicine visits, nursing phone calls, research team phone calls and online patient portal messaging for enrolment into our daily survey. The objectives of this study were to compare characteristics of responders and non-responders to these surveys and to assess whether those who were engaged with our centre virtually, through telemedicine or RPM survey responses, had improved outcomes compared with those who did not engage virtually.
Study design and site
We retrospectively examined all patients testing positive for SARS-CoV-2 from 10 July 2020 to 2 January 2021 (n=6822) to correspond to the time the RPM survey message system was being used. The study was conducted in patients tested during inpatient admissions, visits at outpatient clinics and emergency department (ED) settings affiliated with Emory Healthcare in Atlanta, Georgia. In total, our programme was launched at six large urban hospitals and affiliated emergency rooms (Emory University Hospital, Emory University Hospital Midtown, Emory Johns Creek, Emory Decatur Hospital, Emory Hillandale Hospital and Emory Saint Joseph’s Hospital) and over 120 primary care clinics in urban and suburban settings as part of the Emory Clinic.
We deployed a COVID-19 tracking survey via Tonic Health (R1 RCM, Chicago, Illinois, USA), a platform allowing for customisable surveys including patient-reported outcomes developed by the study team (online supplemental table 1). We were able to link responses to our electronic health record (EHR) (Cerner, Kansas City, Missouri, USA). We designed our survey to be delivered via a secure link sent through short message service (SMS) text messages. The mobile version reminded patients each day via text message to complete a survey. The initial survey on day 1 of survey outreach consisted of behavioural questions about daily activities, social demographics including information about household members, travel history, medical history and any known comorbidities and current vital signs and symptoms. Responses to specific questions helped to guide additional questions (eg, only women were asked about pregnancy, and only pregnant patients were asked about their obstetrician). Follow-up surveys from day 2 to day 14 assessed current vital signs and symptoms.
Study population and RPM programme eligibility
Patients who tested positive via internal testing and had reasonable clinical suspicion for COVID-19 by hospital staff evaluation were invited to participate in the RPM programme. Reasonable clinical suspicion included symptoms consistent with COVID-19 illness or known exposures. Exclusion criteria included age <18, cognitive or behavioural health barriers to participation and requirement of an interpreter to complete the surveys, which were only available in English. The majority of patients who enrolled in the RPM programme did so at the time of COVID-19 test result notification, a standardised phone call placed to patients by the institutional nurse triage centre as test results became available. During this phone call, patients were offered enrolment into the telemedicine clinic, the RPM survey or both. Patients not enrolled via this initial nurse call but were positive for COVID-19 were asked to participate via phone call by research staff if extensive chart review revealed no exclusions. Finally, a small trial of patient outreach via patient online portal communication was attempted but was discontinued once it was found to be less effective than phone call outreach.
Our virtual outpatient management clinic (VOMC) is a dedicated team for monitoring high-risk outpatients with COVID-19 and has been previously described.9 10 Contrary to the SMS RPM programme, a patient must have an identified risk factor for severe illness (such as age, comorbidity and/or lower respiratory symptom) to be eligible for VOMC. An initial intake visit (physician or advance practice provider) was conducted by synchronous audio/video communication and followed by scheduled phone calls by nurses for 7–21 days (depending on risk factors and symptom improvement). Patients with worsening symptoms had the option of acceleration of care to our acute respiratory clinic,6 an in-person outpatient clinic dedicated to patients with respiratory symptoms related to suspected or confirmed COVID-19. Patients whose care did not require in-person follow-up had regular telemedicine visits until symptom resolution, as described in our prior work.10 Use of the VOMC resources did not preclude use of the Tonic SMS services.
Escalation of care based on survey responses
Patients who responded positively to ‘alarm’ questions had their medical information and survey responses reviewed by nursing staff (figure 1). Alarm responses included abnormal vital signs (for those with the ability to measure: heart rate >100, systolic blood pressure <90, diastolic blood pressure <60 and pulse oximetry reading <94), severe lightheadedness, severe difficulty breathing, inability to tolerate oral intake for 12 hours, continuous chest pain and confusion (online supplemental table 1). Nursing staff would call to triage the patient to VOMC, acute respiratory care (ARC), ED or self-care and reassurance.
COVID-19 test result; survey responses; outcome data including hospitalisations, intensive care unit (ICU) and death; and patient demographical data were obtained from the Emory Clinical Data Warehouse (CDW) using MicroStrategy supplemented by direct access to the Oracle database. To identify patients who were at higher risk for clinical worsening, presence of patient chronic medical conditions that qualify for emergency use authorisation (EUA) for monoclonal antibodies for the treatment of COVID-19, as issued by the United States Food and Drug Administration (FDA), was determined using matching of International Classification of Diseases, 10th Revision (ICD-10) visit diagnoses from encounters from the prior 3 years, age and most recent body mass index (BMI) similar to prior studies.11 This was supplemented with text search of clinical terms from Systematised Nomenclature of Medicine Clinical Terms diagnoses, which are based on an internationally recognised clinical healthcare terminology, from active outpatient problem list diagnoses (online supplemental table 2).12 Geographical coverage was determined by most recent zip code of the patients that used the Tonic surveys in the CDW and then deployed into a heat map algorithm in Python.
We performed descriptive statistics for each patient who tested positive for SARS-CoV-2 and the subgroup of positive responders to our survey. We assessed programme engagement by analysing survey responses, length of time in the programme and symptom patterns across the submitted questionnaires for the survey responses and completed encounters in the VOMC. Baseline characteristics were compared between responders, enrolled non-responders and those who declined enrolment using Student’s t-test for continuous variables and the χ2 or Fisher’s exact test for categorical variables. The association of patient engagement type (no virtual communication, VOMC visits only, RPM survey response only and VOMC and RPM survey response) with a stratified and combined outcome variable of ICU care, hospital admission without intensive care and ED visit following positive test collection and initial virtual engagement was examined using a multivariable logistic regression model, adjusted for age, EUA criteria, gender, race and Emory primary care provider (PCP). Data were analysed with the use of SPSS statistical software V.27 and Python software V.3.9.1 (Python Software Foundation, Beaverton, Oregon, USA).
RPM survey improvements
RPM survey intake completion per overall positive PCR test from our centre was used to monitor survey engagement. Using this metric allowed for internal control for fluctuations in COVID-19 cases. This rate was monitored continuously, and several improvements were initiated during the project. Initial response rates were lower than expected after an error in the communication between our health record and the Tonic dashboard. This was corrected on the back end after 6 weeks had elapsed. Additionally, the 21-day period for survey completion was shortened to 14 days after 6 weeks of the initial survey rollout. The number of questions was shortened after 6 weeks as well, and the follow-up surveys only elicited a full review of systems if the patient reported a worsening overall. Finally, reinforcement of the purpose of the programme was initiated after 8 weeks of survey delivery by calling patients who were enrolled in the programme but had not completed a survey. Study members followed a script to educate the patients on the purpose of the programme and encourage completion.
A total of 6822 patients tested positive for COVID-19 from internal PCR testing during the study period (10 July 2020 to 2 January 2021). Of these, 3434 patients were enrolled to receive the RPM survey via nurse outreach, research team outreach or patient portal message, and 1230 of these patients completed at least one survey response.
Baseline characteristics of cohort and responders
The baseline characteristics of the entire cohort are displayed in table 1. The mean age was 51.6±18.1, 3911 (57.3%) were women and 3307 (48.5%) were black. Characteristics of responders, enrolled non-responders and those who declined enrolment were stratified and are displayed in table 1.
Compared with enrolled non-responders, patients who responded to at least one survey were more likely to be younger (mean age: 49.4 vs 52.9 years), female (65.3% vs 55.1%) and white (40.1% vs 32.7%) and had fewer comorbidities including diabetes mellitus, cardiovascular disease and lung disease (p<0.001 for all characteristics). Patients were most likely to be contacted for RPM enrolment by nursing staff, with 1127 (94.9%) of survey responders engaged through this method. Of those who responded to the RPM programme surveys, a large portion also had established care in the VOMC (17.1%). Enrolled non-responders were also enrolled in the VOMC, with 290 (13.2%) using this service.
Survey response rate
Our initial survey response rate was 0.09 surveys per positive PCR collected the same week. A positive trend was observed during the study period with the highest rate being 0.306 with an R2 of 0.4895 (figure 2).
Association of virtual engagement with combined in-person care endpoint and stratified endpoints
Overall, 2206 (32.3%) of patients who tested positive experienced the combined endpoint of ED visit, hospital admission or ICU-level care, with 76 (6.2%) responders and 955 (43.3%) non-responders experiencing the combined endpoint. Our multivariable model (table 2) adjusted for gender, race, age, EUA criteria and system primary care physician demonstrated that patients who were engaged with our centre virtually through RPM responses (OR 0.49, 95% CI 0.41 to 0.59) or both RPM and VOMC visits (OR 0.25, 95% CI 0.16 to 0.41) were less likely to have the combined endpoint requiring in-person care. Women were less likely to experience the end point (OR 0.85, 95% CI 0.76 to 0.96), and black patients were more likely to than white patients on univariate analysis (OR 1.16, 95% CI 1.04 to 1.30), although this finding did not persist in the adjusted model (OR 1.17, 95% CI 0.92 to 1.18). Patients missing race data were excluded from our multivariable model (1440 patients, 14.3% of overall cohort). When stratifying our combined endpoint into ED visit, ICU care and hospital admission, we found that our findings regarding virtual engagement persisted for those who had ICU care or hospitalisation but were no longer significant for responders to both RPM survey and telemedicine visiting the ED (OR 0.62, 95% CI 0.25 to 1.55). Women were less likely to require ICU-level care than men (OR 0.63, 95% CI 0.52 to 0.76), and black patients were more likely than white patients to visit the ED (OR 1.59, 95% CI 1.14 to 2.24). System PCP was associated with lower rates of hospital admission (OR 0.59, 95% CI 0.51 to 0.68) and ICU care (OR 0.67, 95% CI 0.55 to 0.82), but not lower rates of ED visits (OR 1.14, 95% CI 0.84 to 1.55).
Our study primarily demonstrates that individuals who responded to our survey tool were likely to be younger and healthier than those who did not respond. Although we noted an association between virtual engagement and improved patient outcomes, it is challenging to attribute this association to our intervention as opposed to baseline differences in responders, non-responders and those who refused enrolment. However, we do believe that our results support efforts to increase utilisation of telemedicine and automated self-care as they certainly demonstrate no worsening of adverse outcome risk.
Our survey-responding cohort was skewed towards younger people who may have had more access and familiarity with navigating technology. This is similar to associations found in other healthcare technologies, such as telemedicine video visits, in older adults.13 Women were more likely than men to respond to surveys but 15% less likely than men to experience the escalated care combined endpoint after adjustment for other confounders. This is not dissimilar to what other studies assessing survey response bias by gender have found, although no clear rationale for this discrepancy has been described to our knowledge.14 Responders also tended to be predominantly white, even though our overall cohort was primarily composed of black patients. As Black Americans are at greater risk of hospitalisation and death secondary to SARS-CoV-2 infection, elucidating reasons for decreased participation in virtual healthcare monitoring is critical.15 Patients who responded to our survey also had fewer comorbidities. We believe the reasons to this are twofold. First, patients with more comorbidities may be older and less familiar with technology, and second, patients with more comorbidities may be more likely to speak to specialist providers than to seek out care from a PCP.16
Our multivariable model showed that individuals who engaged in RPM or VOMC were about 50%–70% less likely to require hospitalisation or emergency care after controlling for EUA criteria, age, gender and race. Users of these services were skewed towards specific demographical groups who may have been healthier and at lower risk of hospitalisation compared with patients who were not enrolled or responders, and this effect size may be over-represented by our engaged cohort. To this point, when stratifying our combined endpoint, we found that ICU care and hospital admissions were decreased in the group using telemedicine and RPM surveys, but ED visits were not significantly decreased. For RPM survey–only responders, all three outcomes were significantly decreased. This suggests that these groups may have been overall healthier at baseline compared with those who did not virtually engage. Those who used VOMC visits only were not significantly likely to have a decrease in the combined or stratified endpoints, suggesting that these patients were likely sicker at baseline. Overall, fundamentally different baseline levels of health in each cohort may have skewed our statistical analysis. Although we did adjust for age, BMI and comorbidities, it is possible that the fact that our responding cohort was healthier at baseline had a disproportionate impact on our results and generated a large effect size statistically.
A less likely but possibly contributory explanation for this observed association between our responders and a reduction in ED visits could be a result of ‘auto-triage’ by RPM. Self-reported significantly abnormal vital signs or symptoms resulted in a triage call from a COVID-19 hotline nurse who would offer reassurance and self-management, a telemedicine visit and an ARC visit or direct the patient to the ED. In the absence of these services where less concerning vital signs or symptoms were filtered by multiple layers, patients may have self-referred to the ED. Our VOMC clinic is staffed by providers with experience in outpatient management of COVID-19, and our nurse triage team from our COVID-19 hotline has extensive experience in management of COVID-19 as well. Many symptoms that are expected from COVID-19 are alarming for patients, and separating out the most concerning signs or symptoms can be achieved by experienced practitioners. The specialised management from providers experienced with patients with COVID-19 or virtual engagement in general could have helped to reduce the need for some higher-level in-person services. However, we do acknowledge that given no medical interventions such as monoclonal antibodies were offered during the time of our study, we believe that there is insufficient biological plausibility to suggest that remote monitoring services reduced incidence of severe COVID-19 requiring ICU care. If our observation with a reduction in these services is in part reflective of virtual interventions, adaptation of virtual engagement methods may help reduce the burden of healthcare costs for patients who only require self-management and monitoring. Our virtual engagement strategies also allowed patients to receive specialised COVID-19 care from an academic medical centre even from remote locations when transportation may have been difficult due to geographical barriers or impossible given home isolation requirements from COVID-19 infection (online supplemental figure 1).
Multiple strategies to track patient symptoms in real time have been employed during the COVID-19 pandemic in the USA. For example, the Cleveland Clinic developed a phone app called ‘Care Companion’, which incorporated a similar workflow structure to our own RPM tool.17 The daily survey collects information on patient vital signs using a provided device and five symptom questions. This programme has been validated at other institutions, including Mass General Brigham, and is associated with decreased risk of admission to the ED or hospital.5 However, Care Companion was developed in collaboration with Epic, whereas our survey method has the potential to span multiple electronic medical record (EMR) systems. Additionally, our survey contained more questions about patient symptoms and did not require advanced mobile application familiarity. Mt. Sinai developed a similar application, called the Precision Recovery Program; however, this RPM programme was only offered to symptomatic patients when it was initially deployed.18 Their programme integrated both weekly telemedicine visits and daily RPM monitoring with escalation of care when worsening of symptoms was noted.
Our study has several notable strengths. First, we have a large patient population representing a wide range of patient demographics throughout several regions in Georgia. Our population also included patients who were tested at multiple environments—the emergency room, inpatient and outpatient—and was more likely to capture the large spectrum of COVID-19 presentations. Second, our survey tool did not require advanced technological knowledge, nor patient portal enrolment and was thus more accessible to our patients who might have been unfamiliar with technology or unwilling to download an application onto their phones. Third, our population included asymptomatic patients, whereas many RPM studies focused on actively symptomatic patients. This has important implications in characterising clinical sequelae for initially asymptomatic patients whose symptom severity worsens.
Several limitations to our study must be considered. First, because our survey was primarily delivered via text message, there is a possibility that patients who were non-responders thought they were spam and otherwise would have participated. We attempted to ameliorate this possibility by verbal communication about the survey format as well as clarifying language within the text message. Second, the 21-day implementation of the survey may have become repetitive for participants, particularly if they were asymptomatic or showed mild symptoms, and led to their discontinuation. We attempted to address this possibility by shortening the length of time surveys were sent to 14 days. Third, there is a possibility that the questions we posed were too numerous. We attempted to address this problem by asking patient history questions in the initial survey and narrowing down our focus to current symptoms in each follow-up survey with a particular focus on alarm symptoms that would be most predictive of patient outcome.
Additional biases in our analysis may be consequences of variable EMR maintenance differing by provider and facility. For example, our cohort is missing a portion of race data (1440 patients, 14.3% of overall cohort) due in part to EHR variability, and components of the Charlson Comorbidity Index (CCI) calculation may have been misclassified. Similar to most retrospective EHR-based studies, our outcome data also primarily reflect patient care provided at our hospital system and affiliates that we have EMR access to. We cannot conclusively say that non-responders or those who declined enrolment were more or less likely to visit an outside hospital; however, we suspect that the risk of misclassified outcomes applies to both groups equally. Finally, as a retrospective study, unmeasured confounding may exist due to initial indication of COVID-19 testing and evolving practices surrounding patients testing positive as more information about infectious sequelae, contact tracing and preventative strategies became available. Although our multivariate analysis controlled for age, race, gender and EUA criteria, we expect that additional confounding we were not able to control for contributed to the results. For example, technological proficiency and health literacy of the patients may have contributed to our observation of reduced need for higher level of care following diagnosis. Patients who are engaged virtually either through VOMC or Tonic surveys may have been more proactive about management of their health. Ultimately, a prospective, randomised trial would be required to address the possibility of unmeasured confounders and more definitely answer the question of whether virtual engagement affects clinical outcomes.
Future directions in our own survey implementation and for other health centres should focus on targeted outreach based on populations who are under-represented in responses, as well as those populations most impacted by COVID-19 and at risk of hospital or ICU admission. Our centre is planning to supplement our patient-based responses with wearable devices that may provide accurate, objective vital signs and oxygen levels to staff triage centres. Based on RPM programme responses and remotely recorded vital signs, we also hope to generate predictive machine learning algorithms to better refer patients to care centres before symptoms worsen. The strategies developed in this programme including a comprehensive registry have also facilitated the creation of predictive algorithms for identification of patients at highest risk of admission. Given that there are now evidence-based treatments to reduce risk of admission from monoclonal antibody infusion as we have also demonstrated at our own institution, we are poised to hopefully reduce the risk of admission of care through both technological monitoring and medical therapies.19 Additionally, with the increased understanding of implications of long COVID-19, survey questions may need to be revised to capture long-term consequences of COVID-19.
In conclusion, we deployed an RPM programme to monitor and triage patients testing positive for SARS-CoV-2. We observed a reduction in ED utilisation, hospital admission and ICU care in our patients with COVID-19 who engaged with our institution via remote monitoring surveys. We suspect that our observations likely had contributions from confounding not readily apparent, such as differences in baseline health status, given the significant impact on outcomes with insufficient biological plausibility to explain such a large effect size. However, given that we certainly have no reason to suspect any worsened outcomes from self-reported remote monitoring in patients with COVID-19, we propose an increased use of this strategy to reduce burdens on healthcare systems and decrease the risk posed to patients visiting hospitals during an ongoing pandemic. A randomised clinical trial would provide a more conclusive evaluation of the effect of the intervention. Ongoing additional evaluations include utilisation of remote monitoring devices capable of obtaining physiological parameters remotely and creation of a machine learning–derived risk score that can assist in the triage process of outpatients with COVID-19.
Data availability statement
Data are available upon reasonable request.
Patient consent for publication
This study was a quality improvement initiative and was exempt from Institutional Review Board review.
Contributors RS: performed statistical analysis, data cleaning and drafting of the final manuscript. BEA and ZH: assisted in writing of the manuscript and data collection. RK, TMJ, and JBO: designed and directed the project and assisted in drafting the final manuscript. LCP: designed and directed the project, assisted in drafting the final manuscript and assisted in statistical analysis of data. BJA: designed and directed the project, assisted in drafting the final manuscript and performed statistical analysis and visualisation of data.
Funding Work of the CARES GWEP is supported by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS) as part of Award Number T1MHP39056 totaling US$90 625 with zero percentage financed with non-governmental sources. RS was supported by funding from NIH/NIMHD U54 MD008173 and by the National Centre for Advancing Translational Sciences of the National Institutes of Health under Award Numbers UL1TR002378 and TL1TR002382.
Disclaimer The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement by, HRSA, HHS or the US Government.
Map disclaimer The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area of its authorities. This map is provided without any warranty of any kind, either expressed or implied.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.