Article Text

Download PDFPDF

Using web-based technology to improve depression screening in primary care settings
  1. Jessica Jeffrey1,
  2. Minh-Chau T Do1,
  3. Nastassia Hajal1,
  4. Yu-Hsiang Lin2,
  5. Rachel Linonis1,
  6. Mark S Grossman2,
  7. Patricia E Lester1
  1. 1Division of Population Behavioral Health, UCLA-Semel Institute for Neuroscience and Human Behavior, Los Angeles, California, USA
  2. 2Departments of Medicine and Pediatrics, University of California Los Angeles Health System, Los Angeles, California, USA
  1. Correspondence to Dr Jessica Jeffrey; jjeffrey{at}


Background Given the high rates at which patients present with behavioural health (BH) concerns in primary care (PC), this setting has become the de facto mental health system. As a result, screening for depression and other BH conditions in PC has become a critical target for improving patient outcomes. However, integration of screening into busy PC workflows can be challenging due to barriers such as limited time and resources.

Methodology A digital, cloud-based BH assessment tool, which included electronic health record enhancements, was developed and implemented in two urban PC practices as a prelude to a planned larger-scale implementation. The implementation strategies included a reorganisation of workflows within the PC setting, comprehensive training for staff and PC physicians, and institution of an incentive programme for PC clinic managers. To examine whether the introduction of the cloud-based BH assessment tool and associated implementation strategies was associated with increased screening rates, we compared rates of screening from January through June 2017 to rates of screening from January through June 2018 (subsequent to implementation). We also examined BH symptomatology reported by patients in PC.

Results Following the implementation process, rate of BH screening with Patient Health Questionnaire-2 (PHQ-2) increased from 50.5% to 57% (p<0.00000000000000022) and rates of subsequent screening with PHQ-9, for those scoring at risk, defined as a score of ≥1, on PHQ-2, increased from 34.5% to 91.4% (p<0.00000000000000022). Additionally, high rates of ‘moderate’ and ‘severe’ symptoms of depression (40.3%), anxiety (42.6%) and substance use (26.7% alcohol; 31.2% other substance use) were observed among PC patients.

Conclusions Results suggest that a comprehensive implementation plan, including digitisation of BH assessment, reduced the burden of systematic screening. High rates of BH symptomatology underscore the need for comprehensive BH assessment and systems planning to address the high need for BH services among PC patients.

  • health behaviour
  • primary care
  • healthcare quality improvement
  • mental health
  • quality improvement

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


Behavioural health has attained new prominence as a key component in improving population health outcomes. Approximately 8%–25% of adults1–3 within the primary care setting have a diagnosable behavioural health condition, and it is well established that these conditions substantially impact health outcomes and healthcare costs.4 However, behavioural health conditions are often under-recognised within primary care settings. For example, although depression affects at least 8% of primary care patients,5 studies conducted in primary care settings suggest that only 50% of patients with depression are identified.6–8 The high prevalence of these under-recognised and costly conditions, coupled with the rise of accountable care organisations and population health-based reimbursement models, has led health systems to increasingly adopt innovative ways to achieve early identification and treatment of behavioural health conditions.

Universal depression screening and early treatment are foundational components of comprehensive behavioural health management. The United States Preventive Services Task Force (USPSTF) recommends screening for depression in the general adult population. The USPSTF9 further states that ‘screening should be implemented with adequate systems in place to ensure accurate diagnosis, effective treatment, and appropriate follow-up’. Furthermore, both private and public payers now require and incentivise not only depression screening, but documentation of remission from depression.10 Despite this, the management of population-level behavioural health screening and tracking of patient outcomes may be cumbersome for health systems. To address this challenge, technological innovations may be used to enhance behavioural health screening rates, inform thorough assessment, track patient outcomes and guide clinician feedback to patients.11 The current paper has two aims: (1) to describe the implementation strategies employed to increase depression screening rates in two primary care clinics in a large, urban health system, including the utilisation of cloud-based behavioural health tool as a key component of this process, and (2) given that current data on the rates of behavioural health symptoms of patients presenting in primary care settings are limited (the most frequently cited papers symptom severity are over 20 years old)5–8 to document up-to-date rates in an urban, primary care setting.


The process of enhancing behavioural health screening in these two primary care clinics involved two key strategies: (1) the development of the behavioural health check-up (BHC), a digital, cloud-based behavioural health assessment tool that is integrated with the electronic health record (EHC), (2) an implementation process including reorganisation of workflows within the primary care setting, comprehensive training for staff and primary care physicians (including in the importance of behavioural health screening and assessment and administration of the BHC) and institution of an incentive programme for primary care clinic managers. This research protocol was submitted to the Institutional Review Board of a large public university in California, and their determination was that this work did not involve human subjects research, informed consent of participants was not required, and therefore did not require full review.

Patient and public involvement

This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

BHC platform

The BHC is a cloud-based, integrated web-based data collection (Survey Gizmo) and data management and tracking (Salesforce) platform that collects self-report data from patients and provides clinicians with real-time clinical guidance. The BHC may be completed on any internet-enabled device and consists of psychometrically validated measures that are available within the public domain. The assessment battery includes a commonly used measure of depression symptoms, the Patient Health Questionnaire-9 (PHQ-9),12 as well as measures for common comorbidities that are frequently seen in primary care populations such as anxiety (generalised anxiety disorder (GAD-7)), alcohol use (alcohol use disorders identification test (AUDIT-C)) and drug use (drug abuse screening test (DAST-10)). The platform is integrated with a large university health system’s EHR. Please see figures 1 and 2 for screenshots of the BHC platform. Prior to the current primary care implementation, the BHC was successfully implemented within a large university health system’s integrated care programme.11

Figure 1

Behavioural health check-up for primary care—sample login page.

Figure 2

Behavioural health check-up for primary care—sample questions.

Clinical development and functionality

The BHC, developed by behavioural health clinicians at a large public university in California, uses standardised cut-offs for psychometrically validated measures to capture a continuum of risk.11 On patient completion, the BHC is automatically scored and results (both composite scale scores and at individual item-level responses) are immediately available within the notes section of the EHR. Additionally, interpretation of the scores and guidance to address both protective and risk factors ascertained within the assessments are available in the EHR. Clinical feedback was developed for each of the risk ranges within each measure to guide patient psychoeducation and to introduce cognitive-behavioural coping skills. For instance, a PHQ-9 score of 15–19 (moderately severe depression) will generate feedback to the provider that the patient is likely experiencing emotional and functional symptoms of depression. Furthermore, the provider is encouraged to engage in strategies such as emotional regulation and cognitive restructuring to support the patient, in addition to continuing to monitor the patient and consider a referral for behavioural health services as needed .11

The flowsheets within the EHR contain a patient’s answers to each individual item within a given rating scale to allow for tracking of specific symptoms over time. For example, if a patient indicates that he or she is ‘feeling tired or having very little energy’ ‘nearly every day’ (a score of ‘3’ on question 4 of the PHQ-9), this symptom may easily be tracked at the item level with administration of a subsequent PHQ-9 at the follow-up visit. Additionally, a best practice advisory is triggered within the EHR for patients who have elevated scores (≥10) on the PHQ-9, or if the patient endorsed any level of suicidal ideation (question #9 of the PHQ-9) on the rating scale. Clinicians are required to document their treatment plan within the EHR as a response to patients’ PHQ-9 scores and their subsequent clinical assessment.

Technology development and functionality

The BHC was developed at a large public university in California within a large university health system. The platform was developed as an integration between Survey Gizmo and Salesforce, a third-party screening tool, database and consumer management system. Salesforce entered into a Health Insurance Portability and Accountability Act (HIPAA) business associate agreement with the health system, which ensures that the data within Salesforce are encrypted and secure. Furthermore, the BHC has been integrated within the EHR, which allowed for a seamless interface with patients and clinicians; in addition, this allowed for BHC assessment reports to be automatically integrated as assessment notes, and for scores to be inputted within discrete fields in the EHR. This integration has been central to an efficient care flow pathway within primary care, as well as behavioural health treatment settings at the university.

Clinic workflow and implementation within two initial sites

Prior to implementation of the BHC, behavioural health screening in the two primary care clinics consisted of verbal administration of the PHQ-2, with medical assistants entering patients’ responses in real time into the EHR, followed by administration of a paper-and-pencil PHQ-9 for patients who answered affirmatively to at least one of the two questions. The PHQ-2 assesses for the presence of two core symptoms of depression within the last 2 weeks (whether the patient has experienced ‘little interest or pleasure in doing things’, or has been ‘feeling down, depressed or hopeless’). Medical assistants were required to transcribe patients’ PHQ-9 responses into the EHR.

An interdisciplinary workgroup including members from the university health system’s primary care network, ambulatory nursing, the division within the university’s department of psychiatry and the university’s Information Services and Solutions (ISS) was created to increase depression screening rates within primary care, beginning with two initial sites. Implementation strategies included reorganisation of operational workflows and developing trainings for healthcare providers and clinic staff. With the introduction of the BHC, patients continued to be screened through a verbal administration of the PHQ-2 with medical staff. However, if a symptom was endorsed, the medical assistant then provided the patient with a tablet device or desktop computer in order to complete the BHC rating scale battery. To launch the BHC, a medical assistant enters the patient’s date of birth and the last five digits of the patient’s medical record number.

The assessment battery consisted of the PHQ-9 (α=0.86–0.89; Sensitivity=88%; Specificity=88%),12 GAD-7 Scale (α=0.92; Sensitivity=89%; Specificity=82%),13 AUDIT-C (α=0.80–0.90; Sensitivity=92%; Specificity=94%)14 15 and a one-item drug use screening question.16 The one-item drug use screening question asks, ‘How many times in the past year have you used an illegal drug or used a prescription medication for non-medical reasons?’ Patients who respond to this screener question with a response of one or more times over the past year are then gated into completing the DAST-10 Item (α=0.86; Sensitivity=100%; Specificity=84%).17

Healthcare provider trainings

Primary care staff and provider trainings in the BHC were designed with an awareness that integrating behavioural health screening into busy primary care offices requires culture change within the clinic setting. We used a clinic-partnered approach, such that the training team collaborated with key physicians at the pilot clinics who advocated for usage of the BHC. To ease the burden on providers and clinic staff, training sessions were provided either on-site or virtually. Trainings were led by a multidisciplinary team that included a behavioural health clinician (psychiatrist or psychologist), a primary care physician and a member of the ISS team. Trainings included time for soliciting questions and obtaining feedback from staff and providers; this collaborative approach allowed questions and concerns relating to both behavioural health and technical issues to be addressed by an individual with expertise in that particular area. Importantly, in partnering with a primary care physician, primary care providers and staff members understood that unique aspects of their clinical settings were taken into account in the design of the programme. Additionally, the primary care physician was able to share her own experiences having used the BHC, which was helpful in obtaining buy-in and addressing primary care-specific concerns.

Separate trainings were provided for primary care physicians and clinic staff, but all trainings shared the same five core elements as presented in table 1.

Table 1

Healthcare provider trainings

Training consisted of meetings during which the BHC tool was introduced to primary care physicians, medical assistants and other medical staff. Follow-up trainings focused on implementation of the BHC into clinic workflow. Primary care physicians were also provided with trainings on when to refer for behavioural health services and resources for referral. Trainers were prompt in providing guidance in best practices for managing suicidality and other high-risk behaviours depending on clinic and provider need, if such issues arose. Additionally, trainers conducted follow-up appointments with staff at the pilot sites to troubleshoot any difficulties that may have arisen.

Obtaining the buy-in of medical personnel into the importance of including behavioural health outcomes in assessing overall patient health was essential. While one key component to achieving this was to provide staff and physicians with psychoeducation on the prevalence and impact that behavioural health issues have in the primary care setting, another important piece was that the utilisation of the behavioural health assessment tool was included as one of the department’s quality improvement goals. That is, clinics and office managers were incentivised with quarterly financial bonuses if target screening rates of 60% were achieved.


To examine whether the introduction of the BHC and associated implementation strategies (new clinic workflows, staff and provider trainings, performance incentives for clinic managers) was associated with increased screening rates, we compared rates of screening from January through June of 2017 (when behavioural health screening data were limited to a paper administration of the PHQ-9) to rates of screening from January through June of 2018 (subsequent to implementation). We used the same period (January to June) in both years to control for the seasonality of primary care visits. PHQ-2 screening rate is defined as the number of unique adult (≥18 years) patients screened with the PHQ-2 divided by the number of unique adult patients (≥18 years) due for screening during the time period. The PHQ-9 screening rate is defined as the number of unique adult (≥18 years) patients with a positive PHQ-2 who completed the PHQ-9 divided by the number of unique adult patients (≥18 years) with a positive PHQ-2 during the time period.


Initial screening with the PHQ-2

Across both pilot sites, depression screening rates using the PHQ-2 increased from January to June of 2017 (hereafter referred to more simply as 2017) relative to January to June of 2018 (hereafter referred to more simply as 2018). In 2017, 50.5% of patients were screened with the PHQ-2 across both pilot sites (5750 PHQ-2s were administered among 11 381 eligible patients/appointments); in 2018, the rate of screening increased to 57.0% (7303 PHQ-2s were obtained from 12 808 eligible patients/appointments) (χ2=102.37, df=1, p<0.00000000000000022).

Screening with the PHQ-9 and reported symptom severity

In 2017, of the 5750 completed PHQ-2s across both sites, 519 (9.0%) patients endorsed depression symptomatology. Of these 519 patients, only 179 completed the PHQ-9, resulting in a PHQ-9 screening rate of 34.5%. By contrast, in 2018, of the 7303 completed PHQ-2s across both sites, 1243 (17.0%) patients endorsed depression symptomatology. Of these 1243 patients, 1136 completed the PHQ-9, resulting in a PHQ-9 screening rate of 91.4%. That is, from 2017 to 2018 the PHQ-9 screening rate increased from 34.5% to 91.4% (χ2=626.14, df=1, p<0.00000000000000022).

Of the PHQ-9s completed from January to June 2018, 28.6% of patients’ total scores fell in the ‘minimal’ range, 31.1% were in the ‘mild’ range, 30.7% were in the ‘moderate’ range and 9.6% were in the ‘severe’ range. Comparison data are not available for PHQ-9 symptoms in 2017, given inconsistent recording of PHQ-9 results within the EHR during this time period.

Other behavioural health screening

In this same time period, 1168 GAD-7 assessments were completed by patients. Of those completed, 28.2% of scores fell in the ‘minimal’ range, 29.2% fell in the ‘mild’ range, 20.5% fell in the ‘moderate’ range and 22.1% fell in the ‘severe’ range.

From January to June 2018, 1168 AUDIT-C assessments were completed. Of these, 24.7% reported no alcohol-related problems or no alcohol use, 48.5% fell in the ‘mild’ range, 4.9% fell in the ‘moderate’ range and 21.8% fell in the ‘severe’ range.

Lastly, as described above, patients were assessed for substance use with this initial screening question: ‘In the last year, have you used an illegal drug or prescription medication for non-medical reasons?’ If the patient answered with ‘yes’, the patient was then prompted to the complete the DAST-10. From January to June 2018, 96 DAST-10 assessments were completed. Of these, 8.3% reported no issues with substance use, 60.4% reported ‘mild’ problems, 26.0% reported ‘moderate’ problems and 5.2% reported ‘severe’ problems.


Due to the high prevalence of patients presenting to primary care clinics with behavioural health concerns, many health systems are moving towards an integrated care system in order to more effectively identify and provide care to these patients in need. One critical component to building an efficient system for identifying those with behavioural health concerns is the integration of quantitative measures to assess and track patient symptomatology and progress. Despite this, efforts for universal screening remain suboptimal. In this paper, we demonstrated how using an innovative health information technology, associated with an improved workflow, training staff and providers and use of performance incentives for clinic managers increased rates of screening and assessment of patients within two primary care clinics in the university health system. Furthermore, screening demonstrated significant need for follow-up of symptoms by primary care providers, and potential need for referral to behavioural health treatment.

In the time period examined in 2017 versus 2018, the rate of verbal PHQ-2 screening by a medical assistant increased from 50.5% to 57.0% in the two primary care clinics that participated in this pilot study. While this increase is promising, much room for improvement remains. Because completion of the BHC in its entirety is dependent on a patient having initially completed the PHQ-2, the PHQ-2 serves as the ‘gate’ after which a more comprehensive behavioural health screening can be conducted. That is, patients who are not administered the PHQ-2 will not be screened for other depressive symptoms or mental health conditions (eg, anxiety). Thus, increasing the rate of PHQ-2 administration will remain a critical target for continued quality improvement. In fact, an internal survey of primary care medical assistants administered in January 2018 found that time commitment and patient participation were often cited by the university health system primary care staff as the biggest obstacles to universal screening in the primary care setting. These survey results highlight the need for increased efforts at providing additional training of medical staff to further highlight the importance of behavioural health assessment and its role in conceptualising patient health more holistically. Alternatively, it may indicate that PHQ-2 screening needs to be further streamlined into the typical workflow such that it minimises disruptions.

Improving the workflow between administration of the PHQ-2 and PHQ-9 (when indicated by the patient’s positive responses on one of the two PHQ-2 questions) was one of the key targets in implementation of the BHC. In this context, administration of the PHQ-9 showed improvement (from 34.5% to 91.4%). This result is very promising and indicates that the development of our cloud-based platform for gathering assessment data greatly improved administration of behavioural health screening, reducing the burden on medical staff needing to track paper assessment measures and manually input the data into patients’ health records. Electronic assessment also provides an opportunity for increased accountability, as failing to administer additional screening when it is clinically indicated can be more easily tracked. Screening rates for the GAD-7 and AUDIT-C were encouraging as well, with rates nearing that of depression screening. Discrepancies in PHQ-9, GAD-7 and AUDIT-C rates are likely due to instances in which patients do not complete the entire BHC when administered, as the PHQ-9 is the first rating scale presented on the platform.

The implementation of financial bonuses for the clinics and key clinic personnel likely contributed to the significant improvement in screening rates as well. It is difficult to determine the extent to which these bonuses served as an incentive relative to the trainings and other strategies that were employed in our quality improvement intervention. With the increasing call for including quantitative assessment in medical care, it is possible that other health systems would be able to implement a similar incentive as well.

Behavioural health symptomatology in this primary care setting

Results of the convenience sample in this pilot revealed high rates of behavioural health symptomatology, underscoring the need for behavioural health screening in primary care settings. Furthermore, they suggest that primary care providers must be equipped to assess and treat behavioural health symptoms and provide appropriate referrals as needed. Of those who screened positive on the PHQ-2 and were gated into the PHQ-9, 40.3% of patients fell in the ‘moderate’ to ‘severe’ range, suggesting need for further care or a referral to appropriate services. Similarly, on the GAD-7, 42.6% of patients scored in the ‘moderate’ to ‘severe’ range. The pattern of alcohol use reported by patients was similarly alarming, with more than one out of four patients reporting ‘moderate’ or ‘severe’ usage or problems associated with alcohol usage. Of those patients endorsing having used illegal drugs or medications for non-medical reasons, almost one in three patients reported ‘moderate’ to ‘severe’ problems associated with use. This further highlights the necessity of regularly screening patients in this setting, as it presents an invaluable opportunity to assess patients and facilitate appropriate follow-up care, particularly for those presenting with high-risk symptoms, in order to enhance overall health and reduce costly late-stage complications. While the development, training and implementation of the BHC was time and labour intensive, these results highlight the critical importance of strategies targeted at behavioural health screening in this setting.

The BHC and associated implementation strategies have effectively streamlined the depression screening process. The digitisation of the process is critical in allowing our health system to capture key health information about our patients and allows for earlier detection and referral for behavioural health issues. Moreover, this digitisation greatly reduces the burden on medical staff, as it is no longer necessary to manually track and enter behavioural health data, aside from the PHQ-2, in patients’ records. In addition to technological and workflow changes, provider trainings reinforced the importance of conducting universal screening to primary care providers and clinical staff. After the implementation, both offices saw significant increases in PHQ-9 completion rates for those screening positive for depression initially on the PHQ-2.

The current study does have its limitations. One of the main concerns for participating clinicians in the pilot sites was that including digital PHQ-2/PHQ-9 screening may potentially increase visit time with each patient encounter. Unfortunately, we could not study this aspect of the patients’ care quantitatively, as there was no consistent way of documenting duration of each visit prior to implementation of this project. Anecdotally, providers in each pilot site overall found positive experiences with the new workflow. Additionally, clinic manager incentives were put in place to help improve staff engagement. We are unable to determine the extent to which financial incentives affected the rates at which the tool was used. At the outset, institution of financial incentives for clinic managers was conceptualised as an important component to the initiative. Importantly, the clinical results of the behavioural health screening do not reflect a true epidemiologic rate and are subject to potential selection bias, given the patients in this pilot were able to access and agreed to use the BHC platform during their primary care visit. Lastly, data on patients’ treatment plans and clinical outcomes following identification on the screening measures were not analysed in the present study; future work should examine this.

Following the experiences of these two pilot sites, we reviewed the data and conducted further workflow analyses to improve on our processes. We created a dashboard where real-time screening data from each office can be monitored closely, whereby allowing for constant reassessment and readjustment of the workflow. We envision many more clinical endeavours and quality improvement efforts to emerge as a result of this new clinical workflow including, but not limited to, further studies that validate our findings in the two initial pilot sites and expansion of the number of available assessment measures to include other domains and symptoms (eg, Attention-deficit/hyperactivity disorder (ADHD)). The university health system also plans to implement behavioural health screening in paediatric clinics. Additionally, future work may include the tracking of depression screening rates of individual primary care offices and identifying the rates of emergency department admission for patients who endorse suicidality on the assessment tool.



  • Contributors Dr Jeffrey, Dr Do, Dr Lin, Dr Hajal, Dr Grossman, Dr Lester and Ms Linonis conceived of the presented idea. Dr Jeffrey, Dr Do and Dr Hajal and Ms Linonis verified the analytical methods. All authors discussed the results and contributed to the final manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement No data are available.