Improving appropriate imaging for non-specific low back pain
============================================================

* Eyad Al-hihi
* Cheryl Gibson
* Jaehoon Lee
* Rebecca R Mount
* Neville Irani
* Caylin McGowan

## Abstract

Non-specific low back pain (LBP) is a common condition in the USA, with approximately 80% of adults who will have LBP at some point during their life and roughly 30% of the adult population suffering from LBP at any given time. Although LBP is the most common cause of disability in the USA, it often has no identifiable anatomic or physiologic cause. Many patients seeking care for non-specific LBP receive X-rays and other imaging studies. However, for most acute LBP patients, symptoms resolve within 4 weeks and the use of routine imaging may result in unnecessary radiation exposure and add unnecessary costs and wasted time for patients without contributing to patient outcomes. The specific aim of the quality improvement (QI) project was to determine the effect of a multicomponent intervention to enhance the appropriate imaging utilisation for acute LBP to ≥90%. During the first 6 months of the QI project, 191 patients with LBP were seen. Of those patients, 156 (81.7%) received appropriate imaging over the 6-month intervention period, missing our targeted goal. Furthermore, this rate declined to baseline values after termination of the intervention, suggesting the need for additional prompts to sustain the initial intervention effect. Following a health system-wide deployment of practice-based alerts and quality score cards, the appropriate utilisation rate increased again and quickly to the target rate of 90%. To reduce variability in our clinical practice and to sustain an appropriate utilisation rate will require continued work. Health systems must find efficient methods to reduce LBP imaging and increase appropriate management of non-specific LBP in primary care. Increasing concordance with imaging guidelines can lessen harm associated with unnecessary radiation exposure and result in significant cost savings.

*   back pain
*   decision support
*   computerised
*   quality improvement

## Problem

Low back pain (LBP) is a common reason for physician visits in the ambulatory care setting.1 2 In the initial release of the Choosing Wisely campaign, routine imaging for acute LBP was among the most prominent low-value services cited to avoid because of its relatively high out-of-pocket expense and large volume of physician visits nationally.3 4 Although routine use of imaging for non-specific LBP is not recommended by a number of clinical specialty societies, many physicians continue to order imaging without a clear clinical indication.5 6

At our academic institution, we found that our current rate of appropriate utilisation was not optimal. In response, the primary objective of our mixed-method quality improvement (QI) project was to assess the impact of a multicomponent, clinical decision support (CDS)-enabled intervention designed to increase appropriate utilisation in clinical care and to identify patient expectations and educational needs.

## Background

LBP is a common condition in the USA—approximately 80% of adults have LBP at some point during their life and roughly 30% of the adult population suffer from LBP at any given time.7 8 Although LBP is the most common cause of disability in the USA, it often has no identifiable anatomic or physiologic cause.9–12 Many patients seeking care for LBP receive lumbar spine radiography, CT or MRI.13 14 However, for most patients with acute non-specific LBP, symptoms resolve within 4 weeks and the use of routine imaging may result in unnecessary radiation exposure and add unnecessary costs and wasted time for patients without improved patient outcomes.15–19

National quality guidelines, such as The National Committee of Quality Assurance’s Health Effectiveness Data and Information Set (HEDIS), established the quality measure for use of image studies for LBP in efforts to improve rates of appropriate imaging in the healthcare setting.12 Despite these evidence-based guidelines, significant variability in clinical practice exists, with imaging studies often used to diagnose aetiology of acute non-specific LBP.10

## Measurement

The primary aim of this QI project was to determine the effect of the multicomponent intervention on the use of routine imaging for LBP. The appropriate use of imaging for LBP, as measured by the proportion of patients with LBP with an appropriate utilisation, was compared between preintervention (baseline) and 3-month, 6-month and 12-month postintervention. Patients were excluded if they had a diagnosis for which imaging was clinically appropriate, including cancer, recent trauma, neurologic impairment, HIV, IV drug abuse, spinal infection, prolonged use of corticosteroids or major organ transplant.12 χ2 test was performed for the comparisons and an effect size was calculated for each comparison. All analyses were conducted using SAS (V.9.4).20

According to the previous year’s outpatient data, 488 outpatients, ages 18 to 50 years, with a primary diagnosis of non-specific LBP visited our primary care clinics and of those, 381 had appropriate imaging utilisation for LBP (381/488=78.1%). For this study, we expected to observe ≥90% of appropriate utilisation after the intervention, which corresponds to an effect size of OR ≥1.15 (small). Our power calculation using this effect size estimate and G*Power 3.1.9.221 revealed that a sample of 320 patients will provide ≥80% power to detect a change in appropriate imaging utilisation between baseline and post-intervention at 0.05 alpha level.

## Design

We conducted a 6-month multicomponent QI programme with a 6-month follow-up to increase appropriate imaging utilisation among outpatients, ages 18 to 50 years old, with a primary diagnosis of non-specific LBP.

## Strategy

Prior to implementing our intervention, we performed formative research to help refine and develop strategies to more effectively influence appropriate imaging utilisation. To this end, an online survey was conducted to gauge the willingness of our providers to use CDS tools (eg, best practice alerts (BPAs)) for LBP imaging orders. Results indicated most internal medicine providers found clinical decision aids helpful and they would be willing to use them for LBP imaging orders. Open-ended comments regarding concerns about clinical decision-making tools were analysed to identify prevailing themes. Surveyed internal medicine providers were most sensitive about how BPAs cannot account for all patient factors that might require a bypass of hard stops (ie, a response is required before one can move forward) placed in the workflow. Although several were worried about the degree of end user control, many felt CDS tools can facilitate improved care and provide better alignment with evidence-based clinical guidelines for LBP imaging and curb unnecessary healthcare costs.

Given positive survey responses, we implemented our first phase of the QI project, which involved assessing work order queues. Using the queue as a component of the healthcare workflow, we examined the sequence of tasks forming the patients’ specific care plan. Care plans can be considered personalised instances of a particular healthcare guideline (eg, HEDIS quality measure for use of image studies for LBP).22 A detailed review of the process indicated a lack of a standardised workflow for LBP, and the workflow was not sufficiently integrated into the electronic medical record (EMR, Epic Systems Corporation, Verona, Wisconsin) and mapped to the patients’ care plans. In addition, treatment methods were grouped together (eg, muscle relaxer, X-ray, physical therapy), which did not allow for discrete indications for treatment. Diagnosis coding was also unclear to providers and they had difficulty identifying the appropriate diagnosis codes that address chronic LBP. We also identified a lack of provider education on LBP and limited provider education given to patients.

In response, we sought to build awareness and enhance engagement and buy-in by conducting educational sessions with key ordering provider groups in internal medicine (eg, attending physicians, hospitalists and residents) to review clinical practice variabilities and share information from the Choosing Wisely campaign about avoiding imaging studies for acute non-specific LBP without specific indications.11

During this first phase, all LBP imaging orders were examined to see if they met appropriate HEDIS criteria and to isolate where and why HEDIS quality measures may be lower than desired for acute non-specific LBP imaging orders. During clinic rounding, HEDIS criteria were communicated with providers to improve adherence and documentation.

For our next phase of the study, we implemented changes to the EMR. Prior to implementation, guidance was not provided for image ordering for non-specific LBP. Working with information technology, we developed CDS order sets and built practice alerts and reminders in the EMR to marry the clinical guidelines with our institution’s order entry system that established a standardised workflow for non-specific LBP. A new order panel (see figure 1) was developed that required patients to meet specific criteria to minimise inappropriate imaging and improve guideline-adherent practice. After the history and physical examination of the patient, providers were required to answer a series of questions during the order entry process. Provider responses placed patients presenting with LBP into categories (eg, non-specific LBP; LBP associated with radiculopathy or spinal stenosis; LBP associated with other specific spinal causes) that included discrete indications for X-ray, MRI and CT scans based on evidence-based recommendations for appropriate imaging. As such, diagnostic imaging studies were to be performed only in those patients who have severe or progressive neurologic deficits or were suspected of having a serious or specific underlying condition (eg, cancer, trauma). Access to imaging was denied when providers did not document compliance with institutionally approved indications though providers were allowed to proceed with an override by entering an active acknowledgement for opposing recommendations.

![Figure 1](http://bmjopenquality.bmj.com/https://bmjopenquality.bmj.com/content/bmjqir/11/1/e001539/F1.medium.gif)

[Figure 1](http://bmjopenquality.bmj.com/content/11/1/e001539/F1)

Figure 1 
Order entry for imaging. HCC, Hierarchical Condition Category; AP, anteroposterior; UKHS AMB, University of Kansas Health System Ambulatory.

We also conducted three focus group discussions to (1) elicit patient experiences when seeking treatment for non-specific LBP, (2) identify expectations and educational needs and (3) use this information to inform the content of educational sessions and materials for healthcare professionals and patients. A moderator’s guide was developed to ensure consistency of questions across the focus groups. Topics addressed included pain management, treatment expectations, factors considered when making treatment choices and the kinds of information desired about LBP treatment. Discussions were conducted by a moderator (CAG) with experience in qualitative methods and a note taker (RRM) who was not involved in the discussants’ care management. Eligible participants were age 18 or older, receiving primary care at our institution and had made a patient visit for non-specific LBP within the last year. Patients with LBP and severe or progressive neurologic deficits or serious underlying conditions based on history and physical examination were excluded from the focus groups. All participants provided informed consent and received $25 USD for participation.

Digital recordings were transcribed verbatim and checked for accuracy among our research team members. Data were analysed thematically, allowing for the identification of commonalities across the three groups.23

## Results

During the first 6 months, 191 patients with LBP were seen. Of those patients, 156 (81.7%) received appropriate imaging, missing the targeted goal of 90% or greater. Table 1 displays the number of patients during the intervention and follow-up periods and the number of patients with appropriate imaging across each month.

View this table:
[Table 1](http://bmjopenquality.bmj.com/content/11/1/e001539/T1)

Table 1 
Number of patients with non-specific low back pain

Figure 2 displays the percentage of patients who received appropriate imaging orders for LBP in the 6-month intervention (August 2018 to January 2019) and 6-month follow-up (February to July 2019) periods. At the beginning of the intervention, 73.5% of patients (25 out of 34) received appropriate imaging orders for LBP. The appropriate imaging utilisation gradually improved approaching the target rate of 90% (horizontal dotted line in the figure) by the end of the intervention—81.7% (156 out of 191) over the 6-month intervention period. However, this rate declined back to the baseline rate after termination of the intervention (72.7%, 16 out of 22, in February 2019), suggesting the need for additional prompts to sustain the initial intervention effect. Interestingly, the appropriate utilisation rate increased again and quickly to the target rate (87.5%, 21 out of 24, in March to 90.9%, 20 out of 22, in July 2019), following a health system-wide deployment of practice-based alerts and order sets. Additionally, quality score cards of appropriate LBP imaging were created and shared with each provider to help set performance targets. Performance targets emphasised appropriate documentation and specific International Classification of Diseases, 10th revision (ICD-10) codes for billing or associated imaging diagnoses to reflect chronicity of back pain or presence of radiculopathy, trauma or cancer. The results of χ2 test indicated that the rate was not significantly different between baseline versus 3 and 6 months (p=0.19–0.95, Cramer’s V=0.01–0.16) but marginally different between baseline versus 12 months (p<0.10, Cramer’s V=0.21).

![Figure 2](http://bmjopenquality.bmj.com/https://bmjopenquality.bmj.com/content/bmjqir/11/1/e001539/F2.medium.gif)

[Figure 2](http://bmjopenquality.bmj.com/content/11/1/e001539/F2)

Figure 2 
Rate of appropriate imaging for LBP. CDS, clinical decision support; LBP, low back pain.

### Focus group findings

Each focus group discussion lasted approximately 60 min and included 13 participants (7 women, age 49±13 years). Most discussants indicated that LBP had a significant deleterious effect on or prevented normal daily activities.

Two main interconnected themes were identified that highlighted common experiences across the three group discussions and among focus group discussants. Anonymised quotes that captured the sentiment or shared experiences of the overall groups are presented with assigned participant codes in the following sections.

### Insistence on imaging to diagnose aetiology of their LBP

We found that participants with non-specific LBP want imaging to identify the cause of pain and to seek a definitive diagnosis. Most were troubled and confused about why their physicians were reluctant or refused to order imaging even though the course of treatment would unlikely change based on the results.

> I think that [imaging] should be one of the first things done if a patient comes to their office. Because how can you diagnose what’s wrong or say what’s wrong if you don’t see it on a clear picture?–LBP04, woman
> 
> So, I think when patients walk in with a chief complaint about low back, you can’t treat a patient just because the patient’s coming in saying ‘my back is hurting.’ This is why [we need imaging]. Well, we’re coming to the doctor. You’re the expert. You’re the one that went to school; we didn’t. Regardless of what we tell you, you need to see it on the picture—LBP05, woman
> 
> First thing I come back to is when we sit there and talk about back pain and what someone wants. You got to find [through imaging] specifically what it is [that is causing my LBP]—LBP07, man

Clearly, the participants’ assertions that physicians cannot understand the root cause of their pain without imaging and shared feelings that they are not getting an appropriate evaluation for back pain pose significant challenges to clinicians from following the recommendations in practice.

### Imaging to legitimise seeking pain relief and to avoid appearing as opioid seeking

Focus group discussants shared how they are exasperated about not receiving imaging and do not want to appear to be exaggerating their condition just to convince their physician to prescribe opiates to manage their pain.

> …when you go and tell the doctor, the main problem I’ve had, and I don’t know if this has been for anybody else, is when you go and tell the doctor ’Hey this ain’t working for me, is there any way I could get some pain meds, you know, just to get me through it until next time I see you?’ Nope. I’m like dude, I’m just, I’m not trying to seek drugs, but just to get to get me through it ‘cause I don’t see you for two weeks… —LBP01, man
> 
> Something I am afraid, I have a fear of especially when I hear about the opioid epidemic, is that someone might be thinking that I’m chasing [opioids]…but with the back part, if we’re talking about lower back pain, it’s I’m in pain, I want to take the pill, I want to take a little four hour vacation, let me be. I’m not going to sit there and try to be a hustler on it…—LBP07, man
> 
> …when you’re hurting, you’re focused on relieving as much as you can, as quickly and usually as safely, as you can…I expected they [the physician] would take care of it. I expected to get to leave [the physician’s office] better than I left, than I came—LBP08, man

### Lessons and limitations

The strength of this study is that it sought information from different sources to understand both facilitators and barriers to aligning our clinical care to imaging guidelines. Information gathered by survey was anonymous, allowing for candid responses by physicians. Focus group discussants shared their views about imaging and provided insights into the challenges patients and physicians face when addressing imaging for non-specific LBP. As others have found, patient expectations are important barriers to reducing overutilisation of imaging24 25 and have prompted the development of patient-oriented interventions to support patient–provider conversations and shared decision-making that could support reductions in unnecessary LBP imaging.26

To help reduce overutilisation despite the insistence on imaging, information gathered from the focus group discussions was shared with our providers to help them understand patient expectations and to stress how a decision to not obtain imaging might be related to less patient satisfaction and to perceptions that their patient’s pain will not be legitimised. Similar to other investigators, our providers were encouraged to provide brief education to reassure patients that they have been appropriately assessed and their problems were not being dismissed.27 Although we identified patient barriers to appropriate imaging, additional educational efforts are needed to bring patient expectations in line with clinical evidence, such as the use of handouts and online materials.28 29

Limitations of this QI project include its short duration that placed significant dependence on EMRs to improve appropriate utilisation of imaging. After 6 months of implementation, alert fatigue effects became common with many providers ignoring electronic prompts and overriding the CDS system. Similarly, other researchers have found significant overrides of CDS systems, ranging from 49% to 96% of alerts overridden due to alert fatigue.30–32 Because of the sheer volume of alerts received by physicians,33 modifying clinical decision tools and care pathways to provide the latest evidence and addressing efficient clinician workflow and patient education are needed.34 35

Postintervention, when performance targets were shared with providers who emphasised appropriate documentation and use of ICD-10 codes, the appropriate utilisation rate increased to the targeted goal. However, a lack of documentation with the associated ICD-10 billing codes to differentiate between acute LBP and chronic LBP has led to difficulties capturing accurate estimates of appropriate utilisation of imaging.

## Conclusion

Continued work is required to reduce variability in our clinical practice and to sustain an appropriate utilisation rate. Accurate documentation with associated ICD-10 diagnoses is essential for correct claims data to determine appropriate utilisation of imaging for LBP.36 Furthermore, ongoing patient education is necessary to minimise inappropriate imaging without affecting quality of care, safety or patient satisfaction. As others have found, our focus group discussants shared expectations for imaging to diagnose the cause of their LBP, even when it was not indicated.37 Similarly, in a systematic scoping review, investigators also found patients placed much belief in the notion that imaging would help to diagnose their problem and guide treatment options.38 Moreover, they wanted imaging to justify their pain and avoid appearing as a drug seeker. Backache is one of the common complaints used by drug seekers as the underlying aetiology for LBP pain is often difficult to verify objectively.39

Although others have found that strategies for reducing imaging overuse should be multifactorial, addressing not only provider behaviours but also the expectations and education of patients and financial incentives based on how well clinicians adhere to guidelines,40 sustaining these strategies are particularly challenging. Health systems must find methods to reduce LBP imaging and increase appropriate management of non-specific LBP in primary care that are sustainable.

## Data availability statement

Data are available upon request.

## Ethics statements

### Patient consent for publication

Not required.

### Ethics approval

This study involves human participants and was approved by the University of Kansas Medical Center Institutional Review board, ID STUDY00142577. Participants gave informed consent to participate in the study before taking part.

## Acknowledgments

We appreciate Bethany Ramm and Joy Jacobsen for their assistance in pulling needed information for the QI project.

## Footnotes

*   Contributors EA, NI and CG designed the overall study. CG, RRM and NI executed the interventions and contributed to data collection. JL analysed the data and participated in developing tables and figures. All authors participated in drafting and revising the manuscript. All authors reviewed and approved of the final version.

*   Funding This research was funded by Blue Cross and Blue Shield of Kansas City, one of the Blue KC Health Outcomes Research grants. Grant number KCALSI 18-02.

*   Competing interests None declared.

*   Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

*   Provenance and peer review Not commissioned; externally peer reviewed.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  Hsiao CJ, Cherry DK, Beatty PC, et al. National ambulatory medical care survey: 2007 summary. Natl Health Stat Report 2010;27:1–32.
    
    

2.  Balagué F, Mannion AF, Pellisé F, et al. Non-Specific low back pain. Lancet 2012;379:482–91.[doi:10.1016/S0140-6736(11)60610-7](http://dx.doi.org/10.1016/S0140-6736(11)60610-7)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21982256
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(11)60610-7&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=21982256&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000300187800038&link_type=ISI) 

3.  ABIM Foundation. Choosing wisely: five things physicians and patients should question, 2012.
    
    

4.  Hong AS, Ross-Degnan D, Zhang F, et al. Small Decline In Low-Value Back Imaging Associated With The 'Choosing Wisely' Campaign, 2012-14. Health Aff 2017;36:671–9.[doi:10.1377/hlthaff.2016.1263](http://dx.doi.org/10.1377/hlthaff.2016.1263)pmid:http://www.ncbi.nlm.nih.gov/pubmed/28373333
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjM2LzQvNjcxIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpxaXIvMTEvMS9lMDAxNTM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

5.  Colla CH, Morden NE, Sequist TD, et al. Choosing wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med 2015;30:221–8.[doi:10.1007/s11606-014-3070-z](http://dx.doi.org/10.1007/s11606-014-3070-z)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25373832
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1007/s11606-014-3070-z&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=25373832&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

6.  Schwartz AL, Landon BE, Elshaug AG, et al. Measuring low-value care in Medicare. JAMA Intern Med 2014;174:1067–76.[doi:10.1001/jamainternmed.2014.1541](http://dx.doi.org/10.1001/jamainternmed.2014.1541)pmid:http://www.ncbi.nlm.nih.gov/pubmed/24819824
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

7.  Freburger JK, Holmes GM, Agans RP, et al. The rising prevalence of chronic low back pain. Arch Intern Med 2009;169:251–8.[doi:10.1001/archinternmed.2008.543](http://dx.doi.org/10.1001/archinternmed.2008.543)pmid:http://www.ncbi.nlm.nih.gov/pubmed/19204216
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1001/archinternmed.2008.543&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=19204216&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000263202600007&link_type=ISI) 

8.  Murray CJL, Lopez AD. Measuring the global burden of disease. N Engl J Med 2013;369:448–57.[doi:10.1056/NEJMra1201534](http://dx.doi.org/10.1056/NEJMra1201534)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23902484
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1056/NEJMra1201534&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=23902484&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000322547400011&link_type=ISI) 

9.  Chou R, Qaseem A, Owens DK, et al. Diagnostic imaging for low back pain: advice for high-value health care from the American College of physicians. Ann Intern Med 2011;154:181–9.[doi:10.7326/0003-4819-154-3-201102010-00008](http://dx.doi.org/10.7326/0003-4819-154-3-201102010-00008)pmid:http://www.ncbi.nlm.nih.gov/pubmed/21282698
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-154-3-201102010-00008&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=21282698&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000286729200006&link_type=ISI) 

10. Gold R, Esterberg E, Hollombe C, et al. Low back imaging when not indicated: a descriptive Cross-System analysis. Perm J 2016;20:25–33.[doi:10.7812/TPP/15-081](http://dx.doi.org/10.7812/TPP/15-081)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26934626
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

11. American College of Physicians. Imaging for low back pain. Available: [http://www.choosingwisely.org/clinican-lists/american-college-physicians-imaging-for-non-specific-low-back-pain/](http://www.choosingwisely.org/clinican-lists/american-college-physicians-imaging-for-non-specific-low-back-pain/)
    
    

12. National Committee of Quality Assurance (NCQA). Use of imaging studies for low back pain 2017.
    
    

13. Srinivas SV, Deyo RA, Berger ZD. Application of “less is more” to low back pain. Arch Intern Med 2012;172:1016–20.[doi:10.1001/archinternmed.2012.1838](http://dx.doi.org/10.1001/archinternmed.2012.1838)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22664775
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=22664775&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000306189500009&link_type=ISI) 

14. Mathias JS, Feinglass J, Baker DW. Variations in US hospital performance on imaging-use measures. Med Care 2012;50:808–14.[doi:10.1097/MLR.0b013e31825a8c48](http://dx.doi.org/10.1097/MLR.0b013e31825a8c48)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22643196
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1097/MLR.0b013e31825a8c48&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=22643196&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000307869900009&link_type=ISI) 

15. Childs JD, Fritz JM, Wu SS, et al. Implications of early and guideline adherent physical therapy for low back pain on utilization and costs. BMC Health Serv Res 2015;15:150. [doi:10.1186/s12913-015-0830-3](http://dx.doi.org/10.1186/s12913-015-0830-3)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25880898
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1186/s12913-015-0830-3&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=25880898&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

16. Mafi JN, McCarthy EP, Davis RB, et al. Worsening trends in the management and treatment of back pain. JAMA Intern Med 2013;173:1573–81.[doi:10.1001/jamainternmed.2013.8992](http://dx.doi.org/10.1001/jamainternmed.2013.8992)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23896698
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

17. Keyhani S, Falk R, Howell EA, et al. Overuse and systems of care: a systematic review. Med Care 2013;51:503–8.[doi:10.1097/MLR.0b013e31828dbafe](http://dx.doi.org/10.1097/MLR.0b013e31828dbafe)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23552430
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1097/MLR.0b013e31828dbafe&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=23552430&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000319045800006&link_type=ISI) 

18. Graves JM, Fulton-Kehoe D, Jarvik JG, et al. Early imaging for acute low back pain: one-year health and disability outcomes among Washington state workers. Spine 2012;37:1617–27.[doi:10.1097/BRS.0b013e318251887b](http://dx.doi.org/10.1097/BRS.0b013e318251887b)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22415000
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=22415000&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

19. Fazel R, Krumholz HM, Wang Y, et al. Exposure to low-dose ionizing radiation from medical imaging procedures. N Engl J Med 2009;361:849–57.[doi:10.1056/NEJMoa0901249](http://dx.doi.org/10.1056/NEJMoa0901249)pmid:http://www.ncbi.nlm.nih.gov/pubmed/19710483
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1056/NEJMoa0901249&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=19710483&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000269312700005&link_type=ISI) 

20. SAS Institute. SAS/STAT 9.4 user’s guide. Cary, NC: SAS Institute, Inc, 2002-2012.
    
    

21. Faul F, Erdfelder E, Lang A-G, et al. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39:175–91.[doi:10.3758/BF03193146](http://dx.doi.org/10.3758/BF03193146)pmid:http://www.ncbi.nlm.nih.gov/pubmed/17695343
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.3758/BRM.41.4.1149&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=17695343&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=000248501300003&link_type=ISI) 

22. Meli CL, Khalil I, Tari Z. Load-sensitive dynamic workflow re-orchestration and optimisation for faster patient healthcare. Comput Methods Programs Biomed 2014;113:1–14.[doi:10.1016/j.cmpb.2013.06.019](http://dx.doi.org/10.1016/j.cmpb.2013.06.019)pmid:http://www.ncbi.nlm.nih.gov/pubmed/24099624
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

23. Braun V, Clarke V. What can "thematic analysis" offer health and wellbeing researchers? Int J Qual Stud Health Well-being 2014;9:26152. [doi:10.3402/qhw.v9.26152](http://dx.doi.org/10.3402/qhw.v9.26152)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25326092
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.3402/qhw.v9.26152&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

24. Buist DSM, Chang E, Handley M, et al. Primary care clinicians' perspectives on reducing low-value care in an integrated delivery system. Perm J 2016;20:41–6.[doi:10.7812/TPP/15-086](http://dx.doi.org/10.7812/TPP/15-086)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26562308
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

25. Zikmund-Fisher BJ, Kullgren JT, Fagerlin A, et al. Perceived Barriers to Implementing Individual Choosing Wisely® Recommendations in Two National Surveys of Primary Care Providers. J Gen Intern Med 2017;32:210–7.[doi:10.1007/s11606-016-3853-5](http://dx.doi.org/10.1007/s11606-016-3853-5)pmid:http://www.ncbi.nlm.nih.gov/pubmed/27599491
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1007/s11606-016-3853-5&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=27599491&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

26. Madani Larijani M, Dumba C, Thiessen H, et al. Development of a patient-oriented intervention to support patient-provider conversations about unnecessary lower back pain imaging. Int J Environ Res Public Health 2021;18:2786. [doi:10.3390/ijerph18052786](http://dx.doi.org/10.3390/ijerph18052786)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33803431
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

27. Deyo RA, Diehl AK, Rosenthal M. Reducing roentgenography use. can patient expectations be altered? Arch Intern Med 1987;147:141–5.[doi:10.1001/archinte.147.1.141](http://dx.doi.org/10.1001/archinte.147.1.141)pmid:http://www.ncbi.nlm.nih.gov/pubmed/2948466
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1001/archinte.1987.00370010139029&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=2948466&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 
    
    [Web of Science](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=A1987F524600023&link_type=ISI) 

28. Diagnostic imaging for low back pain: advice for high-value health care from the American College of physicians 2011;154:181–9.
    
    

29. Burton AK, Waddell G, Tillotson KM, et al. Information and advice to patients with back pain can have a positive effect. Spine 1999;24:2484. [doi:10.1097/00007632-199912010-00010](http://dx.doi.org/10.1097/00007632-199912010-00010)
    
    

30. van der Sijs H, Aarts J, Vulto A, et al. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006;13:138–47.[doi:10.1197/jamia.M1809](http://dx.doi.org/10.1197/jamia.M1809)pmid:http://www.ncbi.nlm.nih.gov/pubmed/16357358
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1197/jamia.M1809&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=16357358&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

31. Carspecken CW, Sharek PJ, Longhurst C, et al. A clinical case of electronic health record drug alert fatigue: consequences for patient outcome. Pediatrics 2013;131:e1970–3.[doi:10.1542/peds.2012-3252](http://dx.doi.org/10.1542/peds.2012-3252)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23713099
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTE6IjEzMS82L2UxOTcwIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpxaXIvMTEvMS9lMDAxNTM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

32. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104–12.[doi:10.1197/jamia.M1471](http://dx.doi.org/10.1197/jamia.M1471)pmid:http://www.ncbi.nlm.nih.gov/pubmed/14633936
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1197/jamia.M1471&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=14633936&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

33. Drew BJ, Harris P, Zègre-Hemsey JK, et al. Insights into the problem of alarm fatigue with physiologic monitor devices: a comprehensive observational study of consecutive intensive care unit patients. PLoS One 2014;9:e110274. [doi:10.1371/journal.pone.0110274](http://dx.doi.org/10.1371/journal.pone.0110274)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25338067
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1371/journal.pone.0110274&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=25338067&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

34. Kesselheim AS, Cresswell K, Phansalkar S, et al. Clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation. Health Aff 2011;30:2310–7.[doi:10.1377/hlthaff.2010.1111](http://dx.doi.org/10.1377/hlthaff.2010.1111)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22147858
    
    [Abstract/FREE Full Text](http://bmjopenquality.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjEwOiIzMC8xMi8yMzEwIjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpxaXIvMTEvMS9lMDAxNTM5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

35. Wan PK, Satybaldy A, Huang L, et al. Reducing alert fatigue by sharing low-level alerts with patients and enhancing collaborative decision making using Blockchain technology: Scoping review and proposed framework (MedAlert). J Med Internet Res 2020;22:e22013. [doi:10.2196/22013](http://dx.doi.org/10.2196/22013)pmid:http://www.ncbi.nlm.nih.gov/pubmed/33112253
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

36. Miotto R, Percha BL, Glicksberg BS, et al. Identifying acute low back pain episodes in primary care practice from clinical notes: observational study. JMIR Med Inform 2020;8:e16878. [doi:10.2196/16878](http://dx.doi.org/10.2196/16878)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32130159
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

37. Lim YZ, Chou L, Au RT, et al. People with low back pain want clear, consistent and personalised information on prognosis, treatment options and self-management strategies: a systematic review. J Physiother 2019;65:124–35.[doi:10.1016/j.jphys.2019.05.010](http://dx.doi.org/10.1016/j.jphys.2019.05.010)pmid:http://www.ncbi.nlm.nih.gov/pubmed/31227280
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1016/j.jphys.2019.05.010&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

38. Taylor S, Bishop A. Patient and public beliefs about the role of imaging in the management of non-specific low back pain: a scoping review. Physiotherapy 2020;107:224–33.[doi:10.1016/j.physio.2019.08.014](http://dx.doi.org/10.1016/j.physio.2019.08.014)pmid:http://www.ncbi.nlm.nih.gov/pubmed/32026824
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

39. Grover CA, Elder JW, Close RJ, et al. How Frequently are "Classic" Drug-Seeking Behaviors Used by Drug-Seeking Patients in the Emergency Department? West J Emerg Med 2012;13:416–21.[doi:10.5811/westjem.2012.4.11600](http://dx.doi.org/10.5811/westjem.2012.4.11600)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23359650
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=23359650&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom) 

40. Chou R, Deyo RA, Jarvik JG. Appropriate use of lumbar imaging for evaluation of low back pain. Radiol Clin North Am 2012;50:569–85.[doi:10.1016/j.rcl.2012.04.005](http://dx.doi.org/10.1016/j.rcl.2012.04.005)pmid:http://www.ncbi.nlm.nih.gov/pubmed/22643385
    
    [CrossRef](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=10.1016/j.rcl.2012.04.005&link_type=DOI) 
    
    [PubMed](http://bmjopenquality.bmj.com/lookup/external-ref?access_num=22643385&link_type=MED&atom=%2Fbmjqir%2F11%2F1%2Fe001539.atom)